best file system for storage?

[fubuu11]

so i want to know the best fs out there that is best for anime or general archiving...
i am not really knowledgeable at fs's so explain them in simplies form possible xD

[Slysoft]

ntfs cause fat can't store files > 4gb so you can't put any large movies on it.

[vuzedome]

Just stick with NTFS, can't go wrong with that, as long as you're using Windows that is.

[Freedom Kira]

ZFS is supposed to be one of the best filesystems you can use. Unfortunately, its usability is very limited, since Windows does not support it, which kicks out over 80% of all computers on the planet. ZFS supports data integrity protection, continuous checking and automatic repair of bad data, and its own RAID implementation, among other interesting features.

Personally, my 12.7TB volume is formatted to ext4. It's decent, and while it does not match ZFS, it is a lot more advanced than anything Windows supports and its own predecessors (ext3 and below).

If you're using Windows, format to NTFS if you need pre-SP2 XP support, or exFAT if you're on XP SP2 and beyond (and nothing else). NTFS is a very old filesystem that should have been obsolete years ago, but you don't have much other choice on Windows. exFAT was introduced during Windows 7's time and is expected to replace FAT32. It has numerous advantages over NTFS, but like NTFS, it is a Microsoft file system and Microsoft does not like releasing information about it. So, if you use exFAT, you can expect any data on the filesystem to only be readable on Windows machines, at least for several years.

[Xtras]

Not entirely correct. exFAT is functional for read and write on Mac OS (snow leopard onward) and Linux (along with Windows). Its typically your best bet if you are talking universal compatibility, as the only problems you run into are with old machines.

I use NTFS though, since I don't really run into Macs.

[nstgc]

AS said already, ZFS. If I weren't so lazy I would use it myself, but currently I use ext3. I've heard its actually more secure than ext4, although much MUCH slower.

[kitamesume]

redundant storage = raid1
simple storage = NTFS or exFAT
other reasons = ZFS or ext4

anyway, why bother when you aren't gonna be stepping outside of a terrabyte of storage, only start considering once you have over 3TB worth of files.
if you only have 2HDDs or less you could just literally mirror it for redundancy and wouldn't bother with anything else.

[halfelite]

My 20tb box is running ext4 but its not for the faint as the tool for ext4 did not support over 16tb out of the box without recompiling them with the correct flags set. I plan on moving over to ZFS on my next big build. The only issue with zfs is you lose a lot of drive space, if you run raidz each dev gets a drive taken away and its recommend to only keep 4 or so drives in a dev

[ConsiderPhlebas]

redundant storage = raid1
simple storage = NTFS or exFAT
other reasons = ZFS or ext4

Uh, I may be really ignorant, but AFAIK RAID isn't a file system? You still have to choose that...

I just use NTFS, haven't really had any issues. I know it's not state of the art, but good enough for what I do. And I archive on DVD anyway. Real old-fashioned guy, heh 

[buchno]

Uh, I may be really ignorant, but AFAIK RAID isn't a file system? You still have to choose that...

That is correct.

I'd use ext4 if I had some place to store all my stuff in between changing from ntfs...

[kitamesume]

redundant storage = raid1
simple storage = NTFS or exFAT
other reasons = ZFS or ext4

Uh, I may be really ignorant, but AFAIK RAID isn't a file system? You still have to choose that...

I just use NTFS, haven't really had any issues. I know it's not state of the art, but good enough for what I do. And I archive on DVD anyway. Real old-fashioned guy, heh 

yes, but anything under raid1 wouldn't make it better though, so yeah slap anything under raid1 is what i meant.

[ConsiderPhlebas]

yes, but anything under raid1 wouldn't make it better though, so yeah slap anything under raid1 is what i meant.

Well, I understand what you mean, that RAID is a good protection for file corruption, but if your file system messes up, it does not matter if you use non-RAID, RAID 1, or RAID 5; it's not the same level of technology. RAID protects against hardware failure, a disk going bad, while good file systems will protect you against other types of corruption, like power fails during write, etc. But of course there is some overlap.

[kitamesume]

but last time i checked, except the old fat file system though, have good enough managing for the other things to matter.

what matters on file storage is sheer capacity on top of hardware longevity, nothing else would matter really.
power outage during write times is another problem and not exactly a problem with a file storage, and you should have an UPS already imho.

[ConsiderPhlebas]

but last time i checked, except the old fat file system though, have good enough managing for the other things to matter.

what matters on file storage is sheer capacity on top of hardware longevity, nothing else would matter really.
power outage during write times is another problem and not exactly a problem with a file storage, and you should have an UPS already imho.

I'm not quite sure I follow. The OP's question was about what file systems that are good for archiving. Thus, if they are safe. Capacity should not be an issue; as far as I know no file system has ever been so bad that it uses a lot of unnecessary space. What a good file system does, is that it helps protect against "soft" errors, i.e., data corruption due to timing errors and similar. Programs opening files for for transactions and messing things up, or the PC being shut down in the middle of things. Completely different from HW fails, which nothing but storage redundance can protect you from.

A UPS may be nice, but it still will not protect you against a poor file system mishandling a half-assed file transaction from a frozen program or PC.

Overall, I still think most modern file systems and OS will handle this decently, though 

[megido-rev.M]

The OS will need to handle problems in case of incomplete writes.

[Saras]

Wasn't exFAT designed purely with flash/external memory in mind? At least that's how I remember it.

[kitamesume]

but last time i checked, except the old fat file system though, have good enough managing for the other things to matter.

what matters on file storage is sheer capacity on top of hardware longevity, nothing else would matter really.
power outage during write times is another problem and not exactly a problem with a file storage, and you should have an UPS already imho.

I'm not quite sure I follow. The OP's question was about what file systems that are good for archiving. Thus, if they are safe. Capacity should not be an issue; as far as I know no file system has ever been so bad that it uses a lot of unnecessary space. What a good file system does, is that it helps protect against "soft" errors, i.e., data corruption due to timing errors and similar. Programs opening files for for transactions and messing things up, or the PC being shut down in the middle of things. Completely different from HW fails, which nothing but storage redundance can protect you from.

A UPS may be nice, but it still will not protect you against a poor file system mishandling a half-assed file transaction from a frozen program or PC.

Overall, I still think most modern file systems and OS will handle this decently, though 

i did say that, which you should have seen in the first line of my post.

but last time i checked, except the old fat file system though, have good enough managing for the other things to matter.

evidently, if recent file systems were as unreliable as they say, i would've had much trouble by now but i have yet to stumble upon heavy errors during copy-paste sequence, TeraCopy also helped in this reliable file transfers.

[Freedom Kira]

Wasn't exFAT designed purely with flash/external memory in mind? At least that's how I remember it.

Yes, that is true. However, considering the extremely large filesystem size and file size limits on ex-FAT, I would not say it was designed only for external media.

evidently, if recent file systems were as unreliable as they say, i would've had much trouble by now but i have yet to stumble upon heavy errors during copy-paste sequence, TeraCopy also helped in this reliable file transfers.

You're not really getting the point. No one said any filesystems were unreliable. Any unreliable filesystem would inevitably cease to exist. The benefits of advanced filesystems are, as said already, protection against soft errors. Copying and pasting, no matter how much data you are transferring, has an extremely low chance of such errors, and any errors encountered would be single bits at a time, which most copy-paste operations could fix with a simple CRC check (note that this is completely irrelevant to the filesystem).

You can get write errors from all kinds of file operations. One that Phlebas put as an example is a "half-assed file transaction from a frozen program or PC." Imagine that a program has opened a file stream for writing and ended up hanging partway through a write operation. Then, perhaps the program is terminated by the user before it is able to close that file stream. That half-written data sits there, and conventional filesystems like FAT or NTFS don't do anything about it. They might not even let you access the data, since it's write-locked to a file stream that never closed.

Other situations can happen as well. For example, imagine that a disk's cache has some dirty pages that have yet to be flushed to disk, or something is currently being written to the disk or cache. Suddenly, the power goes out. What happens to that data?

Also, keep in mind that data does degrade over time. No media can keep its data in perfect form forever. That's what data checks and repair are for.

Overall, I still think most modern file systems and OS will handle this decently, though 

Keyword "modern." NTFS was created in 1993 (most recent revision in 2001), and FAT32, the newest version of FAT before ex-FAT (ignoring FATX), was released in 1996. The oldest version of FAT, FAT12, was released in 1980.

[kitamesume]

uhh... you said all that, but what you quoted already stated it.

your TL;DR means other file system supports "etc.".

but most file system already has the minimum features for errors to be ever so frequent, as stated.
and no, power outage isn't the system's fault, its a "disaster" outside of normal operations.

so whats the verdict? every file system should be adequate enough for other file system to matter, you'll only need to consider a superior one once you're talking over 3terrabytes of storage, where redundancy becomes too expensive to implement.

edit: theres also an advantage of redundant storage, either raid1 or manual backups, it ensures that files have the same consistency over two drives, specially raid1 where as a mis-write on one disk would be rewritten or rebuilt.
if, i mean IF, ever the written file gets mis-written on both drives, thats not the storage's problem, thats the OS or whatever's problem, TeraCopy or any other file manager with a CRC check built into it should solve this though.

[buchno]

I've read through what you've written several times, and I'm still not sure what you're trying to convey...

and no, power outage isn't the system's fault, its a "disaster" outside of normal operations.

Of course it's not the system's fault, however, the risk for data corruption because of that "disaster" depends on which file system you use.

so whats the verdict? every file system should be adequate enough for other file system to matter, you'll only need to consider a superior one once you're talking over 3terrabytes of storage, where redundancy becomes too expensive to implement.

3TB? FAT32 is out of the question for anyone who wants to store files over 4GB (which often occurs a long time before 3TB of total storage) or wants file permissions. FAT32 and Ext2 are not journaling file systems, which increases the risk for data corruption significantly (although the performance is better, which makes them a good choice for small USB flash drives). There are multitudes of differences in useful features between today's file systems, which you can read about here or here.