Discussion Forums > Technology
best file system for storage?
Saras:
Wasn't exFAT designed purely with flash/external memory in mind? At least that's how I remember it.
kitamesume:
--- Quote from: ConsiderPhlebas on November 04, 2012, 12:03:33 AM ---
--- Quote from: kitamesume on November 03, 2012, 11:36:24 PM ---but last time i checked, except the old fat file system though, have good enough managing for the other things to matter.
what matters on file storage is sheer capacity on top of hardware longevity, nothing else would matter really.
power outage during write times is another problem and not exactly a problem with a file storage, and you should have an UPS already imho.
--- End quote ---
I'm not quite sure I follow. The OP's question was about what file systems that are good for archiving. Thus, if they are safe. Capacity should not be an issue; as far as I know no file system has ever been so bad that it uses a lot of unnecessary space. What a good file system does, is that it helps protect against "soft" errors, i.e., data corruption due to timing errors and similar. Programs opening files for for transactions and messing things up, or the PC being shut down in the middle of things. Completely different from HW fails, which nothing but storage redundance can protect you from.
A UPS may be nice, but it still will not protect you against a poor file system mishandling a half-assed file transaction from a frozen program or PC.
Overall, I still think most modern file systems and OS will handle this decently, though :)
--- End quote ---
i did say that, which you should have seen in the first line of my post.
--- Quote from: kitamesume on November 03, 2012, 11:36:24 PM ---but last time i checked, except the old fat file system though, have good enough managing for the other things to matter.
--- End quote ---
evidently, if recent file systems were as unreliable as they say, i would've had much trouble by now but i have yet to stumble upon heavy errors during copy-paste sequence, TeraCopy also helped in this reliable file transfers.
Freedom Kira:
--- Quote from: Saras on November 04, 2012, 08:17:42 AM ---Wasn't exFAT designed purely with flash/external memory in mind? At least that's how I remember it.
--- End quote ---
Yes, that is true. However, considering the extremely large filesystem size and file size limits on ex-FAT, I would not say it was designed only for external media.
--- Quote from: kitamesume on November 04, 2012, 10:32:22 AM ---evidently, if recent file systems were as unreliable as they say, i would've had much trouble by now but i have yet to stumble upon heavy errors during copy-paste sequence, TeraCopy also helped in this reliable file transfers.
--- End quote ---
You're not really getting the point. No one said any filesystems were unreliable. Any unreliable filesystem would inevitably cease to exist. The benefits of advanced filesystems are, as said already, protection against soft errors. Copying and pasting, no matter how much data you are transferring, has an extremely low chance of such errors, and any errors encountered would be single bits at a time, which most copy-paste operations could fix with a simple CRC check (note that this is completely irrelevant to the filesystem).
You can get write errors from all kinds of file operations. One that Phlebas put as an example is a "half-assed file transaction from a frozen program or PC." Imagine that a program has opened a file stream for writing and ended up hanging partway through a write operation. Then, perhaps the program is terminated by the user before it is able to close that file stream. That half-written data sits there, and conventional filesystems like FAT or NTFS don't do anything about it. They might not even let you access the data, since it's write-locked to a file stream that never closed.
Other situations can happen as well. For example, imagine that a disk's cache has some dirty pages that have yet to be flushed to disk, or something is currently being written to the disk or cache. Suddenly, the power goes out. What happens to that data?
Also, keep in mind that data does degrade over time. No media can keep its data in perfect form forever. That's what data checks and repair are for.
--- Quote from: ConsiderPhlebas on November 04, 2012, 12:03:33 AM ---Overall, I still think most modern file systems and OS will handle this decently, though :)
--- End quote ---
Keyword "modern." NTFS was created in 1993 (most recent revision in 2001), and FAT32, the newest version of FAT before ex-FAT (ignoring FATX), was released in 1996. The oldest version of FAT, FAT12, was released in 1980.
kitamesume:
uhh... you said all that, but what you quoted already stated it.
your TL;DR means other file system supports "etc.".
but most file system already has the minimum features for errors to be ever so frequent, as stated.
and no, power outage isn't the system's fault, its a "disaster" outside of normal operations.
so whats the verdict? every file system should be adequate enough for other file system to matter, you'll only need to consider a superior one once you're talking over 3terrabytes of storage, where redundancy becomes too expensive to implement.
edit: theres also an advantage of redundant storage, either raid1 or manual backups, it ensures that files have the same consistency over two drives, specially raid1 where as a mis-write on one disk would be rewritten or rebuilt.
if, i mean IF, ever the written file gets mis-written on both drives, thats not the storage's problem, thats the OS or whatever's problem, TeraCopy or any other file manager with a CRC check built into it should solve this though.
buchno:
I've read through what you've written several times, and I'm still not sure what you're trying to convey...
--- Quote from: kitamesume on November 04, 2012, 12:37:13 PM ---and no, power outage isn't the system's fault, its a "disaster" outside of normal operations.
--- End quote ---
Of course it's not the system's fault, however, the risk for data corruption because of that "disaster" depends on which file system you use.
--- Quote from: kitamesume on November 04, 2012, 12:37:13 PM ---so whats the verdict? every file system should be adequate enough for other file system to matter, you'll only need to consider a superior one once you're talking over 3terrabytes of storage, where redundancy becomes too expensive to implement.
--- End quote ---
3TB? FAT32 is out of the question for anyone who wants to store files over 4GB (which often occurs a long time before 3TB of total storage) or wants file permissions. FAT32 and Ext2 are not journaling file systems, which increases the risk for data corruption significantly (although the performance is better, which makes them a good choice for small USB flash drives). There are multitudes of differences in useful features between today's file systems, which you can read about here or here.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version