Discussion Forums > Technology

Moving large amounts of data between HDDs... CRC "fail"?

<< < (2/6) > >>

megido-rev.M:

--- Quote from: lapa321 on June 18, 2012, 05:10:59 AM ---It's normal. Our anime just uses so much data that we easily blow through any kind of error margin or data safeguards built into our systems.

--- End quote ---

Not even: I've used checksum checks on single 1~8GiB files and X00GiB sets and they never failed on me like that.

As far as my OS-level disk subroutines knowledge goes, the disk device must return the exact data it contains no matter what, being nonvolatile and all. There is also the fact that a disk loads that data into memory, through DMA and such. So, like the guys above me mention, it's possibly a memory corruption issue-- mind anyone, checksum checks are computationally expensive (read: RAM).

Anyway, if you would like to be minimalist for checksum checking you could try HashCheck, completely stable as far as I know.

Southrop:
What do you use to copy the files? Default OS copy manager? I recommend trying TeraCopy if you use Windows. It's faster and better than the default, and it automatically does CRC checking/testing after copying or moving files. Prediction times are also way better xD

Pzc:
If you decide to do some diagnostics I recommend
the aptly named Ultimate Boot CD. There's some
useful stuff on it, including memtest. It can also be
booted from USB if so desired.

lapa321:
Between my archives, download server, NAS, primary and secondary harddrives. I regularly move several hundred Gigs up to a terabyte of anime. More if a friend comes over with his harddrive to trade his collection. And despite mutiple diagnostic tests and even several hours of burn in tests with the computer coming out clean. CRC errors will still occasionally happen.

These are the kinds of copy operations that takes several hours to a day (hence, why i recommend burn tests on top of diagnostics). In fact i just finished archiving a 300Gigabyte batch yesterday. It took an hour to move from the download to the processing harddrive, and another 5-8 hours packing them into ISOs and archiving to an external backup drive.

I've run into a bad memory when i was copying anime from a friends computer. It was only ten gigs but several files were corrupted, that one was pretty easy to determine. But a 50% chance of a single error in a terrabyte worth of data? Whatever the cause, it's not showing up on any of my diagnostics.

BTW, the utilities i use when moving files for archiving: FastCopy and Beyond Compare.

kitamesume:
im gonna join in into this discussion...

anyway, im getting errors with network transfer via LAN, the files that were transferred becomes incomplete like missing parts or corrupted parts when i recheck it with utorrent(using force re-check). any clues what causes this? or maybe my router is dying =/

edit: btw, USB-to-external drive transfers doesn't cause any of these issues at all =o

Navigation

[0] Message Index

[#] Next page

[*] Previous page