In this post I quickly document how I recovered files from an accidentially formatted SD-card. The disaster of data loss can hit everyone any time. This time, my 17-month old son managed to format the SD-card with all pictures from our vacations while we were still in the train home. After the initial shock and suppressed anger towards my parter under who’s supervision the accident happend, I started to analyze the situation: If and how can our vacation memories be recovered? After a bit of research, I had more hope than Nina Hagen in the classic song Du hast den Farbfilm vergessen (you forgot the color film).
Technology background
SD-card technology
SD-cards are based on the flash technology (for details see the Wikipedia article). To understand the scope of this article, we just need to know that an SD-card consists of a flash controller and an array of memory cells. The individual memory cells are programmable using electical signals and then keep their state even when the power is turned off until they programmed again. The controller maintains a table that maps the files in a folder structure to actual hardware memory cells. Since the number of write-cycles of the memory cells is quite limited, the controller actually balances the wear of the cells by deciding where exactly to save a file in the hardware. The controller also keeps track of the failed memory cells and manges the activation of spare cells to keep the available memory size from shrinking.
Quick vs. full formatting
Quick formatting deletes all the files from a SD-card (in this case) by reinitialyzing the mapping table to a default state. The SD-card seems to be empty and in fact, the whole available physical memory can be used to save new data. However, quick formatting doesn’t alter the state of the memory cells and therefore the previously stored data is physically still there. That is why we can have good hope to recover all or most data after formatting a memory medium, unless we overwrite it by new data. That means, to recover most files from a formatted SD-card, we must stop writing to the memory device, i.e., stop using it.
In contrast to that, full formatting overwrites all data in the memory cells and tests each cell’s propper functioning. After full formatting, there is actually no hope of easily extracting much information from the memory device. Full formating should be limited to initialize a memory medium and when the medium changes the owner. The two reasons:
* full formatting puts extensive wear on the memory cells and thus shortens its life time
* it takes a lot more time than the quick formatting process
Recovery process
Make a back-up
We need to protect us from making our situation worse through human error or hardware failures. It is advisable to make a backup of the SD-card using a suitable program for your platform. I use linux and didn’t want to purchase any software, so I opted for ddrescue. First, you can find your </dev/path> using
df -H
if the SD-card is mounted or in gparted. Verify the size and device label to be sure that you have really identified the right medium. Then,
ddrescue </dev/path> <path/to/diskimage/destination> <path/to/mapfile/destination.log>
will write a copy of the whole medium to the destination file. The destination should accordingly have enough free disk space (and logically not be on the same SD-card). The process can take a while depending on the medium size and the transfer speeds etc. You should see something like this:
GNU ddrescue 1.22
ipos: 63864 MB, non-trimmed: 0 B, current rate: 81723 kB/s
opos: 63864 MB, non-scraped: 0 B, average rate: 46752 kB/s
non-tried: 0 B, bad-sector: 0 B, error rate: 0 B/s
rescued: 63864 MB, bad areas: 0, run time: 22m 45s
pct rescued: 100.00%, read errors: 0, remaining time: n/a
time since last successful read: n/a
Finished
Recover files from formatted SD-card image using PhotoRec
We will use the software PhotoRec to recover the files from the SD-card image. PhotoRec also can work directly on the SD-card, but for the sake of not (accidently) overwriting any data we work with the image. To install PhotoRec, type in a terminal:
sudo apt install testdisk
Then photorec can be launched from the terminal:
photorec <path/to/diskimage/destination>
A text based user interface asks which partition should be recovered (whole device in my case) and for the file system of the SD-card before the formatting (select other for SD-cards used in cameras). Finally, navigate to the folder where recovered files will be saved and hit c to start the process.
PhotoRec will crawl through the hardware memory of the medium (or image thereof) and save any intact file it comes accros regardless whether it is mapping table or not.
PhotoRec recovered for me 64GB of image files and videos, which surprised me a bit as I had formatted the SD-card before the vacations and the memory was far from full when the accidental formatting happened. Exploring the output, I found images older than a year. Also thumbnails in different sizes were present. The image files have correct meta-data except for their file names. The videos were, however, not playable by the VLC player.
Repairing the video files
Video files created by a camera are naturally fragmented in the memory as they are written contiuously and their final size is unknown. The recovery software has basically no chance to know how many such segments a video has. The testdisk manual has a short chapter on merging the segments in simple cases.
According to the manual, the video files can be easily merged when they are in pairs of a *.type file and a *_mdat.mov. In my case,
cat f0405760.mp4 f0409600_mdat.mov > test.mp4
produced a playable video. I had to change the ownership of the containing folder using:
sudo chown <user> <folder>
Conclusions
The data from quick-formatted devices can be recovered quite easily. However, I will have to filter the pictures for the import into my photo workflow to exclude old pictures and the thumbnails. The original filenames are lost, but other meta-data (including the EXIF data) was kept. The video files, unfortunately, need to be processed manually. The finding of such old pictures is a warning to be very careful when giving memory devices out of your hands.