This Blog Has Moved

Tuesday 28 April 2009

Digital Storage & Back Up! Back Up! Back Up!

We push ourselves to the limit; reach the highest peak, dodge bullets and land mines, badger PRs and argue with the Police...all because we have one agenda; that of making a great picture and communicating the story.
So, after all this effort, sometimes whilst risking our lives, other times at the risk of damaging relationships with loved ones, we dump the pictures onto a hard drive and forget about it, as the next assignment beckons.
We only remember as the cold sweat builds up when that hard drive goes down...and my friends, I guarantee, it will. Hard drives break; its only a question of when. I've had a hard drive go bad after three months of use and others which have lasted for many years.
What's interesting though is to see how well the new SSD (Solid State Drive) will do. I for one hope that they will realise our need for safe and archival storage.
So, how do I back up? I use two machines in my office; an iMac for office work and writing and a Mac Pro for photo work. The iMac has an external G-Tech 1Tb drive connected to it and is my Time Machine hard disk. For those who may not be familiar, Time Machine is built into Apple's Leopard OS and is a superb backup utility. It just works and has already once saved me when I deleted all of my archived sent emails by accident! On the iMac I also have a second hard drive (soon to be a G-Tech as the Freecom has given up!) which has a second backup (done manually and also with Apple's Backup utility (part of Mobile Me) of my letters, invoices, articles and my iTunes library. Lastly, the documents are also backed up onto my iDisk every few weeks.
The Mac Pro is a bit more complicated! Before I switched back over to the Apple Mac platform a few years ago, I had several years worth of images on my PCs, so with these I decided to keep them on an HP server, where there are 3Tb of images (all stored as a RAID 01 or Mirror RAID onto another 3Tb of disks, so 6Tb altogether). These images are all referenced into my Aperture library on the Mac Pro. This means that I can search for and see images even when the server is off-line and if I need a high res file, then I simply switch on the server and export the file through Aperture.
On the Mac Pro I also have one hard drive which is my Time Machine volume. This is used for everything but picture, sound and video files. Inside the Mac Pro are four drives and attached externally are eight other drives. Two of these external drives are WD MyBook Studio RAID boxes configured as RAID 01 or Mirror. My working files, pictures and video are stored on the internal drives, and backups on these WDs. This means that anything on these drives is actually stored on two drives; this is great as it saves you from mechanical failure in one of the drives. However, it doesn't save you from a file being accidentally deleted, so these drives are backed up onto external Seagate drives. The other drives are older work from previous years. My thought process is to use two different manufacturers, so the WD is backed up on Seagate. This is just in case in the future it becomes apparent that there are issues with a particular batch or make and model of a hard drive.
I do also have a Buffalo Terrastation NAS box on the network.
Lastly, all the important stuff is also backed up onto a virtual drive; a cloud disk. I've decided to go with Amazon's S3 as it seems a secure bet. I use Jungle Disk to access my S3 cloud storage. I did today see something called ExpanDrive which seems quite neat and does a similar job. So, by important stuff I mean; all the jpegs files I have sent to clients for publication (filed per year), all images are enter into competitions (filed by year), RAW files of mega important jobs, and video and audio associated files of importance.
I think that cloud storage is definitely the way to go as a third backup. You could have ten backups in your office, but if disaster strikes, they can all get damaged. You definitely need a form of off site storage. I initially was doing this with DVDs and hard drives, but it just didn't work. The archival qualities of DVDs is not really known and I've already had issues with CDs from a few years ago stop working. The problem of hard drive swapping off site is that its a job you have to keep on top off. With cloud storage (like iDisk and Amazon S3), its just a simple matter of drag and drop! Simples!

13 comments:

  1. Wise setup! For my work I keep everything in two separate locations and in the cloud!

    ReplyDelete
  2. Impressive setup, and not a dissimilar approach to other photographers I've spoken to. What do you think about the Drobo/DroboPro, is it something you would consider?

    ReplyDelete
  3. Thanks Carlo.

    Cranial, I love the look of the Drobo Pro but wouldn't use it. My philosophy is to keep it simple. RAID 1 works well and doesn't need special hardware/software if you were to remove one of the disks and plug it into another machine.
    I had a friend, a very computer literate chap, loose his RAID 5 setup for no reason at all. It was working fine one day and then the whole system went down. Thankfully the system was fixed by the manufacturer and his data was ok, but it was a nerve wrecking experience for him. The Drobo uses something similar to this. Although with the Pro version you can theoretically have two disks fail and still recover, I personally wouldn't use it.
    If Drobo had RAID 1, or even JBOD, I'd then be very tempted!! In fact, if the Drobo Pro had either of these, I'd buy two tomorrow!

    ReplyDelete
  4. Ironically when I looked at the Drobo option I was also concerned that I would still need a secondary backup.

    My first inclination was to go with G-TECH G-SAFE system, this would also allow me to take one drive out and store off-site. Unfortunately I understand the drives are moulded into the carriers which is a 'no' for me - I want to put WD Green 1TB drives in there.

    Thanks for the thoughts :)

    ReplyDelete
  5. What a pity the Imac doesn't offer an e-sata port. I find the G-Tech pretty slow compared to the Iomega Ultramax in Firewire 800.

    ReplyDelete
  6. Cranial - there are other's who make cases which you can populate with your own drives. Sonnett are one make who come immediately to mind, and Weibtech are another.

    Vinaro - It would be nice to have an eSata port for sure! I've not tried the Iomega drives, but do like the look of them. G-Tech are solid and dependable which is why I'm giving them a try at the moment.

    ReplyDelete
  7. Personally I think you're just showing off... ;-)

    ReplyDelete
  8. Onnik jan,
    I know you're teasing, but not at all. The amount of phone calls and emails I get asking me how I backup and what colleagues should do when they've lost work led me to write this post.
    I realise my system may be a tad over the top, but I'd rather be safe than sorry!
    I hope that it'll be useful to colleagues to see how my setup works.

    ReplyDelete
  9. Edmond, yeah, I was (and perhaps a little bit jealous too...) and also, yeah, your site is a really useful resource. Interesting too. :-)

    ReplyDelete
  10. It's always good to see how real pro work and organise themselve.
    Thanks for sharing Edmond.

    ReplyDelete
  11. Thanks folks; most kind. Glad the blog's of some use!

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. Hi Adam,
    Sorry to hear of your troubles. It just goes to show that hard disks do fail; its just a question of when.

    As far as packages to try; I haven't used these to recover massive amounts of data, but have used them on CF cards and had good success with both Sandik's RescuePro Deluxe and File Salvage by SubRosaSoft.

    Good luck!

    ReplyDelete