Data Management

February 03, 2013  •  Leave a Comment

 

In this blog I would like to take you through a basic flow of how I manage my images from the card though my long term storage system.  There have been a number of articles on this subject and my system is not that different then others I have read but I thought that I would my entire system together in one place.

Cards

I use SanDisk, I do not think they are better than any of the other major name brands but I do think it is important to use a name brand.  What goes on these cards is too important and valuable to try to save a few dollars by going with some unknown generic brand.  I have been shooting for a number of years and I have never had a card go bad on me yet but I may have just been lucky.  I will only format my card in camera and I think that helps a lot.  

I also tend to use smaller capacity cards.  My primary cards are 16 GB and the majority are 8 GB.  There are two reasons for this.  The first being that larger capacity cards tend to be very expensive.  The technology is changing so fast that within a couple years the high capacity cards will replaced with even larger cards and the cost of the old cards drops dramatically.  This is true of many things.  Trying to stay on the cutting edge of technology is really expensive and generally not worth the cost although that is a personal judgement.  Does anyone remember 1 GB cards?

The second reason is the same thing your mother told you.  I don’t want to put all my eggs in one basket.  A 16 GB card in my Nikon D3S will hold 473 images at the highest resolution.  That is a lot of eggs!  Do I really want to put more than that many eggs in one basket?  I don't.

I am sure that a card failure will happen to me at some point but I am trying to minimize the chances and the effect if it does.

Card Storage

When shooting I try not to completely fill a card again for two reasons.  First, I have read though never experienced, that this is bad for the card and that the last image may be damaged.  The second and main reason for me is I like to be prepared and I don’t want something amazing to occur and have only three or four shots remaining on my card.

Earlier I mentioned I shot with a Nikon D3S as my primary camera and as you may know it has two cards slots.  I have it set up so that once the first card is filled it will automatically begin using the second card.  So no, I do not always follow the advice of not filling a card up but when there is a pause I will replace the full card.

I just want to stop for a moment and rant about I trend I see happening.  I will speak about Nikon because that is what I am familiar with but I would not be surprised if other manufactures are doing the same thing.  The D3, D3s and D3X all have dual slots for Compact Flash cards.  The newer cameras, D300S, D4 and D800 also have two card slots but they are for two different types of memory cards.  The D800, for example, uses one CF card and one SD card.  On what planet does carrying around and keeping track of two different storage mediums make any sense at all?  Different battery systems are bad enough although more understandable.  Pick one storage medium and stick with it.  Yes, obviously I would prefer it to be CF cards because this is what Nikon has used with professional cameras in the past and I have a large number of them.  

Once have a full card in will replace it and place in my card holder.  All the cards in the card holder are placed in with the label side up.  The full cards are turned around so the label is face down.  I use a ThinkTank Pixel Pocket Rocket, there are many other card holders out there I just happen to think that they may good products and the Pocket Rocket holds a lot of cards in a relatively small container.  This allows me to easily see which cards are full and which one are empty.

Transferring Images in the Field

At the end of the day once I get back to the hotel room, office or wherever my laptop is I will transfer all the images from the CF cards on to the laptop.  Along with importing to the laptop hard drive I will always import an additional copy on to an external hard drive.  There are two types of hard drives.  Those that have failed and hard drives that are going to fail, it is only a matter of time.  I cannot emphasize enough the importance of having a back up.

There are a number of different card readers available on the market.  The one show is a SanDisk firewire reader.  I also use the Lexar firewire readers which are convenient because they are stackable and able to importer a number of cards in sequence.  I prefer the SanDisk reader when I travel because it is smaller.  Neither one of these are these are available any more so I recommend you research them and try to purchase a fast card reader.  I always make sure I have an extra card reader as well normally one that can import the SD cards from my point and shoot as well.

I use Adobe Lightroom when I import my images in to the computer.  Lightroom will allow me to at the same time import and additional copy to a separate location, in this case my external drive.  Before I import my images I normally will change the file name and add general keywords that apply to all the images.

One thing I try to do and I think it helps immensely is do a review of the images that you captured that day.  Reviewing the images just using the display on the back of the camera is very limited despite continued improvements in resolution.  I try not to delete anything off the camera itself unless it is very obviously out of focus, etc.  

The nightly review also helps with insuring that you captured the images that you thought you did as well as identifying any mistakes you may be making with your technique.  Discovering this now is much more useful than when you have already returned from your trip.  As far as sorting the images shot that days I generally limit this to just deleting the obviously bad ones at this point.  

Normally I do not go too much farther at this point as it is generally late and there is still equipment maintenance etc but I try to at least get the images transferred nightly.  Once the images have been transferred from the card I will not reformat the card until all the blank cards have been filled and I have to start reusing them. 

Many photographers will advocate have two separate external hard drives as backups and while I will normally bring two external drives I find backing up the images in two separate drives as well as the laptop to be very time consuming.  I will generally make a complete copy of my external drive on to the second external drive before returning home.  During the trip I ensure the drives and laptop are all packed in separate bags in the event of loss, theft, etc. 

Data Management at the Office.

There are many options for home storage and they have been discussed at length on the internet.  I just intend on giving a brief opinion on each as they apply to my workflow.

CD/DVD - This option is limited due to the amount of storage it offers.  In my opinion it is too easily damaged and I believe it is a dying media format.  I know many people will not agree with that but I view it much like the 3.5 disk.

Cloud Storage - This storage is relatively new and very interesting with many possibilities.  The disadvantages to this as I see it are: There is a monthly fee, which based on the amount of storage you need may be less than the cost of a large hard drive.  While it claims to allow you to access your storage anywhere in reality what it really means is access it anywhere you have internet access.  In addition to this in many places in the world where you may have internet access you very well may be paying for each megabyte of bandwidth - you may be paying a lot for it with very poor connectivity.  

This may not apply to you.  As I am typing this I have been waiting 15 minutes for Facebook to come up on my laptop tethered to my phone using Ethiopian Telephone - the only cellular service in the country which actually works fairly well compared to many other places.  Paying by the amount of data you use is not just limited to remote regions of the globe it is also fairly common in Europe as well. 

Hard Drives - Hard drive prices continue to drop.  One terabyte drives are now relatively cheap.  Hard drive also satisfies my desire to control things all of my data is in one spot that I can put my hands on it and not controlled by some mysterious “cloud” person who may for some unimaginable reason might be going through my files or go bankrupt and shut down without warning.  Paranoid? Perhaps I may have a touch of that.  Admitting you have a problem is the first step - not that I am admitting that it is a problem. 

OK how do I do it.  I used to travel with my laptop and once I got back to my office export my Lightroom library to my main computer.  I have been trying to cut out my desk top computer from the equation and use my laptop as both my travel computer and my main computer - apple has been helping me with this by refusing to update the Mac Pro computer.  When I am at the office I connect my laptop to a large external monitor and a G-Speed Q RAID for my long term storage.  So far I think it has been working for me but it is too soon to draw any conclusions.

Along with the RAID setup at the office I maintain two complete backup copies at an offsite location.  These backup contain backups of both the data on my laptop and all the data on my G-Speed Q.  The off site backups are updated weekly with each alternating every other week - depending on the amount of images I am processing this may change accordingly.  This ensures that I have two separate backups that go back two weeks.  I store the hard drives in two separate pelican cases.

This system seems to be fairly standard, although some may backup more often.  At the very least you need to have some back up system in place and preferable located at a different location than your main computer because eventually all hard drives will fail.  Take a moment and think about all the information that you have stored electronically and the affect of losing all of that data. 

Here are some links to other photographers data management plans:

David duChein

David duChein selected the G-Speed Q also when I was researching RAID systems

Scott Kelby

Scott Kelby was doing research and trying to find a replacement system for his Drobo but due to the length of time this was taking I couldn’t wait any longer and went ahead with the purchase of a G-Speed Q and was pleasantly surprised when he decided on the same system.  His blog examined many good and interesting alternative to this system as well - really informative.

Chase Jarvis

This system is probably a little more extensive than most people need but very interesting.


Comments

No comments posted.
Loading...

Archive
January February (3) March (1) April May June July August September October November December
January (2) February (1) March April May June (1) July (1) August September October November December
January February March April May (1) June (5) July August September October November December
January February March April May June July August September October (2) November (2) December
January February March April (1) May June (1) July August September October November December
January February March (3) April (3) May (2) June July August September October November December