Backing up as files get bigger

On 23 Sep 2007, at 22:09, Jonathan Ploudre wrote:

> How are people backing up as iMovie/iPhoto/iTunes start exploding in
> size. I'm looking most for bang for buck. If I had $8000 I could just
> get an XServe RAID.

I have a ReadyNAS NV+ as RAID-5, and I am very happy.
I have all iTunes and iPhoto stuff there, connected to a MacMini
without keyboard/monitor.
They are shared, and I use an airport express to listen to the music
on the stereo, with the mithycal Keyspan Remote for iTunes:
<http://www.keyspan.com/products/tvu200c/homepage.spml>,
so that I can have everything under control from everywhere in the
house.
The iPods are syncing with the MacMini.
I sync some playlists with my MacBook with SlingShot.
I use Remote Desktop from my MacBook to control the MacMini.
Also all the iPhotos are there, and they are shared.
The ReadyNAS NV+ can switch on/off automatically with a predefined
schedule (useful for backups).

cheers
--e.

On Sep 23, 2007, at 4:09 PM, Jonathan Ploudre wrote:

> I've been using Joe's Take Control of Backups recommendation of using
> multiple external firewire drives. I just got a refurbished iMac and
> anticipate my storage will dramatically increase (using more video off
> the Tivo for my girls). I can easily imagine exceeding the space of a
> single drive. And large capacity drives cost much more per GiB.
>
> I've looked at NAS like ReadyNAS NV+:
>
> http://www.infrant.com/products/products_details.php?name=ReadyNAS%
> 20NVPlus

I don't have NAS experience but also look at the Buffalo TeraStation.
It looks to be in a similar class and maybe easier on the wallet.

> Or a USB2 multi-drive enclosure like drobo:
>
> http://www.drobo.com/products.aspx

The Drobo sounds interesting but it'll take a lot of time and other
people's experiences for me to trust it.

> And I'm considering using my old PowerMac G4 933 with multiple
> drives in it.

That used to be the way I would go but I've become more conscious of
the power costs related to keeping old desktops running 24/7. Over
time, a NAS device might pay for itself in power savings compared to
putting the same amount of storage in an old Power Mac. Plus, it
should take less time to administer (your time has value, right?) and
can offer features like RAID5 that OS X (non-Server version) doesn't
have. Obviously the equation changes if the computer can be used as
more than just a file server.

On 9/23/07, Jonathan Ploudre <jploudregmail.com> wrote:
> How are people backing up as iMovie/iPhoto/iTunes start exploding in
> size. I'm looking most for bang for buck. If I had $8000 I could just
> get an XServe RAID.

I simply omit certain huge folders (i.e. movies that I've ripped to
disk, iMovie's library, VMWare images, etc.). Those big folders get
burnt to DVD with some regularity, and if they're especially important
(I don't consider Season 2 of Battlestar Galactica to be especially
important, but you may feel otherwise), I'll burn two copies and store
one off-site.

iMovie gets slightly better treatment since those files are pretty
well impossible to replace. I keep the original DV tapes in a safe and
burn the DV footage to DVD immediately after importing.

It's not bulletproof nor especially automatic (although the big
folders DO get copied to my "server" G4 Cube for a first-draft backup
before they're archived to DVD), but it's good enough for files that
are just entertainment.

Every other file I back up (my home folder, pretty much) gets archived
to DVD twice a year as well. It's time consuming, but the backups
should last a good long time and are easy to store and transport. I
pretty much consider the DVDs backups of last resort, with my drive
image and regular documents backup being my primary restoration tools.
The DVDs are backup #3.

iTunes and iPhoto grow regularly, but I don't find them "exploding"
the way you describe. More of a steady climb of a gig or so each
month. I seem to keep up with that by upgrading my backup drives every
year or so. They're just part of the regular backups.

--Nik

If you're just backing up your own system, then the Drobo is probably the best way to preserve the systems you've been using with your external FireWire drives. It's flexible and will take more capacity. But, it's also USB 2.0, which is going to be slower than FireWire.

I had been using a Granite Digital FireWire enclosure with swappable drives, but that had two limitations. One was the capacity of a single drive (which certainly continues to go up), but the other was more important: needing to back up more than one computer, mine and my wife's (and in the future, maybe a media server or other devices).

I bought an Infrant ReadyNAS NV, and have been very happy with it. Both of our computers use SuperDuper to create mirrors of our boot drives into disk images, every day. This is my primary backup.

I also continue to use Retrospect to do incremental backups that have archived versions of files, in case I need to go back into the past. I'll probably drop Retrospect when Time Machine has been proven to be safe (bug free) and effective, because it's really not very good software anymore. It works, and does things nothing else does, but with so many quirks and irritations due to its age that I just don't want to deal with it any more...

One thing that this does not give me, that the removable FireWire drives did, is the ability to take my backups off-site. This worries me considerably (more concern for accident / disaster than theft), but I'm hoping that Jungle Disk or some other Internet-based backup service will emerge as a solid choice, and that'll free me from that concern. For now, I'm just crossing my fingers (and occasionally updating my FireWire backups).

> I don't have NAS experience but also look at the Buffalo TeraStation.
> It looks to be in a similar class and maybe easier on the wallet.

I have a TeraStation and I love it. It has 4 drives configured as
RAID 5 by default (although you can change that). It also has 4 USB
ports for sharing a printer or external drives. I hacked mine to add
NFS support and mt-daapd for iTunes library sharing.

Jonathan Ploudre wrote:

> How are people backing up as iMovie/iPhoto/iTunes start exploding in
> size. I'm looking most for bang for buck. If I had $8000 I could just
> get an XServe RAID.

You might look at a firewire tape drive. Sony makes them for their AIT1 & AIT2. Plus I think they are made for some other setups. Once you get over the cost of the drive, the tapes hold way more than disk drives for the same $. And are much easier to manage physically.

David

On Sep 26, 2007, at 3:45 AM, David Ross wrote:

> Jonathan Ploudre wrote:
>
>> How are people backing up as iMovie/iPhoto/iTunes start exploding in
>> size. I'm looking most for bang for buck. If I had $8000 I could just
>> get an XServe RAID.
>
> You might look at a firewire tape drive. Sony makes them for their
> AIT1 & AIT2. Plus I think they are made for some other setups. Once
> you get over the cost of the drive, the tapes hold way more than
> disk drives for the same $. And are much easier to manage physically.

Tape for home backups? Ugh, no way. Then you get into issues with
tape library management (both physically and in cataloging), possibly
tape spanning, yuck. If he has a 750GB hard drive to back up, how
many AIT2 tapes is that? If Remember, when most of your files are
media, just ignore the compressed capacity of the tapes, the files
are so compressed already you're only going to get the native
capacity out of the tape.

Also with tape, you're basically forced to use the same software to
do a restore whereas there are many hard drive backup schemes that
allow you access the files with nothing but the Finder.

Tape makes more sense when you need to keep a long history of
snapshots so you can pick any day and restore from that point but
when most of your GB are media files that won't change over time, you
only really need 1, maybe 2 copies of each in your backup.

And if I needed tape backups, I would pick LTO, an open standard with
many tape and drive manufacturers, rather than AIT. Also because it's
a linear format rather than AIT's helical format but that's too nit-
picky for here.

At 12:45 AM -0700 9/26/07, David Ross wrote:
>Jonathan Ploudre wrote:
>
>>How are people backing up as iMovie/iPhoto/iTunes start exploding in
>>size. I'm looking most for bang for buck. If I had $8000 I could just
>>get an XServe RAID.
>
>You might look at a firewire tape drive. Sony makes them for their
>AIT1 & AIT2. Plus I think they are made for some other setups. Once
>you get over the cost of the drive, the tapes hold way more than
>disk drives for the same $. And are much easier to manage physically.

Actually, I don't think they make AIT1 anymore. When I bought my AIT2
from LaCie two+ years ago it was already being replaced by AIT3.

The tapes on AIT2 hold about 50GB each (uncompressed, which is video
and audio), which isn't much data volume these days.

I just bought a 465GB HDD for US$189, to back up the 465GB I bought
two years ago for well over US$300. And the new drive is also eSATA.
(Not that I have anything that handles that, but it's FW400 and FW800
as well.) Either one requires 10 tapes - value US$500 - if the HDD
is full. And the tape drive is now flashing the blue indicator light,
so repair will cost another $365.

Tape drives may be becoming an old (slow, expensive,
non-random-access) paradigm for us home users.

On 27-Sep-2007, at 06:26, Dan O'Donnell wrote:
> Tape drives may be becoming an old (slow, expensive, non-random-
> access) paradigm for us home users.

Tape has never been a viable option for home users. It's almost
never held enough data to make backing up realistic, it is very
expensive (I recall buying a $4000 8mm SCSI Tape unit for a Sun), and
the tapes aren't cheap, and it takes a certain amount of managing the
tapes, storing and cataloging and such. Recovering files is also a
huge issue with tape.

There was a time when CD-Rs were a good choice, and a time when DVD-
Rs were a good choice. Right now, there is NO good choice.

I backup my 'small' data regularly<1>, but the video and MP3s and all
of that I mostly just pray and try to keep two copies of the MP3s.
The video is just ridiculous to try to backup though, so I keep the
original tapes.

(I have an in-progress iMovie 05 project that is over 50GB).

Hopefully data BluRay will be along soon so that once more we have a
viable backup solution, at least for a couple of years (BluRay
promises capacities up to 100GB per blank, so it would still take 20
of them to full backup my computer... :/ HD-DVD, on the other hand,
promises capacities of 30GB, and maybe 60GB max, so on that score at
least I am pulling for BluRay)

<1> Keychains, preferences, and documents. Email is all IMAP, so
that gets backed up in the server backups and not something I need to
manage. everything I backup still fits on a single DVD (or more
often a thumbdrive)

At 5:26 AM -0700 9/27/07, Dan O'Donnell wrote:

>--snip--
>I just bought a 465GB HDD for US$189, to back up the 465GB I bought
>two years ago for well over US$300. And the new drive is also eSATA.
>(Not that I have anything that handles that, but it's FW400 and FW800
>as well.) Either one requires 10 tapes - value US$500 - if the HDD
>is full. And the tape drive is now flashing the blue indicator light,
>so repair will cost another $365.
>
Dan:

If backup time matters and you have an open PCI slot, adding eSATA
port is about $50. Its a lot faster than Firewire.

--Jim

--On 9/29/2007 6:43 AM -0700 Google Kreme wrote:

> There was a time when CD-Rs were a good choice, and a time when DVD-
> Rs were a good choice. Right now, there is NO good choice.

I will disagree--hard drive prices are low enough and interfaces easy
enough that a pair of external hard drives is an excellent choice. I feel
pretty confident in my current backup approach:

(1) Daily backup to a second internal hard drive. (This could use an
external USB2 drive if you have an iMac or Mini; it's a bit harder if you
have a laptop since you need to know when the external drive will be
attached.) This is my primary backup--the only limitation is that it's
inadequate for disasters like fire or theft.

(2) A pair of USB2 hard drives for offsite backup. At least one always
lives in my office (seven miles from home, third floor), so there's a
backup there when the other drive is being taken home for a new backup and
returned to the office. I will confess that I don't do this as often as I
should; I'm looking around for some sort of carrying case for the backup
drive that will help me make it a bit more automatic.

I assume that if a disaster takes out the several-mile radius including my
home and my office at the same time, I will have things other than my data
to worry about.

Dave

On Sep 29, 2007, at 6:43 AM, Google Kreme wrote:

> On 27-Sep-2007, at 06:26, Dan O'Donnell wrote:
>> Tape drives may be becoming an old (slow, expensive, non-random-
>> access) paradigm for us home users.
>
> Tape has never been a viable option for home users.

I used it for several years, several years ago. Painful, but less
painful than bunches of floppy disks.

I moved away from it when I could.

   --John

On 29-Sep-07, at 6:43 AM, Google Kreme wrote:

> I backup my 'small' data regularly<1>, but the video and MP3s and all
> of that I mostly just pray and try to keep two copies of the MP3s.
> The video is just ridiculous to try to backup though, so I keep the
> original tapes.

As things have grown larger and larger, my backup strategy has (like
Google's) moved to split files into two groups. Smaller things, like
home directories (including some over the net, like my parent's
machine) get backed up regularly to a 3 way mirror raid. One of the
three disks is moved to my office at work on a regular basis to give
me off site backup, and the other two are at home giving me a mirror
raid (so a single disk failure will still leave me two copies, one of
which may be a bit older (the one at work)). I also keep my music
collection on this disk as it is rather large and I really don't want
to re-rip 4000 disks (plus I get a copy at work so I can listen to it
there).

All the large files (mostly rips of our DVDs and TV shows so my wife
(who has M.S.) has access to things easily) live on a 2TB RAID 6 disk
that could be mostly rebuilt if I had to by re-ripping 200 or so
DVDs. The RAID 6 should protect us from disk failure, but, of course,
fire or theft would be a problem. Since I put this together myself,
this isn't a solution for any but those with either deep pockets (not
me) or some technical savvy. I would feel better if there was a way
to backup this much data, but the really important stuff is backed up.

My question is what are people using as software to do backups. I
used to use Retrospect but ran into some problem a year or so ago
which I could not get past. I switched to CrashPlan, but I think it
still needs to mature a bit. I like that it does backups across the
net (so I can backup things like my parents machine), but it can't
handle backups to an external disk at all well, and I am constantly
worried about backups silently failing. Is there any other solution
that will work across the net, even better if it worked for Linux as
well as Mac OS.

Ray Davison

On 29-Sep-2007, at 08:04, Dave Scocca wrote:
> --On 9/29/2007 6:43 AM -0700 Google Kreme wrote:
>
>> There was a time when CD-Rs were a good choice, and a time when DVD-
>> Rs were a good choice. Right now, there is NO good choice.
>
> I will disagree--hard drive prices are low enough and interfaces
> easy enough that a pair of external hard drives is an excellent
> choice. I feel pretty confident in my current backup approach:

I said:
>> (BluRay promises capacities up to 100GB per blank, so it would
>> still take 20 of them to full backup my computer... :/

at 100GB per blank and 20 blanks, we're talking about two TERABYTES
of files. There is no good choice for that amount of data.

On 30/09/2007, at 19:36, Ray Davison wrote:
> My question is what are people using as software to do backups.

For the past couple of years I've used SuperDuper to backup to a pair
of external FireWire hard drives, rotating them week in, week out,
keeping one of them attached to the computer at home and one in a
safe place at work.

I've just upgraded to an iMac with a 750 GB HD so to maintain my
backup strategy I've just purchased two identical Seagate mechanisms
and two new Sarotech HardBox enclosures which have FW400/800, USB2
and eSATA. Shame the iMac (and no Macs natively) don't have eSATA
ports yet.

The one downside to SuperDuper is that, whilst backups can be
scheduled, they can't take place if no user is logged in. I've
recently been experimenting with the new version of Carbon Copy
Cloner which doesn't have that limitation (but which has a slightlye
more peculiar user interface and less flexibility in scheduling).

Cheers, Andrew

At 07:58 09/29/07 -0700, Dave Scocca wrote:
>hard drive prices are low enough and interfaces easy
>enough that a pair of external hard drives is an excellent choice.

This may now suffice for home use. For even small business use, only two
copies is not a good thing, especially when one of them is on site. A fire
or other disaster on site could easily destroy both the live copy and one
backup, leaving you with only one copy. An error in the media, problems
retrieving it, a PEBKAC error under the pressure of doing the unfamiliar
restoration task -- you could lose the last copy.

Certainly we are moving in the direction of the mechanism being so cheap
that including a mechanism with every piece of media is becoming viable.
That was the issue in the past: the tape or disk drive cost so much that
you wanted removable media for backups. I see the current situation as on
the cusp for those with up to a few hundred GB of data and only a need for
current protection, not archives. For those with under 100GB, it's already
feasible to buy three or four external disks. For those with 2TB of data
(you know who you are) or needing archival backups (as many businesses do),
you are still looking for removable media. But even that is changing.

Jungle Disk /Amazon S3 remains a viable extra tier for those with
sufficient high speed Internet pipes. Storing 1TB, with 100GB/month of
transfer, would cost $170/month. That'll make a home user blanch, but it's
worth considering for many businesses, since it has the advantage of being
a different kind of backup, with much higher reliability (effectively
additional backups behind it).

Edward

My problem that I have with drives is that if I big the biggest
available drive (say 1TB), I need something bigger than that for doing
an archival backup. I suspect that the sweet spot for hard drives is
750GB which is double my current space. I don't think it'll take long
to fill it up. I've been using duplicate backups but would like
archives. Unfortunately that sort of size won't fit through a
consumer's broadband for offsite backup.

Jonathan

On 9/30/07, Ray Davison <raysfu.ca> wrote:
> My question is what are people using as software to do backups. I
> used to use Retrospect but ran into some problem a year or so ago
> which I could not get past. I switched to CrashPlan, but I think it
> still needs to mature a bit. I like that it does backups across the
> net (so I can backup things like my parents machine), but it can't
> handle backups to an external disk at all well, and I am constantly
> worried about backups silently failing. Is there any other solution
> that will work across the net, even better if it worked for Linux as
> well as Mac OS.

rsync is fantastic and cross-platform (you can use it on Linux, MacOS,
and if you install Cygwin, it can run on Windows). There's some fairly
serious issues with the version that ships with OS X, but you can use
Fink to install a patched version. (The package is called
"applersync") Unison is another good one, which has a native version
for all three major OSes.

Both have some problems with Mac metadata, but nothing fatal. Mostly
date, extended attributes and ACL problems.

Personally, I use three apps for my backups:

SuperDuper creates a simple backup image on a network drive which gets
updated monthly. (It takes a looooooong time to run, so I don't run it
too often.)

My user (~/) folder gets backed up to the same drive using ChronoSync.
I also create snapshot backups using gnu cp to create hard linked
copies every night. That way I have a snapshot rolling back in time on
that same disk.

Finally, I use Interarchy to copy critical files (my Documents folder
and iPhoto's library) to an SFTP site. I use its automatic file
conversion feature to encode the files in the Interarchy Backup format
which ensures all metadata and other Mac weirdnesses are maintained,
even on a Linux server.

Then I (occasionally) burn DVDs of my Documents, iPhoto library and
iMovie events folder. Usually I make a couple copies so I can take one
off-site. I ought to automate this, but I never got around to it. As
it is, I kind of like having a single set of DVDs that has EVERYTHING
on it

I also use .Mac backup for a few files because, well, I don't have any
other use for my 10 gigs of .Mac space. (I bought .Mac for iSync,
pretty much)

---
Nik <nikinik.net>
www.inik.net | notions.inik.net