Skip to content

Product idea: RAID Backup Enclosures

Cory Doctorow at Boing Boing links to an article at TechCrunch that lists Better and Cheaper Online File Storage as a product that needs to be made. However, Ben Laurie does the sums on online storage as a useful backup medium, and found them not exactly compelling (e.g. 100GB of data will take 75 days to upload over an 128Kbps link).

I tend to agree. An online host isn't great as a backup host, since, in my experience, there are two types of backups required:

  • The important small files (for example: encrypted password lists, my address book, my ~/bin directory)
  • The massive big filesets (for example: MP3s, photos)

The first kind of fileset is amenable to an online backup-storage service, at first glance. However -- in my opinion you're better off going the whole hog for these files, and using the distributed, versioned backup method of putting it in a good networked revision control system, and checking it out everywhere, so you can also make changes and check in from any host; otherwise, you face the perils of syncing up a single backup from multiple "writers", without conflicts. So far, none of the online file storage services offer SVN as an access method, so a shell account at a colo server still seems more useful on that count.

The second kind of fileset, as Ben notes, will take donkey's years to upload and sync as a backup mechanism; and the economics are hardly compelling for the service provider.

I think I prefer Brad Templeton's idea to deal with large-data backups --

I propose a software RAID-5, done over a LAN with 3 to 5 drives scattered over several machines on the LAN.

Slow as hell, of course, having to read and write your data out over the LAN even at 100mbits. Gigabit would obviously be better. But what is it we have that's taking up all this disk space ? it?s video, music and photos. Things which, if just being played back, don?t need to be accessed very fast. If you're not editing video or music, in particular, you can handle having it on a very slow device. (Photos are a bigger issue, as they do sometimes need fast access when building thumbnails etc.)

This could even be done among neighbours over 802.11g, with suitable encryption. In theory.

As a commenter notes, Linux has support for this already, in the form of software RAID and the network block device.

So: take an external IDE enclosure, add a GumStix board running Linux with software RAID, LVM, and nbd, and add wifi. Then add DAV, SMB and NFS export of the disk, and some decent UI code to organise the volumes into a single exported RAID volume (hopefully automatically!), and it'd be a pretty compelling product, in my opinion!

(hey Craig! I said GumStix! ;)

backups disks gumstix hardware linux lvm nbd product raid

12 Comments

  1. November 24, 2005

    Full disclosure I do this for a living, benefit materially, all opinions are mine and mine alone, not in any way connected to any device named here unless I explicitly say so, blah, blah, blah...

    "But what is it we have that's taking up all this disk space ? it?s video, music and photos. Things which, if just being played back, don?t need to be accessed very fast. If you're not editing video or music, in particular, you can handle having it on a very slow device. (Photos are a bigger issue, as they do sometimes need fast access when building thumbnails etc.)"

    I say that's not backup, it's archiving. Backups are for fast restores of operational data, getting you moving as quickly as possible after an event or failure, not near line (slow) file access. He's not talking about backup anymore he's talking about data classification and Storage Resource Management. Migrating data between different tiers of storage. Yes if you want to speed up your backups you should be pulling stuff off the backup target and moving it into an active archive.

    There are consumer level devices on the market in that space, such as Storage Central http://www.netgear.com/promotions/2005/storage_central.php , which can serve as easy to use RAID protected NAS target, it's just a bloody shame that in Storage Central's case it requires a Windows only driver to communicate with it.

  2. November 24, 2005

    wow, that Storage Central box does look cool; halfway there. Mind you, it's NetGear -- no bloody way I'll ever touch any of their crap again after suffering the MR814 router.

  3. November 24, 2005

    Oh I totally agree about NetGear, but the software for that particular product is OEM'd from a startup called Zetera, so it's bound to show up in another vendors offering at some stage.

    Maybe then someone will deliver OS X/Linux support (Translation: Tell Zetera to write the required code), and I could roll it out at home.

  4. December 5, 2005

    heh. Zetera are just down the road from me, here in Irvine! Must be a Maxtor spin-off.

    Bit of a turd of a review here -- some atrocious feedback in the comments. That's not a good sign, at all.

  5. December 6, 2005

    Ha. I knew the proprietary IP stuff, which is kludged UDP if I recall, was a pain in the ass but I didn't bloody well know that there wasn't any insight into or management of the unit.

    That's a data roach motel right there. I'll stick to RAID 1/RAID 10 and NFS/SMB for my home NAS needs.

  6. December 27, 2005

    dunno if its of interest but www.24sevenbackup.ie are doing an online backup product.

    at the moment i am giving it a whirl as a freebie for a year. i'll let you know how it goes when i start getting some serious usage of it.

    rwb

  7. December 27, 2005

    Hey Justin, it's a sign from god. Throw out all that smelly socialist Linux crap and run Windows so you can back it up. ;-p

  8. May 5, 2006

    Been there.....

    After going through the mill with norton ghost / Acronis / Verital......
    I ended up using the online option. Safer, automatic, email reporting and instant retrieval. I priced around and got the feeling that most of them were only agents and were using standard software. I ended up signing with backupanytime.com
    They weren't any cheaper but they support just about everything including mac and linux both of which i use and have little knoledge of (seperately...lol)
    Have it going now three months and I've tested it and the support. All round - excellent.

  9. January 10, 2007

    We signed up with backupanytime.com and yes it works perfectly. They introduced us to their sister company spamseal.com so we could ensure we were not paying to backup spam mail. We took this with a pinch of salt as we have tried many so called antispam solutions but guess what, we tried it and it worked -- perfectly -- we get no spam now and we have registered on some crazy sites to check it. Nice one guys.

  10. January 10, 2007

    Spam warning: "Jason" is advertising his own sites.

    whois irishmedical.com:
    Domain name: irishmedical.com

    Registrant Contact:
    O'Neill e-business
    John O'Neill (jon@galwaygigguide.com)
    +353.868101404
    Fax:
    22 Portacarron, Knocknacarra
    Galway, Galway -
    IE

    whois backupanytime.com:
    O'Neill e-business
    NA NA (NA)
    NA
    Fax:
    22 Portacarron, Knocknacarra
    Galway, GALWAY -
    IE

    whois spamseal.com:
    O'Neill e-business
    John O'Neill (jon@galwaygigguide.com)
    +353.868101404
    Fax:
    22 Portacarron, Knocknacarra
    Galway, Galway -
    IE

    Author : Jason (IP: 86.43.99.210 , 86.43.99.210)
    E-mail : info@irishmedical.com
    URI : http://www.irishmedical.com

    It looks like "B.T." is similarly a spammer, probably the same guy.
    Nice try, spammer....