When it comes to my data, I’m a suspenders and belt kinda’ guy. It can’t be in too many places or have too many layers of security. As with investing one’s hard-earned cash, diversification is critical to success. As such, I have loads of internal backup and security methods that are part of my routine. I ghost a copy of my primary drive in my desktop to an auxiliary drive inside the same machine; I have a Windows Home Server in my house which does a differential backup of my files every few days; and I even sync critical files with a USB memory stick that I can take with me if I need/want to. OK, maybe that’s a couple of sets of suspenders and a belt or two. What can I say?
I’ve been thinking about also syncing and backing up some data to the cloud over the last six months and took the plunge a couple of months ago. I’ve thought about what I really want out of cloud storage and have tried several offerings. I’ll talk about these specifically, but first, a little background on my thinking and what I was looking for.
It seems to me, the when it comes to the storage of data in the cloud, as opposed to the actually use of it, there are three general types of storage solutions – raw file storage, synced/backup file storage, and content-specific storage. Raw cloud-based file storage is simply disk space somewhere on the internet that you can do whatever you want with (think Amazon S3). Synced storage is similar, but it’s usually set up specifically to facilitate the synchronization or backup of data between a PC and disk space similarly elsewhere on the net. Content-specific storage is specifically set up for particular data types like email, photos, music, etc.
When cloud storage is segmented this way, one quickly realizes that all email users have been cloud storage consumers for a while. Whether you use a basic POP or IMAP server for your email or something heavier duty like Exchange or Notes, your email has been in the cloud at least for some period of time. So, you, like me, are already likely a user of cloud storage. This rationalization helped me feel more comfortable about moving my data to someplace unknown.
In the end, I found I was most interested in having storage for backups and syncing to keep multiple computers up to date. Services for the latter often assume the former – a cloud-based synced storage provider often has nice backup capabilities as well. After all, backup is the same storage mechanism without the sync function. I also wanted to expand my specialized storage to include my large photo collection. For this, I wanted a photo-specific site that offered galleries and photo management. These, of course, are not offered by the raw or synced backup folks.
While I hardly tried all services available, I did try a few including, Amazon S3, Microsoft’s Skydrive, Microsoft’s Live Mesh, Syncplicity, KeepVault, SmugMug and Flickr. Here are my thoughts:
- Amazon S3 – S3 is simply raw storage and it lies underneath many of the other, higher-level cloud storage services out there. There’s no high level interface per se and, as it states clearly on the Amazon AWS site, it’s “intentionally built with a minimal feature set.” At $0.15/GB/Month it isn’t even that cheap compared to some other services – 200GB of backup costs $360. Oh yeah, I can do basic math . . .
- SkyDrive – It’s “integrated” with Microsoft’s unbelievingly confusing array of Windows Live services. I consider myself pretty knowledgeable about Microsoft stuff, but this Windows Live thing is hard to understand. It works nicely, but there isn’t any client on the PC side, really. Uploading files is done a handful at a time and there is no syncing. It’s really about sharing files and doesn’t offer any automated backup or syncing. Even for bulk storage, it’s too difficult to use. They offer 25GB of storage for free.
- Live Mesh – I like Live Mesh a lot. Live Mesh is all about synchronization between multiple machines, including Macs (beta) and mobile phones (“soon”) as well as online through a web browser. It works totally behind the scenes, is fast and has the best reporting about what it did and what it’s doing of any service I tried. It also offers features like accessing the desktop of a Live Mesh-connected computer and a nice chatting and feedback facility for sharing and commenting on shared documents. My only problem with Live Mesh was the level of file granularity for syncing. Live Mesh only understand directories, not individual files. Sometimes, you just don’t want the entire directory synced. The initial 5GB of storage is free. It’s still in beta.
- Syncplicity – It’s my favorite of all the sync/backup solutions so far. It makes assumptions about the directories you want to sync or backup and adding different ones is a tad confusing, but once you get it, it’s all a piece of cake. The reporting on what it’s doing isn’t as nice as Live Mesh, but it’s just as seamless and it’s pretty fast (like Live Mesh). Unlike Live Mesh, individual files can be added or removed from a sync tree by right-clicking them (Windows) and just specifying whether or not the file should be included or not. Also, it’s easy to specify whether you want files to be synced with other machines or just backed up. I’m still not completely content with how Syncplicity deals with conflicts. No data is ever lost, but it can be duplicated leaving copies scattered in your directories. Also, I had one really nasty problem with the service. The Syncplicity client was sucking up 10%-50% of the CPU time on my machine – all the time. I sent messages to Syncplicity support and complained about the problem on their forum. Nothing, zero, no response for weeks. In fact, to this day, I’ve gotten no response. I eventually figured the problem out myself. A TrueCrypt encrypted volume in a directory on my machine was screwing the client up. Once removed from the sync tree, the problem was gone. Just horrible service. There is a free 2GB trial and then $99/year for the first 100GB. This is a 50% discount offer that’s been running for a while.
- KeepVault – I tried this out because it integrates nicely with the Windows Home Server Console. I’m using it specifically to back up my server – no desktops included and no synchronization, just backup. It seems to work well, but the initial backup of 150GB of data took about 16 days even when I was not throttling the speed of the connection (a nice option for a server, BTW). Additionally, the backup process stalled about 20 times during the initial backup. Now that it’s only dealing with a handful of files, albeit big ones, at a time, it seems to be working well. Jury’s still out. No trial, but a 30-day money-back guarantee. $180 for 200GB of backup.
- SmugMug – I have 42GB of photos on my server which represent the most cherished of all data I have. At the very least, I needed to backup these files to another physical location. At best, it would be nice if the data could be organized and viewed from that location as well. I looked at many sites, including Flickr (the relative standard in this space) and chose SmugMug. The difference is that SmugMug is aimed at photographers who at least think there is some level of professionalism in their shots. SmugMug’s pages are totally customizable and they understand not to mess with pictures being uploaded (unless you want them to). It’s about the gallery first and about sharing second. Just what I wanted – I’ve never learned how to share well 🙂
There are loads of other services out there including some I considered, but decided not to try on this first pass – DropBox, ZumoDrive, iDrive, Soonr, Jungle Disk, etc. In general, I’m feeling better about having my data somewhere else. The process is easy and, as far as I can tell, secure. Syncing can certainly get better, though, and when there’s a failure, it’s very hard to debug, even if you can detect that it happened in the first place. Sometimes, as with any backup, you don’t know there was a problem until an emergency happens and you really need to restore a file. Not painless, but fairly low barriers to experience. Come on in, the water’s fine . . . so far.
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=fa95b143-89d4-464e-bfe7-d150bcdc2832)
Will – regarding your problem with the TrueCrypt drive: was the problem with TrueCrypt or with Syncplicity’s inability to accommodate it? We’re about to dive into using TrueCrypt for some things (although the volumes will not be mounted most of the time).
Will – regarding your problem with the TrueCrypt drive: was the problem with TrueCrypt or with Syncplicity’s inability to accommodate it? We’re about to dive into using TrueCrypt for some things (although the volumes will not be mounted most of the time).
Now you’re talking about something I can understand, Will! I’ve been writing about this stuff at my blog for a while now, and you’re dead on with your observations.
One thing to point out is that Amazon S3 and Nirvanix aren’t aimed at the consumer/prosumer/soho/whatever market you’re addressing. They’re offering more managed services for the SMB market (Amazon) or the enterprise (Nirvanix) and the price reflects that. Also, that “raw capacity” market isn’t quite what most home folks are looking for: It’s API-driven rather than a drive or sync target, so you have to either rewrite your application or use an interface layer like JungleDisk (Amazon/Rackspace) or CloudNAS (Nirvanix) to allow old apps to get data into the cloud.
The geek in me wants an rsync target, though!
Now you’re talking about something I can understand, Will! I’ve been writing about this stuff at my blog for a while now, and you’re dead on with your observations.
One thing to point out is that Amazon S3 and Nirvanix aren’t aimed at the consumer/prosumer/soho/whatever market you’re addressing. They’re offering more managed services for the SMB market (Amazon) or the enterprise (Nirvanix) and the price reflects that. Also, that “raw capacity” market isn’t quite what most home folks are looking for: It’s API-driven rather than a drive or sync target, so you have to either rewrite your application or use an interface layer like JungleDisk (Amazon/Rackspace) or CloudNAS (Nirvanix) to allow old apps to get data into the cloud.
The geek in me wants an rsync target, though!
Most of us believe we want “backups” and think about daily, weekly, incremental, full, etc.. But what we really want are “restores.” We want to restore an accidentally deleted file, a failed disk drive, or all the data wiped out in a fire in the home, stolen laptop, etc.. Rarely we want to restore data after an earthquake or flood taking out the data center.
Most of the “restores” we want are of the form “I had the file in this folder two or three days ago and now it’s gone/wrong/messed-up.” Or “My disk crashed and I need the data that were on it.”
When you look at the restore capabilities instead of “backup” you want online snapshots locally (your shadow disks) and you want off-site slowly trickling large cloud storage like carbonite or live mesh.
Most of us believe we want “backups” and think about daily, weekly, incremental, full, etc.. But what we really want are “restores.” We want to restore an accidentally deleted file, a failed disk drive, or all the data wiped out in a fire in the home, stolen laptop, etc.. Rarely we want to restore data after an earthquake or flood taking out the data center.
Most of the “restores” we want are of the form “I had the file in this folder two or three days ago and now it’s gone/wrong/messed-up.” Or “My disk crashed and I need the data that were on it.”
When you look at the restore capabilities instead of “backup” you want online snapshots locally (your shadow disks) and you want off-site slowly trickling large cloud storage like carbonite or live mesh.
Dave,
I’m not exactly sure what the TrueCrypt/Syncplicity problem is. Syncplicity has never got back to me on it – crappy support. I’m guessing that Syncplicity is just looking at a Windows event trigger that’s telling them there has been a file change in one of the directories they watch. Perhaps when unmounted, TrueCrypt is triggering such a change. BUT, it’s only happening on a 64-bit Vista machine. No problems on my laptop with a TrueCrypt volume running Windows 7.
Sorry, not much help . . .
Dave,
I’m not exactly sure what the TrueCrypt/Syncplicity problem is. Syncplicity has never got back to me on it – crappy support. I’m guessing that Syncplicity is just looking at a Windows event trigger that’s telling them there has been a file change in one of the directories they watch. Perhaps when unmounted, TrueCrypt is triggering such a change. BUT, it’s only happening on a 64-bit Vista machine. No problems on my laptop with a TrueCrypt volume running Windows 7.
Sorry, not much help . . .
Stephen,
Great summary! Yup, rsync basically covers the bases 🙂 There are some very good, smart synchronization tools for Windows. I use Beyond Compare, for example. I still need to be able to give it a target it understands, though – a Windows share, a mounted volume or an FTP site. AS you say, thus the services lying on top of the raw providers.
Stephen,
Great summary! Yup, rsync basically covers the bases 🙂 There are some very good, smart synchronization tools for Windows. I use Beyond Compare, for example. I still need to be able to give it a target it understands, though – a Windows share, a mounted volume or an FTP site. AS you say, thus the services lying on top of the raw providers.
Mitch,
In terms of backup, I totally agree – restores *are* what’s important. Personally, though, I like synchronization at least as much. Of course, I run multiple machines and have a server, since I “backup” files among these already and transparently (think Windows Home Server which, I wish would move to Server 2008 already, BTW), I already have some diversification and multiple restore locations. Good and transparent sync service (like Live Mesh, as you say) make life with multiple machines WAY easier.
Mitch,
In terms of backup, I totally agree – restores *are* what’s important. Personally, though, I like synchronization at least as much. Of course, I run multiple machines and have a server, since I “backup” files among these already and transparently (think Windows Home Server which, I wish would move to Server 2008 already, BTW), I already have some diversification and multiple restore locations. Good and transparent sync service (like Live Mesh, as you say) make life with multiple machines WAY easier.
Will,
Nice writeup. If you are suspender and belts, then what am I? With NO access to the cloud at sea (or in remote ports) I’ve never had the chance to play around with any of it. You wouldn’t believe how many people don’t even backup their computers while sailing, and it plays such a critical role: communication, navigation, and memories (photos, videos, journals etc). It’s NOT like a boat and the salt air environment is the optimal for long-term electronics life. Even if you do have a backup it’s right there with the primary source .. all ready to go down with the ship.
We keep our backup drives in a watertight case in the abandon ship back (as if THAT will help).
FriendFeed Discussion
Will,
Nice writeup. If you are suspender and belts, then what am I? With NO access to the cloud at sea (or in remote ports) I’ve never had the chance to play around with any of it. You wouldn’t believe how many people don’t even backup their computers while sailing, and it plays such a critical role: communication, navigation, and memories (photos, videos, journals etc). It’s NOT like a boat and the salt air environment is the optimal for long-term electronics life. Even if you do have a backup it’s right there with the primary source .. all ready to go down with the ship.
We keep our backup drives in a watertight case in the abandon ship back (as if THAT will help).
FriendFeed Discussion
Will,
I have two questions, I was looking at dropbox and it seems their block level handling of files is the better way to go on frequently modified work files. Would this mean I can park my Outlook .pst file there? Of course this is probably one of the most bass-ackwards way of having e-mail being synced on two or three different machines. One of the other services iDrive implies they can handle that.
Second, how do these folks handle storage discrepencies on different devices? Say I want (what is essentially) a copy of a set of work files in a folder that is 40 GB and I am using a netbook with 8GB space? I am only working with less than 100MB of files at any given time, so I sort of just need an “image” of what is in the “folder in the clouds”.
Ideally, I suppose, what I am looking for is an offsite “network drive”, but I haven’t quite figured out how they resolve “syncing” multiple devices that may not have equal/sufficient storage space. 90% of the time I can probably connect and get a current copy of what I need, work on it and leave the bulk of the other files in the cloud, I just want to have some way of specifying certain files be available off-line ahead of time (like say I am going to be working on Project X while I am watching the kids at the park). I would want all of project X folder to be on my local drive of my netbook, I don’t need Project A thru W on the local netbook drive (which wouldn’t fit anyway). When I return to the house, I would be back on-line and it would sync back up to the cloud.
Does that make sense?
John
Will,
I have two questions, I was looking at dropbox and it seems their block level handling of files is the better way to go on frequently modified work files. Would this mean I can park my Outlook .pst file there? Of course this is probably one of the most bass-ackwards way of having e-mail being synced on two or three different machines. One of the other services iDrive implies they can handle that.
Second, how do these folks handle storage discrepencies on different devices? Say I want (what is essentially) a copy of a set of work files in a folder that is 40 GB and I am using a netbook with 8GB space? I am only working with less than 100MB of files at any given time, so I sort of just need an “image” of what is in the “folder in the clouds”.
Ideally, I suppose, what I am looking for is an offsite “network drive”, but I haven’t quite figured out how they resolve “syncing” multiple devices that may not have equal/sufficient storage space. 90% of the time I can probably connect and get a current copy of what I need, work on it and leave the bulk of the other files in the cloud, I just want to have some way of specifying certain files be available off-line ahead of time (like say I am going to be working on Project X while I am watching the kids at the park). I would want all of project X folder to be on my local drive of my netbook, I don’t need Project A thru W on the local netbook drive (which wouldn’t fit anyway). When I return to the house, I would be back on-line and it would sync back up to the cloud.
Does that make sense?
John
Chris,
If I am a suspenders and belt kinda guy, you’re just not wearing any pants at all. Anyone who owns a box they refer to as “the abandon ship [p]ack” is probably not a person who spends a lot of time consumed with the thought of risk.
Have you ever considered backup up to a USB thumb drive and mailing it to yourself back home (or a friend who could Fedex back to you when you need it) when you stop somewhere? I know it’s not very frequent, but at least you get the data off the boat and it wouldn’t cost too much – your really critical data should fit in a 16GB thumb drive I’d suspect.
John,
I don’t know about DropBox, but it’s pretty similar to SyncPlicity. The “block” level syncing may help with syncing PST files, but I wouldn’t bank on it. Depending on how you use Outlook, your PST file may be changing constantly and that may cause problems – even if they’re not major, you may not know if you’re completely in sync at any particular time. However, the differential syncing is a nice feature that SyncPlicity does not yet have.
I think you can pretty easily get donw what you’re looking for with respect to different size drives. I’ll speak to SyncPlicity, although it appears that DropBox does the same thing. Sync *everything* you may be interested in sharing with the SyncPlicity server from your biggest machine. Then, from every other machine, specify which files and directories you want synced with that particular machine. you don’t have to sync all files/directories with every machine. Even if you don’t know what you want at any time, you can sync different files/directories at different times. That said, depending on the amount of data, this could take some real time. It’s be cheaper to get a bigger disk on those small machines
Both SyncPlicity and DropBox have 2GB trial accounts. Download them both and give ’em a try.
Hope that helps.
John,
I don’t know about DropBox, but it’s pretty similar to SyncPlicity. The “block” level syncing may help with syncing PST files, but I wouldn’t bank on it. Depending on how you use Outlook, your PST file may be changing constantly and that may cause problems – even if they’re not major, you may not know if you’re completely in sync at any particular time. However, the differential syncing is a nice feature that SyncPlicity does not yet have.
I think you can pretty easily get donw what you’re looking for with respect to different size drives. I’ll speak to SyncPlicity, although it appears that DropBox does the same thing. Sync *everything* you may be interested in sharing with the SyncPlicity server from your biggest machine. Then, from every other machine, specify which files and directories you want synced with that particular machine. you don’t have to sync all files/directories with every machine. Even if you don’t know what you want at any time, you can sync different files/directories at different times. That said, depending on the amount of data, this could take some real time. It’s be cheaper to get a bigger disk on those small machines
Both SyncPlicity and DropBox have 2GB trial accounts. Download them both and give ’em a try.
Hope that helps.
Will,
Maybe I’m wearing shorts.. with a stainless steel cage for underwear. I am consumed with the thought of risk, it’s just easier to plan for because it is so final. There is no maybe someone can recover my lost data.. it’s gone!!
I sync critical data to thumb drive(s) on a daily basis, so my work ALWAYS lives in at least two places. We usually use DVDs to send home because 1GB sticks are NEW where we travel. We just have way too much “critical” data, last time we were home we left a 500 GB drive filled with photos, video, journals etc.. and I think that would be small for the average person.
How many “average people” don’t back up their data anywhere? If they do, are they saving monthly backups off-site? Seems like a pretty HUGE untapped market, especially for solutions that are seamless and unobtrusive; i.e. my Dad could use it .. or better yet, he didn’t even know he was.
I’ll leave you to the topic at hand and sorry for the detour. I just find it all so fascinating.. Maybe I’ll start a new blog called “It’s all new to me” – “Technology reviews and observations from a high tech cast away”. Think anyone would read it? The first post would be “Observations from a Twitter Virgin”.
Will,
Maybe I’m wearing shorts.. with a stainless steel cage for underwear. I am consumed with the thought of risk, it’s just easier to plan for because it is so final. There is no maybe someone can recover my lost data.. it’s gone!!
I sync critical data to thumb drive(s) on a daily basis, so my work ALWAYS lives in at least two places. We usually use DVDs to send home because 1GB sticks are NEW where we travel. We just have way too much “critical” data, last time we were home we left a 500 GB drive filled with photos, video, journals etc.. and I think that would be small for the average person.
How many “average people” don’t back up their data anywhere? If they do, are they saving monthly backups off-site? Seems like a pretty HUGE untapped market, especially for solutions that are seamless and unobtrusive; i.e. my Dad could use it .. or better yet, he didn’t even know he was.
I’ll leave you to the topic at hand and sorry for the detour. I just find it all so fascinating.. Maybe I’ll start a new blog called “It’s all new to me” – “Technology reviews and observations from a high tech cast away”. Think anyone would read it? The first post would be “Observations from a Twitter Virgin”.
I’ve had good success with Syncplicity as a single PC home/office user, but it does not deal with real-time syncing or backing up of Outlook (PST) files. Any suggestions? I don’t want the expense of Microsoft Exchange for just me, so it’s a cost issue as much as anything. Google just started an Apps that allows for syncing of Outlook, but it seems geared towards larger users such as the requirement of having a domain name. Any suggestions for a cloud application for real-time backup/sync of Outlook for an individual is much appreciated.
I’ve had good success with Syncplicity as a single PC home/office user, but it does not deal with real-time syncing or backing up of Outlook (PST) files. Any suggestions? I don’t want the expense of Microsoft Exchange for just me, so it’s a cost issue as much as anything. Google just started an Apps that allows for syncing of Outlook, but it seems geared towards larger users such as the requirement of having a domain name. Any suggestions for a cloud application for real-time backup/sync of Outlook for an individual is much appreciated.
James,
I understand what you’re trying to do and I’m sure something does it, I just don’t know what it is. There are so many changes to PST files undere the covers, that I would be worry that something didn’t sync up before I shut down and I would be propagating incomplete data to another machine and beyond.
Have you looked at SherWeb? They do Exchange hosting for $8.95/month with a 3GB mailbox. It’s not free, but it’s reasonable. I have used them for over a years and I’m pretty happy with them. Their uptime is great and their speed is reasonable. No one comes close to their price as far as I know. he only downside is they use Barracuda for spam filtering which stinks. They don’t even have contextual filtering, just bayes and blacklists. I am not affiliates with the company FWIW.
The other reason to use Exchange instead of trying to sync PST files is that you can easily coordinate with mobile devices when you use Exchange.
Good luck. If you find anything that does the trick for you, please pass it along.
James,
I understand what you’re trying to do and I’m sure something does it, I just don’t know what it is. There are so many changes to PST files undere the covers, that I would be worry that something didn’t sync up before I shut down and I would be propagating incomplete data to another machine and beyond.
Have you looked at SherWeb? They do Exchange hosting for $8.95/month with a 3GB mailbox. It’s not free, but it’s reasonable. I have used them for over a years and I’m pretty happy with them. Their uptime is great and their speed is reasonable. No one comes close to their price as far as I know. he only downside is they use Barracuda for spam filtering which stinks. They don’t even have contextual filtering, just bayes and blacklists. I am not affiliates with the company FWIW.
The other reason to use Exchange instead of trying to sync PST files is that you can easily coordinate with mobile devices when you use Exchange.
Good luck. If you find anything that does the trick for you, please pass it along.
Chris,
If I am a suspenders and belt kinda guy, you're just not wearing any pants at all. Anyone who owns a box they refer to as “the abandon ship [p]ack” is probably not a person who spends a lot of time consumed with the thought of risk.
Have you ever considered backup up to a USB thumb drive and mailing it to yourself back home (or a friend who could Fedex back to you when you need it) when you stop somewhere? I know it's not very frequent, but at least you get the data off the boat and it wouldn't cost too much – your really critical data should fit in a 16GB thumb drive I'd suspect.