The Survival Podcast Forum

Survivalism & Self Sufficiency Topics => Homesteading and Self Reliant Living => Home And Business Security => Topic started by: FreeLancer on August 02, 2015, 02:35:01 AM

Title: Data Backup Strategies
Post by: FreeLancer on August 02, 2015, 02:35:01 AM
While not a particularly sexy survival topic, we've all got data that is our responsibility to protect, from important business files to family photos and home videos.  I searched the site and couldn't really find a prior discussion, so I thought I'd start one since I've been thinking about it this last week.

I've heard that a good rule of thumb is to not consider your files sufficiently backed up until they exist in at least three separate forms, with at least one located off site from your computer.  That's the tough part.  I guess a decade ago it wasn't that hard to load stuff on CD-ROM or DVD discs and mail them to grandma's house, but multimedia files are so large that you'd have to send it to her on a couple-terabyte hard drive now.

Cloud storage sounded promising as an offsite backup mode, but downloading large files is tedious on a home broadband connection and uploading them in the first place is even worse.  It's one thing to store Office documents online at Google Drive, but that doesn't work well for those times when you need to move large media files, like all your family photos, in bulk.  Plus, what if you don't have an internet connection and you really need you're files? 

So, to satisfy the offsite portion of a good backup strategy, without waiting around for the internet, does that mean we're stuck packing a hard drive to and from separate locations, or sticking them in a good fire safe?  The company ioSafe makes several fire and waterproof external drive and NAS units, and they apparently actually survive 30 minutes of house fire and 3 days of submersion in salt water.  I thought it was gimmicky, but there's enough people doing torture tests on Youtube (https://youtu.be/u2yEVUMQyZY) that I'm thinking it is actually the real deal.  The problem, though, is they are expensive, like 3-4 times the cost of their pedestrian counterparts.

The other issue that I always worry about is the uncertainty over hard drive failure.  A brand new drive could fail within three weeks, or it could last 8 years, but once you get to about 5 years you pretty much know you can't really trust much to it, anymore.  The way around this is having multiple backup drives, or expensive/complicated RAID systems with one or two disk redundancy built in.  But if you read much about those systems, you'll hear the line about how you need to back them up, too, because there might be an error in rebuilding the array after swapping drives.  At some point you just have to say my one backup of my computer, plus a backup of the backup, and maybe whatever I've managed to upload to the cloud, and that's it.  Otherwise you'll just go insane.

What are the rest of you doing?
Title: Re: Data Backup Strategies
Post by: artephius on August 02, 2015, 08:59:04 AM
I too have not yet come up with a good solution for offsite backup... that's a tough nut to crack when you're dealing with terabytes.

For now 3 drives of the same size, with a simple script that clones them from one to the others (using rsync) every night is my solution. I also never buy the same brand for all three or even two of them if I'm buying at the same time. I figure two identical drives purchased at the same time are more likely to fail at the same time. I've had plenty fail, but only one at a time.

Eventually (if I ever have enough extra money)I plan to set up a larger nas/raid array and run some sort of incremental backup software on a daily/weekly basis. Backups aren't just for harddrive failures, there's nothing worse than realizing a day too late that you stupidly deleted something you actually needed, and your wonderful backup script deleted it from your backup drives as well! Sometimes I keep old crappy drives around and use them as a forth point of backup that I manually run every now and then (less often than my normal backups) just in case something like that happens -- sometimes I get lucky.

Also on linux there are the smartmon tools which allow you to read the SMART self diagnostic data from the drive itself (power on hours, reboot count, error count, reallocated sectors count, temperature, etc) and keeping an eye on those numbers from time to time is a good way to know when you got one that's about to die or start corrupting your data. I don't know if something similiar exists for windows/mac but it probably does. The hard drive itself keeps track of those things internally, you just need the appropriate software to read it.
Title: Re: Data Backup Strategies
Post by: Carl on August 02, 2015, 11:52:09 AM
I use three of the 3 Terabyte USB drives and the on PC files are copied to the USB drive when I plug them in.
I have one connected and one in a fire box in my auto and one in my BOL (nearby)
and each time I go to the auto...I trade with the house drive and when I go weekly to the BOL,I trade with the auto's drive...
this way I get good rotation and none of the drives see much use as they alert me when in sync and I remove them from the PC which has TWO duplicate internal drives.

I keep all personal files,photos ,etc and a full set of install files for my software PLUS the DOCUMENTS folder from each of my in use PC's
so if both PC's fail or fry,I have in house,in auto,and in BOL backup. I buy NEW USB drives every 4 years or so and "OLD" drives go as archives to the BOL where I have plenty or room for storage in a secure,controlled environment.

My files are mostly personal documents and photos ,plus a lots of PDF and video collection and are of little value to anyone other than me...but I have time and have relied on a backup more than once due to remote scanning customer PC's for virus etc for businesses too small to pay an IT person ( a little side job I earn a bit from)

Title: Re: Data Backup Strategies
Post by: nkawtg on August 02, 2015, 01:25:29 PM
If you have terabytes of multimedia files you only need to back it all up one time to store off site, after that use an incremental approach to your backups (only the files that change or are added).
If you backup your multimedia files to the cloud, it will take a long time but once it's done only changed or added files are backed up.

At work my backup scheme is disk to disk, disk to tape, and cloud.
Title: Re: Data Backup Strategies
Post by: jerseyboy on August 02, 2015, 01:49:11 PM

For now 3 drives of the same size, with a simple script that clones them from one to the others (using rsync) every night is my solution. I also never buy the same brand for all three or even two of them if I'm buying at the same time. I figure two identical drives purchased at the same time are more likely to fail at the same time. I've had plenty fail, but only one at a time.


If you are on Linux, you can make those three drives into a RAID1, which is a mirror, using mdadm.  Even though it is called a mirror, you can have as many drives as you want in the mirror and it will still function with only one drive if all of the others fail.  Write speed will be the same but read speed will increase with more drives in the array.  Each drive would always be up to date.  Also, you can set up mdadm to email you when one of the drives fail.

The only problem is converting what you have now to the mdadm RAID1 array.  Best thing to do would be to buy at least one new drive and take one of the old drives and make a 2-way mirror and copy all of the data over.  Then take a second old drive and add it to the RAID1 array.  Then take the last remaining old drive and store it offsite.

Also, RAID, even a 3-way mirror, is not considered a backup.  Which I know you know, but just for others' reading pleasure.

Jerseyboy
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 02, 2015, 02:13:30 PM
Also, RAID, even a 3-way mirror, is not considered a backup.  Which I know you know, but just for others' reading pleasure.

I have a vague idea of why this is the case, but can you explain why? 
Title: Re: Data Backup Strategies
Post by: Carl on August 02, 2015, 02:24:59 PM
I have a vague idea of why this is the case, but can you explain why?

TRUE backup (in my opinion) has to not be tied to power,or PC ,or on site .
To avoid POWER SURGE,LIGHTENING,FIRE,and VIRUS along
with all of the things that can cause local data loss or theft.
Title: Re: Data Backup Strategies
Post by: artephius on August 02, 2015, 02:36:22 PM
TRUE backup (in my opinion) has to not be tied to power,or PC ,or on site .
To avoid POWER SURGE,LIGHTENING,FIRE,and VIRUS along
with all of the things that can cause local data loss or theft.

I'd agree with this, and add that RAID (1) also suffers from the same problem as my current setup... no incremental/historical backups. ie, it'd be nice to be able to go back in the 'history' of your backups to say a month ago when you had a file you've since ruined or deleted... much like version control for software development. I need to do some research but I'm fairly certain such solutions exist.

Title: Re: Data Backup Strategies
Post by: artephius on August 02, 2015, 03:23:23 PM
rdiff-backup looks like a good free solution. I've never used it so I can't vouch for how well it works, but it sounds pretty good.

Perhaps a good setup would be a local RAID 1 with two or three drives, used to store the incremental backups created by rdiff-backup. The RAID 1 would provide protection against any one drive failing, and rdiff-backup would provide the "time-machine" style backups.

I do also like the idea of read performance increases with the raid array so it may even be worth taking it a step further and setting up 2 local RAID arrays, one for the primary system, and the second for the backup, which would add even more redundant protection against hard drive failure... (Yeah, I've killed a lot of hard drives in the past.. I don't trust them at all  >:() and give read performance a little boost on the primary system.

A second server/nas at your BOL (if you are fortunate enough to have one) or somewhere else off-site could also have a RAID 1 that stores a second copy of the rdiff-backup snapshots -- changed files/diffs could be transmitted securely to the BOL over the internet via ssh.

A third (or forth, or more) backup location could be usb drives kept in different locations. Each created and updated as often as you see fit, again using rdiff-backup for the incremental snapshots.

I can't imagine how you could possibly lose anything with a setup like that.
Title: Re: Data Backup Strategies
Post by: Carl on August 02, 2015, 03:28:52 PM
I'd agree with this, and add that RAID (1) also suffers from the same problem as my current setup... no incremental/historical backups. ie, it'd be nice to be able to go back in the 'history' of your backups to say a month ago when you had a file you've since ruined or deleted... much like version control for software development. I need to do some research but I'm fairly certain such solutions exist.

I run reliable old school Windows XP as the few problems I ever had came from MicroSoft Update . SYSTEM RESTORE is handy ...
But an old program called CASPER does either FULL BOOTABLE DRIVE COPY or INCREMENTAL file copy...BUT if you get digital
virus or malware....you wind up copying that too...so a full scan BEFORE incremental ,or full,backup is needed.
Title: Re: Data Backup Strategies
Post by: artephius on August 02, 2015, 06:26:31 PM
I run reliable old school Windows XP as the few problems I ever had came from MicroSoft Update . SYSTEM RESTORE is handy ...
But an old program called CASPER does either FULL BOOTABLE DRIVE COPY or INCREMENTAL file copy...BUT if you get digital
virus or malware....you wind up copying that too...so a full scan BEFORE incremental ,or full,backup is needed.

That's a good point, as a linux user I tend to forget about viruses... its not really an issue on linux. I could be wrong but I would think that if you had a known "clean" full or incremental backup the virus shouldn't be able to infect that especially if it was compressed and had some sort of checksum.

As far as I know viruses are just malicious programs that can't actually do anything unless they are actually running -- ie a malicious DLL replacing the real one in your system32 folder on windows that gets executed when windows starts up and goes on to cause whatever damage it's meant to do. Even if it were to be copied to your backup it shouldn't be able to actually do anything unless you were to completely restore your system from that particular backup and reboot, and therefore "activate" the virus.

I would think that if you can scan your system and find the virus like you normally would, you should be able to also scan your backups and eliminate it the same way.

I'm just guessing here though. Viruses are NOT my specialty.
Title: Re: Data Backup Strategies
Post by: Carl on August 02, 2015, 06:51:26 PM
Well I am glad you told me about this ,I have just spent too much time worrying about it.
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 02, 2015, 07:04:38 PM
Would any of you consider an ioSafe product in the house, in combination with another device, as an offsite backup equivalent? 

For example, say the complete system includes the computer's internal drive with an attached RAID 0 device holding the large media files, then a RAID 1 or 5 device set to store incremental Time Machine backups of the previous two drives, and finally an ioSafe device that also gets the same incremental Time Machine backup but is able to survive fire and flood damage. 
Title: Re: Data Backup Strategies
Post by: jerseyboy on August 02, 2015, 10:21:17 PM
I have a vague idea of why this is the case, but can you explain why?

A quick example is it you over write a file on the RAID array, it is duplicated on all three drives and the file is gone.  An offsite backup would still have the old file.

Jerseyboy
Title: Re: Data Backup Strategies
Post by: jerseyboy on August 02, 2015, 10:32:43 PM
I'd agree with this, and add that RAID (1) also suffers from the same problem as my current setup... no incremental/historical backups. ie, it'd be nice to be able to go back in the 'history' of your backups to say a month ago when you had a file you've since ruined or deleted... much like version control for software development. I need to do some research but I'm fairly certain such solutions exist.

What you are looking for is the ability to do a snapshot. Some file systems that do this are ZFS, BTRFS, and NTFS. The NTFS one is called volume shadow copies. You can force a snapshot in window 7 premium/ultimate. Not sure about windows 8 or 10.

Jerseyboy
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 02, 2015, 10:39:47 PM
A quick example is it you over write a file on the RAID array, it is duplicated on all three drives and the file is gone.  An offsite backup would still have the old file.

But if you're using RAID for Time Machine (Mac's incremental backup) would it still not considered a backup? 
Title: Re: Data Backup Strategies
Post by: artephius on August 03, 2015, 07:46:19 AM
But if you're using RAID for Time Machine (Mac's incremental backup) would it still not considered a backup?

I think the Time Machine snapshots would be considered a backup (at least a local one) whether on RAID or not, but the RAID array itself wouldn't be a backup technically, just a hard drive redundancy.

What you are looking for is the ability to do a snapshot. Some file systems that do this are ZFS, BTRFS, and NTFS. The NTFS one is called volume shadow copies. You can force a snapshot in window 7 premium/ultimate. Not sure about windows 8 or 10.

Jerseyboy


Thanks I'll look into that. I've always stuck with ext for simplicity sake, never had a compelling reason to use anything else.

I like the idea of a single full snapshot, with incremental "snapshots" being either hardlinks to non changed files or some sort of deltas/diffs to save space... I never seem to have enough space...
Title: Re: Data Backup Strategies
Post by: archer on August 03, 2015, 09:28:15 AM
someone I work with recommended backblaze.com for an offiste backup solution. I've not tired them so I cannot tell how it works.
Title: Re: Data Backup Strategies
Post by: outoforder2day on August 03, 2015, 09:48:22 AM
someone I work with recommended backblaze.com for an offiste backup solution. I've not tired them so I cannot tell how it works.
I've used Backblaze and recommend them. I'll write up my dream backup system, which I'm in the process of putting together, in another post.
Title: Re: Data Backup Strategies
Post by: outoforder2day on August 03, 2015, 10:42:45 AM
Systems:
In the family, we have:

Backup Goals:
My backup goals are as follows:

How I do it now:
Quite simply, I don't. I have a collection of hard drives that I make a best effort to keep updated and moved between my house and my grandmothers. The current setup sucks, hence my design of the new setup.

New design:

Explanation of setup:
All local systems will back up to the local NAS device using sync tools (QNAP Sync (https://www.qnap.com/i/en/trade_teach/con_show.php?op=showone&cid=62), for example). This way you have a near-real-time copy of your files on your backup device. This will not prevent you from deleting a local file, though, and having it delete on the backup device. Some have an option to keep deleted files for X days, and I'd recommend setting that to at least retain for two days.  That's what nightly copies are for. Using the supplied backup software, or Cobian Backup (http://www.cobiansoft.com/cobianbackup.htm), have all internal systems back up to another section of the NAS device. I would use a weekly full backup with daily incremental to reduce the amount of data transfer. (I don't trust the perpetual incremental backup that many of these devices offer, but I might be swayed in the long term.) Once the individual jobs are complete, configure the NAS itself to run a daily sync to your cloud service (or your buddy's house). This should only be your nightly job, and not the synchronized files. It should help you save on space. Once that's complete, another job should kick off and backup the NAS to the ioSafe drive.

If you lose a laptop, you've got your data on the local NAS. Awesome! Quick restore of all files.
If you accidentally delete a file, no problem. You can search your NAS' archive and restore it!
If your house burns down, you might be able to do a quick restore from your ioSafe. Great!
If not, you still have everything stored at a cloud provider or your buddy's house.

This setup isn't exactly simple, or cheap. But it puts an enterprise level backup system at your disposal. It also provides a good place to keep your pictures, movies, and other data so that it isn't always on your desktop or laptop in the first place. I mean, what happens if someone steels your device and now has all your tax documents, or worse. That kind of stuff is usually better on a NAS. Preferably and encrypted one. Both QNAP and Synology support encryption, btw. As do most cloud providers like Amazon.
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 03, 2015, 11:49:25 AM
This setup isn't exactly simple, or cheap. But it puts an enterprise level backup system at your disposal. It also provides a good place to keep your pictures, movies, and other data so that it isn't always on your desktop or laptop in the first place. I mean, what happens if someone steels your device and now has all your tax documents, or worse. That kind of stuff is usually better on a NAS. Preferably and encrypted one. Both QNAP and Synology support encryption, btw. As do most cloud providers like Amazon.

+1 on that write up.  Now we're getting somewhere.

My Synology DS1515+ and 5 x 4 TB drives have just arrived, and with your instructions I now know what to do with them.  I'd really like the corresponding ioSafe 1515+ to match it to and connect as a failover, but that's an obscene amount of money.
Title: Re: Data Backup Strategies
Post by: outoforder2day on August 03, 2015, 11:55:18 AM
+1 on that write up.  Now we're getting somewhere.

My Synology DS1515+ and 5 x 4 TB drives have just arrived, and with your instructions I now know what to do with them.  I'd really like the corresponding ioSafe 1515+ to match it to and connect as a failover, but that's an obscene amount of money.

Honestly, you don't need the ioSafe. It's really just a local point in time cache of the Synology. For that, any external drive will work. The only difference from my "ideal" solution is that, if you have a fire (god forbid), you'd have to restore from the cloud provider. Honestly, the ioSafe may not be accessible after a fire, and may have to be sent back to them for recovery (that's the little blip they don't advertise). They won't charge you for the recovery, but it'll take time.

In fact, now that I think about it even more... I don't know if it's worth the price if you're going to do the cloud backup option. Hrmm....
Title: Re: Data Backup Strategies
Post by: Docwatmo on August 03, 2015, 12:03:01 PM
I use a round robin approach on my home network that includes both an onsite and linked offsite NAS units. 
Each computer (3 desktops, 3 laptops) backs up to free drive space on one of the other computers each night (Just document/changed file backups).  Then each runs a full backup (including the backup files from the other devices) to the local NAS device weekly and Monthly.  The local NAS device has a VPN connection to a remote NAS device I keep at work.  Both NAS devices are synced constantly.    Right now the NAS Devices are holding approximately 4TB of data.  (Lots of Multimedia). 

I also dupe Critical files to a couple of 1TB USB hard drives that float between work, home and the in-laws house.   

The reason for this much backup.  We lost a hard drive in my wife's computer many years ago that had 200,000 pictures.  I had slacked at checking her backups and found out the hard way that her backup had been disabled.  So there were no duplicates or backups of about 40,000 of those pictures.  We had to send the hard drive to a recovery service (About $2700.00 later) we got the data back.  Only lost about 5 images that were corrupt.  Everything else was salvaged luckily.   

Eddted to add:  I use the QNAP NAS Devices.  ROCK solid and lots of great features.

So you learn that lesson once.
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 03, 2015, 12:24:05 PM
Honestly, you don't need the ioSafe. It's really just a local point in time cache of the Synology. For that, any external drive will work. The only difference from my "ideal" solution is that, if you have a fire (god forbid), you'd have to restore from the cloud provider. Honestly, the ioSafe may not be accessible after a fire, and may have to be sent back to them for recovery (that's the little blip they don't advertise). They won't charge you for the recovery, but it'll take time.

In fact, now that I think about it even more... I don't know if it's worth the price if you're going to do the cloud backup option. Hrmm....

Maybe the better thing is to get a single drive ioSafe and have it set to regularly clone my iMac drive and as much of my multimedia files as I can fit on its 4TB drive. Then in a fire I can break it open and extract the drive and boot from it with a USB 3.0 drive cradle. Otherwise with the NAS models I'd have to have an identical Synology NAS to place the ioSafe drives in to get the data off, which wouldn't be an easy task.

But that brings up another question. Would a regular bare hard drive in a fire/water proof strong box that was inside a fire proof gun safe be just as secure, or more so, than an ioSafe, albeit with the added effort of having to actually move the drives into the safe on a regular basis.
Title: Re: Data Backup Strategies
Post by: artephius on August 04, 2015, 08:07:08 AM
Regular fire safes are not necessarily suitable for things like hard drives, tapes, electronics, etc. Despite the fact that they will protect things like paper for a certain length of time, the internal temperature can get high enough to possibly fry your drive.

I believe the ones suitable for that are called "media safes". Here is an example of one. (http://www.gardall.com/view-product/data-media-safes)

I don't have time right now to compare prices, but the outrageous price of the IOsafe may not be much more outrageous than the price of data/media safes from what I saw at a quick glance.

A usb drive would seem to be the way to go though if you could store one that way.

EDIT: Not sure about gun safes though, a good gun safe might provide enough protection. Either way its worth researching.
Title: Re: Data Backup Strategies
Post by: jerseyboy on August 04, 2015, 09:23:28 PM
The android app keeps crashing on me so I will keep this short

FreeNAS for NAS, ZFS, and snapshots.

VM on FreeNAS running OwnCloud to sync files between am computers and snapshot periodically

BackBlaze is awesome. Monthly fee for unlimited storage

Try SpiderOak for Linux

Amazon S3 is very expressive for backup purposes

Amazon prime has unlimited picture backup if you want Amazon looking at an odd your pictures

Jerseyboy
Title: Re: Data Backup Strategies
Post by: Carl on August 05, 2015, 05:01:44 AM
Three $50 USB drives in rotation and in different locations oughta' be looking pretty good now,
theft,fire,flood,and loss prevention with  technical independence and pretty foolproof. 8)
Title: Re: Data Backup Strategies
Post by: outoforder2day on August 05, 2015, 09:38:50 AM
Three $50 USB drives in rotation and in different locations oughta' be looking pretty good now,
theft,fire,flood,and loss prevention with  technical independence and pretty foolproof. 8)
Only if you remember to do it. The advantage of the other systems is that they're automatic. Not saying your setup is bad or wrong, just that it's a lot more labor and time intensive, and that you have to remember to do it. For me, it wouldn't work.
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 05, 2015, 09:39:31 AM
Three $50 USB drives in rotation and in different locations oughta' be looking pretty good now,
theft,fire,flood,and loss prevention with  technical independence and pretty foolproof. 8)

I agree....but it's only foolproof if done on a regular basis, though. The spirit is willing but the flesh is weak, so anything I can do to automate the backup process and take me (the fool) out of the equation, the better.
Title: Re: Data Backup Strategies
Post by: Carl on August 05, 2015, 09:58:24 AM
I agree....but it's only foolproof if done on a regular basis, though. The spirit is willing but the flesh is weak, so anything I can do to automate the backup process and take me (the fool) out of the equation, the better.

I admit,it hurts my arm to plug in a USB cable when it is time to rotate each weak...but I push through the pain and get 'er done.
Title: Re: Data Backup Strategies
Post by: Docwatmo on August 05, 2015, 10:48:39 AM
One of the initial points in the question was backing up with large amounts of multimedia.  Most multimedia is static (Doesn't change) so once it's copied it doesn't need to be recopied with each backup.  I like applications like "Always Sync" because it compares the two directories (Remote and local) and only backs up the changed files.  Makes backing up to USB sticks or external drives or FTP sites etc, SUPER fast.  (After the initial full backup).

But the biggest reasonable USB sticks are in the 64 to 128 gig range (You can pay way more for 256 and up but the price per gig is way off the charts).
Portable hard drives are far cheaper.  1 TB drives are in the $60 range while 256 gig thumb drives are in the $100 to $200 price range.  Not enough storage for my needs. 

Manual swapping of external drives is not difficult by any means.  But anything that automates the process will help to ensure it's not forgotten or skipped.  (Always sync works with USB sticks so you can set it up to auto copy to the mapped stick drive, so the only thing you have to remember to do is rotate the sticks periodically).  But that is still not as nice as a fully automated backup system like mine.  I NEVER have to remember a thing.  I just go out once ever couple of months and test the backups to ensure it has my changed data in it.

Title: Re: Data Backup Strategies
Post by: FreeLancer on August 05, 2015, 01:29:48 PM
I admit,it hurts my arm to plug in a USB cable when it is time to rotate each weak...but I push through the pain and get 'er done.

It's not my arm, it's my brain, that's lame.  I've proven to myself many times that I will get lazy and stop rotating those drives like I should.  A man's gotta know his limitations.

I've got the DS1515+ online now (only took me three tries before I finally figured out how to do the Synology Hybrid RAID equivalent of RAID 6) and using it as a target for my iMac's Time Machine, as well as Synology's Cloud Station app syncing my data and media folders (started last night and still grinding away).  I thought I was going to get rid of my 5 year old DS211j, but now I'm thinking of up-sizing and mirroring the drives and continuing to use it as a secondary Time Machine target, and also backup the new NAS's data to it periodically, at least until it runs out of capacity. 

I do have Carbonite running on my machine as an offsite backup, but I don't fully trust it, as it frequently goes spinning beachball on me and I might not notice it for days to weeks.  I may need to look at Back Blaze when my subscription runs out.  I'm also trying out the new Google Photos free online storage, it's super slow uploading, but freaky good at face recognition and categorizing images, really freaky good!  I might try Amazon Glacier as well, but if it's anything like my foray into S3 a few years ago, it may be too hard for me to figure out.  I'm pretty much at the end of my capabilities just getting a NAS online, but I will say that Synology's latest Disk Station software is much easier to use now compared to when I set up my old one 5 years ago, and there wasn't a Cloud Station app back then either.

This upgrade leaves me with 4 older 2Tb drives that I'll try to keep rotated through a backup scheme similar to what Carl suggests, but I lack faith in myself to keep up the habit.  I purchased a 1 hour fire/water box from Honeywell (http://www.amazon.com/gp/product/B004IPR22C?psc=1&redirect=true&ref_=oh_aui_detailpage_o01_s02) and will keep it in my gun safe, so hopefully that combination will protect the bare drives, and other media, if the house goes up in smoke.
Title: Re: Data Backup Strategies
Post by: r_w on August 06, 2015, 06:50:35 AM
Anybody do off-site ONSITE?  I have been thinking of putting a device out in the shop or storage shed, somewhere in Wi-Fi range but shouldn't burn in a house fire (forest fire is another problem).  I have pathetic internet speeds so cloud is not a viable option except for manual uploads of uber critical static data--the stuff that is hard to trust to the cloud.
Title: Re: Data Backup Strategies
Post by: Carl on August 06, 2015, 06:55:31 AM
Anybody do off-site ONSITE?  I have been thinking of putting a device out in the shop or storage shed, somewhere in Wi-Fi range but shouldn't burn in a house fire (forest fire is another problem).  I have pathetic internet speeds so cloud is not a viable option except for manual uploads of uber critical static data--the stuff that is hard to trust to the cloud.

Faster to put it on TWO USB drives,in fire boxes,one in the shed...one elsewhere...why use WIFI when you can hand carry the faster loaded,more secure hard drives?As lightening tends to effect wired devices and lightening is your most likely problem...then fire...oh ,do you store gas and lawn mowers in the shop? I would not transfer (transmit) data a distance I could easily walk..Just my thoughts.
Title: Re: Data Backup Strategies
Post by: outoforder2day on August 06, 2015, 07:10:33 AM
Anybody do off-site ONSITE?  I have been thinking of putting a device out in the shop or storage shed, somewhere in Wi-Fi range but shouldn't burn in a house fire (forest fire is another problem).  I have pathetic internet speeds so cloud is not a viable option except for manual uploads of uber critical static data--the stuff that is hard to trust to the cloud.
I've been considering this, too, however it's not the best option. If you need to bug out, or if something happens that affects your entire geography, it will affect your backup. Flooding would be an example concern here. Since you don't have the best bandwidth, then the second hard drive approach is the best bet, but consider leaving it at your in-laws or some place further away.   

A key component of this discussion, as you pointed out, is the change rate of your data. Is most of your content pictures? Movies? or text? How often does it change? How hard is it to replace? Unless you're generating gigs of data a day, most of this stuff would be overkill and two hard drives could suffice. And things like your social security card or mortgage paperwork don't exactly change over time. It all depends.
Title: Re: Data Backup Strategies
Post by: Docwatmo on August 06, 2015, 08:11:10 AM
My primary residence is 18 miles from my work.  I have a VPN and 20 Meg internet service at home and 15 meg internet service at work.  I picked up 2 QNAP NAS devices a couple of years ago on the cheap.  Spent the next year finding cheap hard drives and putting in about 8TB on each unit (Usuable) in a Raid 5 with hot spare on each device.  I run a Secure SSL VPN connection between home and work and the QNAP's have a built in "Sync to other Qnap" option.  So my local stuff is backed up locally and then synced to my work device.  The first time I set this up, I synced them at home on the same Gigabit network.  (about 2.8 TB of data).  They have been running for 2 years and the sync is block level so when I drop a new multi Gig movie from the camcorder or camera on the local side that is the only thing that takes time and generaly over night it has a full copy synced up again.

This was an expensive option.  The Qnaps run about $500 to $900 each.  (I got mine discounted for about half).  And then it took me a year to find and buy 10 x 2TB hard drives to fill them with.   But now it's out of site, out of mind.  Only takes a minute to access data and verify it's backing up correctly. 

Which brings us to the other elephant in the room when discussing backups.  How often do you test your backups?   

I used to do computer work on the side and for a local chiropractor in town, when I first started, they asked me to set up their new server and move their backup drive to the new server.  I went down and they had 2 tapes that they had swapped for several years.  Both tapes were worn so thin you could see through them.  The backups somehow could still be read, but with lots of errors.   I immediately made a full image copy of the system on an external hard drive and then burnt it to a DVD.  Set up the new system and set up a 6 tape (5 daily and 1 monthly archive) backup system for them. 

It's not backed up if you can't get the data out of your backups.  So they MUST be tested periodically.   
Title: Re: Data Backup Strategies
Post by: FreeLancer on August 11, 2015, 09:02:33 PM
In case anybody else is thinking about using Synology's Cloud Station, be extremely careful with what, and how, you sync.  I've learned the hard way that iTunes gets seriously screwed up when Cloud Station accesses the iTunes folder, which cascaded into foul-ups with my entire podcast library and iPhone.  Being able to roll back 3 days, thanks to multiple incremental Time Machine backups (one was broken!) saved me.
Title: Re: Data Backup Strategies
Post by: zeugirdor on December 30, 2015, 03:49:48 PM
This is a great thread.  Something that most people completely overlook in their preps.

I have actually set up a small network of QNAP NAS stations in 3 geographically diverse locations.  I have "cloud" access to my main device and it keeps all of my files synced with two other NASs that I have squirreled away in my parent's, and my sibling's houses.  I also use Amazon Glacier for the family photos and digital videos that obviously never change, they just get added to.

This solution is not the least expensive, but in my experience the reason why people loose data is because they either never do a backup or don't keep up with doing backups... The most likely piece of your computer (or NAS) to fail is the hard drive. I love this solution because you set it once and forget it until you notice that you're not receiving the emailed log reports.  Granted that's not necessarily perfect because things do happen and you could find that you need to go to the backup and it hasn't been synced in 6 months, but it's better than never having had a backup done in the first place.
Title: Re: Data Backup Strategies
Post by: d3nni5 on December 31, 2015, 07:14:29 AM


 :popcorn:
Title: Re: Data Backup Strategies
Post by: FreeLancer on September 05, 2019, 10:21:31 PM
I've got the DS1515+ online now (only took me three tries before I finally figured out how to do the Synology Hybrid RAID equivalent of RAID 6) and using it as a target for my iMac's Time Machine, as well as Synology's Cloud Station app syncing my data and media folders (started last night and still grinding away).  I thought I was going to get rid of my 5 year old DS211j, but now I'm thinking of up-sizing and mirroring the drives and continuing to use it as a secondary Time Machine target, and also backup the new NAS's data to it periodically, at least until it runs out of capacity. 

Woke up this morning and noticed my DS1515+ had powered off and won't turn back on.  An online search showed that apparently there was a fatal flaw in the Intel chipset used in several Synology NAS models that would brick a certain number of machines without warning, just as mine did, and it's not fixable.  Synology extended their replacement warranty period on these machines into 2018 and my machine decided to hang on a year past the limit, so I'm out of luck until I can get another Synology box to put these drives into to pull the data off those drives.  And I'd literally just finished upgrading from 4TB to 6GB drives before it crapped out, a long process that took several days to get through, one new drive at a time. 

It just goes to show that having multiple drive redundancy doesn't mean squat if the box that controls them all gives up the ghost.  Meanwhile the ancient DS211j keeps chugging along and refuses to die.
Title: Re: Data Backup Strategies
Post by: Greekman on September 06, 2019, 09:44:20 AM
indeed, offsite is the key.
In a photography forum we have a member whose house burned down. despite him having triple copies of his works, this meant hat his business was burned down too.
Title: Re: Data Backup Strategies
Post by: FreeLancer on September 08, 2019, 01:04:01 PM
Amazon Warehouse sells returned items as used at a decent discount and I was able to pick up a Synology DS1618+ that arrived last night.  This is a 6 bay machine so I put the 5 drives from the 1515+ in the new box and Synology Web Assistant easily migrated all the settings and data over.  The extra slot in the new machine will be set up as a hot spare.  Fairly painless except for the cost of a new machine and it was good practice on what's involved with the recovery process in this type of failure.