What is the best RAiD to use for Backups?

Discussion in 'Computer Hardware' started by Bunford, Apr 7, 2024.

  1. Bunford

    Bunford Audiosexual

    Joined:
    Jan 17, 2012
    Messages:
    2,503
    Likes Received:
    956
    I am inbetween contemplating a NAS enclosure with 2.5Gbps Ethernet+ or just a simple multi-bay RAID enclosure with 3.1 USB C output for 10Gbps. I have no need for the NAS to be attached to the network, and it would only be connected to my studio computer anyway, so both, as I understand it, will be virtually the same without the need to utilise the network attachment.

    My preferred solution at the moment is probably to look at a 4-5 bay RAID enclosure, and populate it with HDDs to fill it up. I would ideally like to start with 2 drives, and then add in the other 2-3 later on.

    One thing I want to build in is redundancy for data protection. The enclosure will house everything (I hope to populate with 2 x 16TB drives initially, adding further 16/18TB drives over time as required). That means I will put my sample libraries, projects, Kontakt libraries, media, software backups, and so on all on there so it's all in a single location.

    I will probably use a cloud drive like Dropbox or Google Drive, and have the location on the enclosure too, storing my mission critical stuff in there so that they are duplicated into the cloud for backup.

    The hope is that I can then get rid of my expensive €30 per month MEGA Pro III 16TB cloud account, being covered instead by local redundancy and a more basic and cheap cloud solution for mission critical stuff.

    However, despite the above, I've never really used RAID before and have little understanding of how they work or which types would best suit my need. Any advice, thoughts, or recommendations would be hugely appreciated!
     
    • Useful Useful x 2
    • Interesting Interesting x 1
    • List
  2.  
  3. twoheart

    twoheart Audiosexual

    Joined:
    Nov 21, 2015
    Messages:
    2,092
    Likes Received:
    1,298
    Location:
    Share many
    Ofc one could do it that way.
    BUT
    A WARNING:
    RAID does not and is not intended to replace proper backup. Instead it is meant to make it possible to use the RAID storage even tough a single (or more) drive is broken. This way you can work as usual while the RAID rebuilds with a new (or hot spare) drive.

    The problems with your proposed config is, that it is in the same enclosure, it is in the same room and it runs at the same time(s)
    If there is a major problem, all data is lost.

    The only way to avoid this is a backup that ist NOT in the same enclosure, is NOT in the same room and is NOT running at the same time.

    For instance a cloud backup.
    Or a second NAS in another building connected by a network but only starting for the backup then stopping for x hours.

    If the data has no inner worth for you (e.g. if you could get it back from other sources) that is not that of a problem. But if your business (or happiness) relies on your data you need backups.

    Just a few conceivable situations in which a RAID with redundancy does not protect against data loss:
    - A fault in the power supply unit will result in the ARID enclosure being destroyed, e.g. by fire.
    - Burglary into the building, A water ingress in the level above the studio
    - A criminal encrypts the entire contents of your RAID (ransomware)
    In all these cases, a proper data backup with several generations will help you NOT to lose all your data.
     
    Last edited: Apr 7, 2024
    • Agree Agree x 2
    • Useful Useful x 2
    • List
  4. SAiNT

    SAiNT Creator Staff Member phonometrograph

    Joined:
    May 21, 2011
    Messages:
    2,062
    Likes Received:
    1,755
    Location:
    ZiON
    is there a point these days to use local backup if you will be using cloud anyways?

    which leads us to your previous topic:
    https://audiosex.pro/threads/cloud-backup-storage-in-2024.74092/

    you could go with some good VPN + Mountain Duck + GoodSync while keeping your work projects on SSD.
    i don't know where you live, but these days we all should consider being more "portable" :sad:

    Related topics:
    https://audiosex.pro/threads/backups-2018.38845/
     
    Last edited: Apr 8, 2024
    • Like Like x 1
    • Useful Useful x 1
    • List
  5. Bunford

    Bunford Audiosexual

    Joined:
    Jan 17, 2012
    Messages:
    2,503
    Likes Received:
    956
    The point is that I have a lot of digital stuff accumulated over 2 or 3 decades. Some of which are more sentimental than critical, admittedly, but means I cannot find any reasonably priced cloud offerings with big enough capacity (except MEGA Pro III) and also have several drives of varying capacities I want to try to rationalise down to as few drives as possible in as few enclosures as possible.

    The hope is to have one enclosure with local storage, and ideally redundancy for protection so I can rebuild a drive failure, and have critics stuff backed up to a reasonably priced cloud storage.
     
  6. BlackHawk

    BlackHawk Platinum Record

    Joined:
    Nov 28, 2021
    Messages:
    351
    Likes Received:
    171
    The Mar-A-Lago cardboard boxes bathroom system.

    (Sorry, absolutely couldn't resist. :)))))))
     
  7. SAiNT

    SAiNT Creator Staff Member phonometrograph

    Joined:
    May 21, 2011
    Messages:
    2,062
    Likes Received:
    1,755
    Location:
    ZiON
    buy the most reliable HDD (like WD Green) and bury your sentiments on it, connect it only when you need it, set a reminder in calendar to run tests on that HDD once a year and another reminder to replace that drive every 4-5 years.


    anyway, i'd love to find a decent and reliable cloud solution to send all that sort of stuff to. with mountain duck and good connection i can use it just as a regular drive.
     
  8. clone

    clone Audiosexual

    Joined:
    Feb 5, 2021
    Messages:
    7,356
    Likes Received:
    3,245
    @ op, Why do you need to keep that much stuff?
     
  9. twoheart

    twoheart Audiosexual

    Joined:
    Nov 21, 2015
    Messages:
    2,092
    Likes Received:
    1,298
    Location:
    Share many
    There are good reasons for this. Just for instance:
    Data security/dependability
    A few years ago, Amazon had massive server failures in one area of its cloud. It then became apparent that its backup servers were also affected. Customers who did not have offline backups lost their data for good.

    Data protection:
    Microsoft has become the executive body for law enforcement agencies and has reported customers to the FBI who allegedly had criminal content stored on their cloud. In the past, this has led to cases such as that of the German father who had beach pictures of his small children on his cloud and received a visit from the FBI analog on suspicion of...

    Repressive administrations are able to pass laws that further weaken data protection. And I'm not talking about China, but countries like the UK or Germany. As far as I know, there are not even data protection laws in any form in the US like there are in the EU, for example

    Control over my own Data:
    In times of big data, I am sure that companies will not stop at private clouds. We will see what consequences this may have.
     
    Last edited: Apr 8, 2024
  10. SineWave

    SineWave Audiosexual

    Joined:
    Sep 4, 2011
    Messages:
    4,431
    Likes Received:
    3,569
    Location:
    Where the sun doesn't shine.
    Look for some DAS - direct access storage enclosures from like QNAP with support for 4 or more drives, set 'em up for RAID-5 - that's the one that supports parity across all disks (you will lose about 1/3 of space to parity) and that's it. They go for around $200-300. It's a nice and simple solution for a very safe backup. of course, cloud backup is the only 99% safe solution, but in my experience NASes and DASes are killer for a regular user, especially if you don't run it 24/7. If you turn it on once every couple of days DAS or NAS could last you for years and years, possibly decades.

    One of the big advantages of DAS and NAS is that they use ZFS or XFS or similar safe-designed for data-backup file systems that are in themselves safer for storing your data. You don't get that with NTFS or any common FS. My NAS is running on XFS (my choice) with Open Media Vault software and it's been running for years without any problems (knock on wood :)).

    I'm so satisfied with it. Especially because the computer I use for it is just an old Intel CoreDuo 8400. It really doesn't matter much what computer you use for NAS-DAS for as long as the network card or USB is fast enough. Great thing I can take all these HDs and put into another computer with Open Media Vault and not lose any data in the process. Also, with RAID-5 if a HDD-SDD fails you can easily replace it and rebuild RAID array. Truly a storage solution that lasts you a lifetime. Especially when you're over 50 like me. :rofl:

    Cheers!
     
    Last edited: Apr 8, 2024
  11. SAiNT

    SAiNT Creator Staff Member phonometrograph

    Joined:
    May 21, 2011
    Messages:
    2,062
    Likes Received:
    1,755
    Location:
    ZiON
    yes, shit happens, but from what i understand data centers are far more protected from those occasions than a regular user. (the one that doesn't wear foil hat :yes:)

    yes, of course, no serious company can refuse to coop with law enforcement. apple, facebook, coinbase, hotmail - they all known for their glorious cooperation.

    on the other hand, a good service should provide encryption, but even if it doesn't, once again both goodsync and mountain duck have great encryption features (damn i should be paid for how many times i've mentioned it here), so you can feel safe even on google drive.

    and since you've mentioned it, in this regard you will be in even worst position if you keep your sensitive stuff on hard drives at home, because it can be easily confiscated and you probably will give that encryption password.
     
  12. Xupito

    Xupito Audiosexual

    Joined:
    Jan 21, 2012
    Messages:
    7,231
    Likes Received:
    3,995
    Location:
    Europe
    I'm not sure about which redundancy would recommend you. Probably a RAID 5 despite the valid points made against it.
    I would go directly for the USB, that's for sure. DAS (oops we have another acronym) all the way if it's only one computer.

    Hmmm... now that you mention it, it sounds fishy all right...
    Any secret you wanna share Saint? Remember, you are in a safe space here. Especially because you're the boss

    PS. Ahem... just in case my ass gets kicked out, it's been a pleasure to know y'all guys :wink: :rofl:
     
    Last edited: Apr 8, 2024
  13. Bunford

    Bunford Audiosexual

    Joined:
    Jan 17, 2012
    Messages:
    2,503
    Likes Received:
    956
    What are the best high capacity drives at the moment, in the 16/18/20TB arena?

    Seagate IronWolf Pro? Seagate Exos X20? Something else?
     
  14. BaSsDuDe

    BaSsDuDe Guest

    ..The lesser of two evils...
    The one thing many people forget is that anything that has moving parts or can wear over time has a limited lifespan.
    Every one of us can say my (brand name) SSD / HDD / RAID has lasted longer than... but one day it will show signs of needing replacing, hopefully before it dies.

    The important topics of redundancy needs vs safety in each have been covered already so I won't regurgitate. The amount of data will always determine my choices and I can only speak for myself because everyone's needs are different, whether personal or business. I work out what I would be devastated about most if I lost it, and then ensure it has more than one form of security. I only mention this because the odds on losing three copies at once are small, unless they are all in the same location.

    @Bunford - The Iron wolf is fairly sturdy for basic commercial use, as in us people. Some say there is no point spending small fortunes on the super expensive data redundancy drives. More often than not, they are not billion dollar companies whose data is mission critical with formulas, development products (R&D)....corporate or government secure data....etc...etc.. If you want military grade drives that is a whole different ballpark and tailored to billion dollar data, with a cost attached - regularly RAID 10 (RAID 0 mirrored) and RAID 50 (RAID 5 supported by RAID 0 for speed), which is generally only seen in data centers. That's an overly generalized summary.
     
    Last edited by a moderator: Apr 8, 2024
  15. tzzsmk

    tzzsmk Audiosexual

    Joined:
    Sep 13, 2016
    Messages:
    3,626
    Likes Received:
    2,224
    Location:
    Heart of Europe
    stop.
    what exactly are you trying to achieve?

    1) all RAIDs by design don't allow mixing drives of various sizes (unless you go with proprietary Synology Hybrid RAID)
    2) RAID is not a backup - I assume you'd have another drives elsewhere?
    3) rebuilding faulty drive of RAID5 with 4-5 20TB drives has much higher chance of any other drive failing, loosing all data at once
    4) fastest rebuild (zero downtime) is on RAID1 (mirror) but you waste half the capacity potential (1 drive capacity of 2 drives)
    5) 2.5Gbit ethernet is much slower than casual USB 3.0 at 5Gbit or ancient SATAIII (SSD) at 6Gbit
    6) if you don't need networked storage, why not just use those drives inside your PC via Storage Spaces ?
    7) do you have very fast internet upload to backup/mirror onto cloud? because for ex. with casual 30Mbit upload it'll take around 2 months to transfer 16TB
    8) virtual instruments, libraries and samples will always work better on internal storage, due to nature of network protocols like SMB or NFS there's always too much overhead, not even talking about network limitations where 10Gbit infrastructure is expensive, yet much slower than casual 56Gbit PCIe 4.0 M.2 NVME SSDs; and iSCSI is a completely different thingy to begin with, overly complicated and expensive way to pretend networked pool is attached as if was local drive

    my humble opinion:
    research and buy some decent Synology NAS, 4-bay DS923+ or 5-bay DS1522+ (or search used market for DS918+ or DS920+ or DS1621+), the convenience it gives is hard to describe, but totally worth the price, especially if you're not a tinkerer to go TrueNas diy route, and want something relatively low-power draw 24/7,
    you will want to go through the data, sort/clean things up, be able to access it from other places than your main computer,
    if you have 2-3 decades worth of data, then plan at least 2-3x storage space (so if you have 16TB data, then 4x16TB at SHR1), to have enough headroom and good performance (pool/volume beyond 80% capacity used becomes slower with fragmented data writes and reads),
    also with SHR1 (Synology Hybrid Raid-1) you'll be able to expand capacity later without need to buy all 4 new drives at once,
    such setup should be plenty for years ahead and unless you get power surges, theft or some natural disaster, you'll be good to go (ofc still do have backup of valuable data on some off shelf disk(s)
    :chilling:
     
  16. northwinds

    northwinds Kapellmeister

    Joined:
    Jan 11, 2014
    Messages:
    147
    Likes Received:
    41
    I have a NAS I built from parts a while back - had some parts and had to buy some.
    I started off using FreeNAS to run it but found them to be a bit off on their forums - suggesting more expensivce hardware was necessary, so ended up switching to NAS4Free which is now called Xigmanas and find it to be much better and the community surrounding it to be much friendlier and less up their ass than Freenas.

    I've got four striped RAID pairs in it with 4 (x2) on three and one pair of 8GBs.
    Use them for all kind of data.

    I use Seagate Ironwolf drives - which are slightly more expensive but made specifically for NAS use and have good guarantees.
     
  17. Bunford

    Bunford Audiosexual

    Joined:
    Jan 17, 2012
    Messages:
    2,503
    Likes Received:
    956
    What I am trying to achieve is to rationalise everything down to as few drives/enclosures as possible, providing some redundancy, and to also have a backup. I want the redundancy to be able to cover at least 1 drive failure, which I can then switch a new one in as a replacement if ever needed.

    I think my preferred option at the moment is to use something like the QNAP TR-004 4 bay DAS enclosure with onboard hardware RAID controller (currently scoping options for 10Gbps transfer), house with 4 identical brand and capacity drives, and then set up in RAID (probably RAID 5 being best option from what I can see?).

    I will then end my $30 per month MEGA Pro III subscription and instead subscribe to something like the $9 per month Backblaze, which back up the OS drive and any attached storage, including DAS, and includes unlimited data.

    This would then rationalise everything into one device locally (including music production materials like sample libraries, Kontakt libraries, software backups and so on, as well as personal items like media, photos etc), meaning one power source, one transfer cable into my computer, can easily switch between my desktop and laptop, and give me a cheap and affordable cloud backup of everything.

    To note, within my desktop, I do also have these internal drives (which will all be duplicated across to the DAS for local backup):

    C: - a 2TB NVMe OS drive for WIndows 11
    D: - a 2TB NVMe drive used to store VST data, like Nexus library, Steinberg data, and so on
    S: - a 4TB NVMe drive used to store my sample libraries, presets, and MIDI
    K: - a 4TB SSD used to store my most used Kontakt libraries
    P: - a 4TB SSD used to store my projects​

    and I do also have a 2 TB NMVe USB C external drive I use for my DJ music and Pioneer Rekordbox database and so on.

    Does this seem like a rational and sensible solution? And are there any RAID setups that would work best in this scenarion?
     
  18. tzzsmk

    tzzsmk Audiosexual

    Joined:
    Sep 13, 2016
    Messages:
    3,626
    Likes Received:
    2,224
    Location:
    Heart of Europe
    sounds good to me :yes:
    you'll get up to USB 3.0 speeds which is enough for casual HDDs in RAID5 and way faster than 1Gbe network,
    do note that reaching consistent 10Gbps transfers from/to RAID array would need 6-8 HDDs anyway, not worthy imo
     
  19. SAiNT

    SAiNT Creator Staff Member phonometrograph

    Joined:
    May 21, 2011
    Messages:
    2,062
    Likes Received:
    1,755
    Location:
    ZiON
    you need to look for HDD benchmarks and ratings.
     
Loading...
Similar Threads - best RAiD Backups Forum Date
Best Falcon expansions Software Monday at 11:07 PM
Best GM Soundfont for orchestral music Genre Specific Production Sunday at 5:55 PM
Best Eq app for Android Mobile Oct 30, 2024
Best Tools for Extracting Audio from Video? Working with Video Oct 27, 2024
Best 808 preset (Serum, Spire, Diva etc.)? Presets, Patches Oct 22, 2024
Loading...