I am inbetween contemplating a NAS enclosure with 2.5Gbps Ethernet+ or just a simple multi-bay RAID enclosure with 3.1 USB C output for 10Gbps. I have no need for the NAS to be attached to the network, and it would only be connected to my studio computer anyway, so both, as I understand it, will be virtually the same without the need to utilise the network attachment.

My preferred solution at the moment is probably to look at a 4-5 bay RAID enclosure, and populate it with HDDs to fill it up. I would ideally like to start with 2 drives, and then add in the other 2-3 later on.

One thing I want to build in is redundancy for data protection. The enclosure will house everything (I hope to populate with 2 x 16TB drives initially, adding further 16/18TB drives over time as required). That means I will put my sample libraries, projects, Kontakt libraries, media, software backups, and so on all on there so it's all in a single location.

I will probably use a cloud drive like Dropbox or Google Drive, and have the location on the enclosure too, storing my mission critical stuff in there so that they are duplicated into the cloud for backup.

The hope is that I can then get rid of my expensive €30 per month MEGA Pro III 16TB cloud account, being covered instead by local redundancy and a more basic and cheap cloud solution for mission critical stuff.

However, despite the above, I've never really used RAID before and have little understanding of how they work or which types would best suit my need. Any advice, thoughts, or recommendations would be hugely appreciated!

 

Ofc one could do it that way.
BUT

A WARNING:
RAID does not and is not intended to replace proper backup. Instead it is meant to make it possible to use the RAID storage even tough a single (or more) drive is broken. This way you can work as usual while the RAID rebuilds with a new (or hot spare) drive.

The problems with your proposed config is, that it is in the same enclosure, it is in the same room and it runs at the same time(s)
If there is a major problem, all data is lost.

The only way to avoid this is a backup that ist NOT in the same enclosure, is NOT in the same room and is NOT running at the same time.

For instance a cloud backup.
Or a second NAS in another building connected by a network but only starting for the backup then stopping for x hours.

If the data has no inner worth for you (e.g. if you could get it back from other sources) that is not that of a problem. But if your business (or happiness) relies on your data you need backups.

Just a few conceivable situations in which a RAID with redundancy does not protect against data loss:
- A fault in the power supply unit will result in the ARID enclosure being destroyed, e.g. by fire.
- Burglary into the building, A water ingress in the level above the studio
- A criminal encrypts the entire contents of your RAID (ransomware)
In all these cases, a proper data backup with several generations will help you NOT to lose all your data.

 

Last edited: Apr 7, 2024

The point is that I have a lot of digital stuff accumulated over 2 or 3 decades. Some of which are more sentimental than critical, admittedly, but means I cannot find any reasonably priced cloud offerings with big enough capacity (except MEGA Pro III) and also have several drives of varying capacities I want to try to rationalise down to as few drives as possible in as few enclosures as possible.

The hope is to have one enclosure with local storage, and ideally redundancy for protection so I can rebuild a drive failure, and have critics stuff backed up to a reasonably priced cloud storage.

 

The Mar-A-Lago cardboard boxes bathroom system.

(Sorry, absolutely couldn't resist. ))))))

 

@ op, Why do you need to keep that much stuff?

 

There are good reasons for this. Just for instance:
Data security/dependability
A few years ago, Amazon had massive server failures in one area of its cloud. It then became apparent that its backup servers were also affected. Customers who did not have offline backups lost their data for good.

Data protection:
Microsoft has become the executive body for law enforcement agencies and has reported customers to the FBI who allegedly had criminal content stored on their cloud. In the past, this has led to cases such as that of the German father who had beach pictures of his small children on his cloud and received a visit from the FBI analog on suspicion of...

Repressive administrations are able to pass laws that further weaken data protection. And I'm not talking about China, but countries like the UK or Germany. As far as I know, there are not even data protection laws in any form in the US like there are in the EU, for example

Control over my own Data:
In times of big data, I am sure that companies will not stop at private clouds. We will see what consequences this may have.

 

Last edited: Apr 8, 2024

Look for some DAS - direct access storage enclosures from like QNAP with support for 4 or more drives, set 'em up for RAID-5 - that's the one that supports parity across all disks (you will lose about 1/3 of space to parity) and that's it. They go for around $200-300. It's a nice and simple solution for a very safe backup. of course, cloud backup is the only 99% safe solution, but in my experience NASes and DASes are killer for a regular user, especially if you don't run it 24/7. If you turn it on once every couple of days DAS or NAS could last you for years and years, possibly decades.

One of the big advantages of DAS and NAS is that they use ZFS or XFS or similar safe-designed for data-backup file systems that are in themselves safer for storing your data. You don't get that with NTFS or any common FS. My NAS is running on XFS (my choice) with Open Media Vault software and it's been running for years without any problems (knock on wood ).

I'm so satisfied with it. Especially because the computer I use for it is just an old Intel CoreDuo 8400. It really doesn't matter much what computer you use for NAS-DAS for as long as the network card or USB is fast enough. Great thing I can take all these HDs and put into another computer with Open Media Vault and not lose any data in the process. Also, with RAID-5 if a HDD-SDD fails you can easily replace it and rebuild RAID array. Truly a storage solution that lasts you a lifetime. Especially when you're over 50 like me.

Cheers!

 

Last edited: Apr 8, 2024

I'm not sure about which redundancy would recommend you. Probably a RAID 5 despite the valid points made against it.
I would go directly for the USB, that's for sure. DAS (oops we have another acronym) all the way if it's only one computer.

Hmmm... now that you mention it, it sounds fishy all right...
Any secret you wanna share Saint? Remember, you are in a safe space here. Especially because you're the boss

PS. Ahem... just in case my ass gets kicked out, it's been a pleasure to know y'all guys

 

Last edited: Apr 8, 2024

What are the best high capacity drives at the moment, in the 16/18/20TB arena?

Seagate IronWolf Pro? Seagate Exos X20? Something else?

 

..The lesser of two evils...
The one thing many people forget is that anything that has moving parts or can wear over time has a limited lifespan.
Every one of us can say my (brand name) SSD / HDD / RAID has lasted longer than... but one day it will show signs of needing replacing, hopefully before it dies.

The important topics of redundancy needs vs safety in each have been covered already so I won't regurgitate. The amount of data will always determine my choices and I can only speak for myself because everyone's needs are different, whether personal or business. I work out what I would be devastated about most if I lost it, and then ensure it has more than one form of security. I only mention this because the odds on losing three copies at once are small, unless they are all in the same location.

@Bunford - The Iron wolf is fairly sturdy for basic commercial use, as in us people. Some say there is no point spending small fortunes on the super expensive data redundancy drives. More often than not, they are not billion dollar companies whose data is mission critical with formulas, development products (R&D)....corporate or government secure data....etc...etc.. If you want military grade drives that is a whole different ballpark and tailored to billion dollar data, with a cost attached - regularly RAID 10 (RAID 0 mirrored) and RAID 50 (RAID 5 supported by RAID 0 for speed), which is generally only seen in data centers. That's an overly generalized summary.

 

Last edited by a moderator: Apr 8, 2024

stop.
what exactly are you trying to achieve?

1) all RAIDs by design don't allow mixing drives of various sizes (unless you go with proprietary Synology Hybrid RAID)
2) RAID is not a backup - I assume you'd have another drives elsewhere?
3) rebuilding faulty drive of RAID5 with 4-5 20TB drives has much higher chance of any other drive failing, loosing all data at once
4) fastest rebuild (zero downtime) is on RAID1 (mirror) but you waste half the capacity potential (1 drive capacity of 2 drives)
5) 2.5Gbit ethernet is much slower than casual USB 3.0 at 5Gbit or ancient SATAIII (SSD) at 6Gbit
6) if you don't need networked storage, why not just use those drives inside your PC via Storage Spaces ?
7) do you have very fast internet upload to backup/mirror onto cloud? because for ex. with casual 30Mbit upload it'll take around 2 months to transfer 16TB
8) virtual instruments, libraries and samples will always work better on internal storage, due to nature of network protocols like SMB or NFS there's always too much overhead, not even talking about network limitations where 10Gbit infrastructure is expensive, yet much slower than casual 56Gbit PCIe 4.0 M.2 NVME SSDs; and iSCSI is a completely different thingy to begin with, overly complicated and expensive way to pretend networked pool is attached as if was local drive

my humble opinion:
research and buy some decent Synology NAS, 4-bay DS923+ or 5-bay DS1522+ (or search used market for DS918+ or DS920+ or DS1621+), the convenience it gives is hard to describe, but totally worth the price, especially if you're not a tinkerer to go TrueNas diy route, and want something relatively low-power draw 24/7,
you will want to go through the data, sort/clean things up, be able to access it from other places than your main computer,
if you have 2-3 decades worth of data, then plan at least 2-3x storage space (so if you have 16TB data, then 4x16TB at SHR1), to have enough headroom and good performance (pool/volume beyond 80% capacity used becomes slower with fragmented data writes and reads),
also with SHR1 (Synology Hybrid Raid-1) you'll be able to expand capacity later without need to buy all 4 new drives at once,
such setup should be plenty for years ahead and unless you get power surges, theft or some natural disaster, you'll be good to go (ofc still do have backup of valuable data on some off shelf disk(s)

 

I have a NAS I built from parts a while back - had some parts and had to buy some.
I started off using FreeNAS to run it but found them to be a bit off on their forums - suggesting more expensivce hardware was necessary, so ended up switching to NAS4Free which is now called Xigmanas and find it to be much better and the community surrounding it to be much friendlier and less up their ass than Freenas.

I've got four striped RAID pairs in it with 4 (x2) on three and one pair of 8GBs.
Use them for all kind of data.

I use Seagate Ironwolf drives - which are slightly more expensive but made specifically for NAS use and have good guarantees.

 

What I am trying to achieve is to rationalise everything down to as few drives/enclosures as possible, providing some redundancy, and to also have a backup. I want the redundancy to be able to cover at least 1 drive failure, which I can then switch a new one in as a replacement if ever needed.

I think my preferred option at the moment is to use something like the QNAP TR-004 4 bay DAS enclosure with onboard hardware RAID controller (currently scoping options for 10Gbps transfer), house with 4 identical brand and capacity drives, and then set up in RAID (probably RAID 5 being best option from what I can see?).

I will then end my $30 per month MEGA Pro III subscription and instead subscribe to something like the $9 per month Backblaze, which back up the OS drive and any attached storage, including DAS, and includes unlimited data.

This would then rationalise everything into one device locally (including music production materials like sample libraries, Kontakt libraries, software backups and so on, as well as personal items like media, photos etc), meaning one power source, one transfer cable into my computer, can easily switch between my desktop and laptop, and give me a cheap and affordable cloud backup of everything.

To note, within my desktop, I do also have these internal drives (which will all be duplicated across to the DAS for local backup):

C: - a 2TB NVMe OS drive for WIndows 11
D: - a 2TB NVMe drive used to store VST data, like Nexus library, Steinberg data, and so on
S: - a 4TB NVMe drive used to store my sample libraries, presets, and MIDI
K: - a 4TB SSD used to store my most used Kontakt libraries
P: - a 4TB SSD used to store my projects​

and I do also have a 2 TB NMVe USB C external drive I use for my DJ music and Pioneer Rekordbox database and so on.

Does this seem like a rational and sensible solution? And are there any RAID setups that would work best in this scenarion?

 

sounds good to me
you'll get up to USB 3.0 speeds which is enough for casual HDDs in RAID5 and way faster than 1Gbe network,
do note that reaching consistent 10Gbps transfers from/to RAID array would need 6-8 HDDs anyway, not worthy imo