I have 4 x 18TB Seagate IronWolf Pro drives to put in an enclosure and going to be using on Windows 11 Pro machines. I was looking for a 4 bay hardware RAID box with USB 3 Gen 2 10Gbps enclosures, but seems like they don't exist for some reason (mind blown!).

I am therefore left with needing to use a software storage pool/RAID management solution, and have no idea in this territory as never explored nor done anything like this before.

Anybody here us or got experience with Windows Storage Spaces? Or is there anything else that useable, user friendly, and allows for an external enclosure to be moved between Windows machines and still have the storage pool recognised?

 

keyword you should search for is "USB DAS",
some options mentioned here: https://nascompares.com/recommended-usb-c-raid-das-storage/

Windows Storage Spaces are designed for servers, I wouldn't rely on Windows host system managing RAID on external enclosure which can drop/unsync fairly easily, and especially not when moving between PCs,
USB JBOD enclosures (RAID-less box) isn't prefered solution either, because again USB is less reliable than proper SATA and SAS (on HBA),

but again (at least I think I mentioned that previously), I'd rather suggest proper NAS, with additional 10Gbe interface set as static IP address and directly connected to PC with additional 10Gbe interface you'll basically get DAS, but with convenience of being also a standalone RAID box with its own operating system, data scrubbing and such


PS: if cost is the concern, then you should reconsider value of your stored data

 

I've been researching for ages already, and know it's a USB 3 Gen 2 DAS I need for my use case. However, there aren't any with USB 3 Gen 2 10Gbps, 4+ bays, and have hardware RAID. Therefore, I am having to explore how to manage dumb enclosure with software RAID solutions, of which Windows Storage Spaces keeps cropping up in my research. The link you gave shows this, with the 4 bay suggestion being a 3.1 Gen 1 Drobo device, which is a company that's now ceased to exist, and the HighPoint is no longer manufactured or available.

I am therefore contemplating something like the Asustor AS5004U or the TerraMaster D8 Hybrid and setting them up in something like RAID10/RAID5, but have no idea on best way/software to use for this as it's all new to me.

The NAS is not covered by Backblaze, my off-site cloud backup host. Backblaze charge $9 per month for computer backup that includes DAS. If I use a NAS it then costs something like $9 per TB, which becomes super expensive, super quick. Therefore, it's a specific reason for not considering a NAS, and to do with exortionate costs rather than not valuing my stored data.

I think what you're suggesting it to basically use a NAS as a DAS, connecting with ethernet rather than USB, but as it's ethernet it is what then makes it a NAS for Backblaze purposes.

Plus, my X570 Aorus Master motherboard has a 1GbE and a 2.5GbE port, so would also need to factor is costs for an extra 10GbE card for my desktop too if going the NAS-as-a-DAS route.

 

Last edited: May 30, 2024

eeh you gotta decide if you really wanna jump into the rabbit hole or not,

USB protocol/interface on its own isn't really great for any critical storage,
if you want reliable storage, then it needs to be self-operable computer basically,
you can build DIY NAS (or storage server) from some older PC literally,
maybe you don't need 10Gbe for a start, frankly 2.5Gbe or even 1Gbe might be more than most people realistically need,
not sure how did you calculate NAS being expensive (or you meant Backblaze specifically?),
NAS/server can also provide iSCSI over network, so PC sees directly attached storage, another variant for you to consider?

 

My drives are 4 x 18TB = 72TB of drives. At $9 per TB for a NAS, that's over $600 per month with Backblaze. If its a DAS, backing it up to the cloud is within the set $9 month fee as it includes "unlimited data".

It's not critical storage either. My aim is to have 2 enclosures, each set up in RAID0 and cloning each other, giving me speed benefits of RAID0 but a duplicate enclosure for backup.

I have 18TB internally in my desktop (1 x 2TB NVMe with OS and 2 x 4TB NVMe and then 2 x 4TB SSDs) that's my critical daily driver stuff.

The enclosures are to back up the desktop onto, as well as to store things I don't necessarily use daily/regularly, so more a repository of thing less used.

 

Last edited: May 30, 2024

1st you should continue to educate yourself on raid. and raid products. there are more solutions than you know of and more methods of data transfer than usb3.

most people solve a problem, by looking for answers. You need to look around, you've decided on a single solution but can't make it work. that would lead me to suspect its not the solution.

try looking on ebay at the vast array of raid solutions, read up on the different models. make a choice after you understand what your choices are.

for example
https://www.ebay.com/itm/2355787971...xAH3G55b/y0JA+ZyKZpfrotw==|tkp:Bk9SR7zdjJ75Yw

Terramaster D5-300 - USB 3.0 - Raid Storage System - 5 HDD Bays



or

https://www.ebay.com/itm/1562243366...6zDpLOzuEWdQX/23sRZZVCdQ==|tkp:Bk9SR77djJ75Yw



try these search categories

https://www.ebay.com/sch/i.html?_fr...irewire&_sacat=0&_odkw=raid+storage&_osacat=0

https://www.ebay.com/sch/i.html?_fr...+storage&_sacat=0&_odkw=raid+server&_osacat=0

https://www.ebay.com/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=raid+server&_sacat=0

good luck

no good thing comes without effort...

 

I bet you don't really need 72TB backed up,
I mean, at such capacity you should organize things by tiers and priorities,
no solution will be economically viable at such scale,
if you want cheap backup, then have another NAS elsewhere...

 

No, but with 2 enclosures in RAID0 and the enclosures mirrored, it's 36TB each and between audio projects, video projects, as well as other stuff like software backups, Kontakt libraries, personal content, media etc, I'm currently at around 20-25TB and feels like it's always expanding

 

This feels like a classic Garamondo patronising and stating the obvious post....no offence

I have been doing tons of research and come across hundreds of devices, including the ones you listed. They are USB3.0 or USB 3.1 Gen 1, and therefore limited to 5Gbps, or in real world probably around 400-450MB/s read/write rates. With each Seagate IronWolf Pro capable of a potential 285MB/s read/write speed, I wouldn't even get the benefit of putting 2 disks in RAID0 with 5Gbps USB 3 Gen 1, as the 400-450MB/s would only be about 1.5 drives at potential speeds of 285MB/s per drive. Also, some of these USB 3 Gen 1 devices also use SATA2 instead of SATA3 ports, further limiting speeds, and why devices like the QNAP TR-004 is almost capped at about 250MB/s, even if you put in 4 IronWolf Pro drives in RAID0.

I also considered adding a Thunderbolt port to my machine, but unfortunately as I own the Gigabyte X570 Aorus Master motherboard, I am limited to the Gigabyte GC-Titan Ridge 2.0 Thunderbolt 3 PCI add in card, which is disconuted, no longer on sale anywhere, and like gold dust on the used market or sell for 3 times the original price due to their compatibility with Macs/Hackintoshes and prevalence in that space. The current Gigabyte GC-Maple Ridge Thunderbolt 4 PCI add in card is not compatible with my motherboard and I've reached out to Gigabyte support who have confirmed this.

Therefore, presuming I've not done enough research or considered other options is a little bit a dead end I'm afraid

See, it's easy Googling random keywords and then posting random eBay links in a forum. I think I'm looking for more constructive and useful information though

 

Last edited: May 30, 2024

Since this is a "for the rest of my life" DAS, how are you going handle non-Windows devices in your setup? What if in 4 years you want to add a Mac laptop to the mix? Or a Linux desktop further down the line?

 

The same way as I plan to accommodate expanding capacity if needed.

Reformat one of the enclosures and set up for new machine, copy over other enclosure content onto it, then reformat and set up the second enclosure, then set up the auto-cloning, meaning both would now be working the same way with any future machine. Or at least that's the theory in my head as any foreseeable-future machine should read NTFS without issue one would hope.

Or, worst case scenario, reformat all drives and enclosures and just download everything from Backblaze onto one enclosure, and then locally copy that one onto the second enclosure.

 

Hey Bunford, on German eBay.. is 180 euros 3 times the original price?
https://www.ebay.de/itm/176234853879?epid=10034694257&itmmeta=01HZ59QZHTQAD07Y6B7VVXVJ1C&hash=item29086a75f7:g:i1kAAOSwAW9lyQXx&itmprp=enc:AQAJAAAA8B7hpYkkQxfKobnasKUdDEhfd9KTJNJfxubY7j5xorSgyKe1MVzy9sh/Xa57SnPdFhiEL4NKguEQ5pQrRyyFsKie8CXIdXkJ6BAQVPiDmBvyDjQA/9PCj0q+IYjzBAVDnbv7OEVXJlKRWbAFb/uQlrsX54uBlI7zwU77DX4NNRQwbu10j49TNmhDQQhvBjT6ghei/reYTQbnyk/Dx5T4CwTT0QQIxNGb21Qrt15eZKHH7abUvGWqiPAobOsb87fQVxhYFFdUTH7vB6M8+6jgSKmAfQW3vqw2jbVisNE9lsfhpnfAAFIVoQHXK54Tc0FahA==|tkp:Bk9SR_j436n5Yw

If you still have money left, you could also buy this box: https://www.orico.cc/us/product/detail/8521.html

 

treat it like an information Technology project.

write up a specification of what you wish to accomplish, include what you already own.
once you have a specification
start browsing technology to see what fits your bill, consider firewire,scsi and fiber drive arrays. There are likely technolgies I've left out.
These interfaces are used in broadcast and medical industries - for a reason, they are robust and are expandable.

You don't need cutting edge usb 3.x?? to do what it sounds like you want to do. But you do need to write up what you want to do, so you can look for errors of reasoning as well as any missing pieces.

its your project, only you can make it a failure or a successs. I don't know all your needs, as well as your budget or your space and power requirements.

To solve your problem you need to describe as precisely what you wish to accomplish and how much you want to spend, then you need to figure out how reasonable that is based on hardware prices and what your technical abilities are. can you rebuild a used array? how many watts will it draw, where do i have space for it, is the ventilation sufficient. Can I step down a tier in tech and afford to buy 2 arrays, so I have an onsite spare? etc, etc.

don't get so hung up on usb3, consider scsi. a scsi cable physically locks to the socket making it impossible to accidentally unplug it during a read or write, not so with usb3

sorry if you think I'm condescending. in the future I'll just block your requests so you won't have to deal with me...

 

This is what I was saying. The scalping and profiteering from these is insane! Thanks for the heads up, but seems too expensive for me still, more out of principle than hard cash. I might go for USB 3 Gen 2 and catch up with Thunderbolt when I upgrade my machine when we’re at Thunderbolt 7 or something with another different connector

 

@MRFEENIX yes I've been experimenting with AI for some time...

 

In my experience dealing with a software RAID "solution" is not a good path to go down in the long run. No type of software RAID solution I have ever messed around with can compete with a proper Hardware RAID controller and a backplane setup to run RAID and support the full compliment of redundancy features RAID has to offer. I used a LSI/AVAGO controller when I implemented a SATA-6 SSD RAID solution back in 2013. I run Micron SSD drives which are certified for RAID configurations. The thing is still running strong to this day. The backplane used was a supermicro one that was included in a server enclosure I bought especially for the machine build in question.

Hardware RAID - from the controller, driver, backplane, drive cages, etc, will give you the reporting and the redundancy required of a true RAID system. If a drive(s) fails or becomes flaky you want to know about it as soon as it happens and be able to replace it and have the array(s) rebuild autonomously - even thru a system reboot. Software cannot do this nor can simple 20$ RAID "controllers". Admittedly SSD drives, if you have chosen them to be well suited for a RAID configuration will hardly ever fail... If you're implementing RAID I assume you're doing for it reason. It would pay in the long run to construct the system to last and provide the best solution possible.

 

Interesting you say that, because from my research, it seems like the industry is killing off hardware RAID and everyone moving across to hardware RAID. I assumed hardware RAID was better and safer, but this guy's videos led me down a bit of a rabbit hole to find out the manufacturers kind of agree that hardware RAID is all but dead and software RAID is the solution:

 

I've stumbled across a piece of software by StableBit called Drivepool and Scanner, which aren't strictly RAID, but is a solution many enthusiasts seem to swear by. It essentially allows you to use any combination of drives as a pool and also pools their performance apparently. It also have a automated duplication feature, in effects creating RAID1-like mirrors of everything on separate disks within the pool so you always have your data. As I understand it, it does this from one big pool without the need to instantly lose half your drives to a mirror, like RAID1, so I could assign 4 x 18TB to a pool, and as it fill up, it is duplicating writes onto 2 separate disks at the same time, so there is no lag/additional taks, as it utilises the available throughput to write things simultaneously. Alternatively, as I understand it, you can turn this default behaviour off and also choose to have it write the duplicates to a schedule if preferred, e.g. during night time/non-peak time use. It also writes pool identity files to each disk, so the pool can also be shared between machines and be fully recognised as one pool with the same behaviours on a different computer, and identified this way after a completely fresh OS install too, even if an entirely different OS.

The Scanner bit will also carry out scheduled scans (monthly by default) on all disks to asses S.M.A.R.T. data and other information to determine potential failures of a drives, and will then automatically begin a background process to evacuate the data on that drive to other drives in the pool as a safeguard before any failure occurs.

If a drive does fail, due to the duplications, there is no down time and Drivepool will allow you to hot swap in a drive, either as a replacement or to expand a pool, and it will redistribute the data according to its schedules and re-duplicate any non-duplicated files (if due to a disk failure).

To me, this appears to be everything and more that a RAID hardware controller could do, as you have to get to very expensive RAID controllers to do things like predict failures, evacuate vulnerable data, allow full RAID0-like performance during RAID1-like mirrored (duplicated) setup without have to write off all mirrored disk space until you get to that capacity usage.

From what I'm researching, it does seem to be a pretty good software solutions. I'm also reading some reports that Windows Storage Spaces is a Windows in-built RAID controller that's not called RAID essentially, and does a lot of things similar to Drivepool.

As I will only likely be using WIndows machines in the foreseeable future, one of these two appears to be the no-brainer solution for me.

 

The difference in speed between USB3 gen1 and 2 isn't really that important and there is always a certain, not small, overhead (especially when using network) so you will get really nice speeds either way but never the maximum possible speed. I'd just relax and go with a good 500MB/s up USB3 DAS that supports hardware RAID0 and be done with it. Remember that HDD speed varies significantly and manufacturers always tell you the maximum speed, never the lowest speed which is usually almost half maximum speed.

Nice thread, chaps. Well, for us "hardware professor" types. Enjoyed it with morning coffee. Got my synapses going.

 

Last edited: May 31, 2024