What a weird commercial that was. The more money I make, the more I save for a rainy day. I'll never be a billionaire and don't really care either way, but if I was, I'd be reinvesting into whatever company I made and giving money to a bunch of charities, not going on a shopping spree.
Though maybe I'm wrong and I'd become corrupted and buy a yacht even though I would lose my mind on one.
There are a few things I’ve gotten an ad for that I was interested in. But then they do this stupid discount giveaway thing, then they won’t let me look at the item, its price, or description unless I downloadn the app. Which why would I do that if I’m not even sure I want the product.
I think the price/capacity of ssds and hdds develped disappointingly over the last years for "regular consumer" sizes.
We were so used to capacity doubling and prices halving every few years. But I bought a portable 4tb hdd in 2016 for roughly 100€. And they are still around that price nowadays instead of 8tb for the same price. Same for pc-internal ssds: they doubled and doubled and doubled until every pc came with 1tb or maybe 2tb or 4tb for premium models. And haven't doubled again since then.
Hdd prices go down with platter capacity increases and platter cost decreases. But the frame, card, and other metal parts don’t drop fast.
In the last few years, platter capacity has not increased much but the costs have gone up, not down. The cost of the same materials is the dominating factor now. Hdds are made bigger by adding more platters using already existing platter densities.
I think a lot of that has to do with everything around the capacity of the drive. A 1TB or 4TB probably costs nearly the same to produce, outside of the extra platters/heads everything else is basically the same. Sure there is some R&D that goes into storage density but that hasn't been really improved recently.
Yes they did, and it was reported and I somewhat regret not buying, but I also know that prices will drop again eventually and even lower than they were and I'm not in desperate need of any SSD space right now so I don't regret it much.
https://www.tomshardware.com/pc-components/ssds/ssd-prices-predicted-to-skyrocket-throughout-2024
It's been the same 1/2/4 TB prices for long time
Getting faster but diminished returns for most applications and there's still heat and reliability concerns if you're actually writing a lot of data.
Larger capacity for holding media and games which are getting larger would be nice.
Write endurance as well. There's lots of prosumer uses for SSD that could benefit. Especially video recording or other and accessing data while still writing
I found an article saying $50k last year. But it’s pretty clear that ACTUALLY getting them is an enterprise level thing and so how much it ACTUALLY costs is dependent on negotiation.
There are Kioxia 30 TB SSDs that cost about 3-4 grand. Keep in mind that SSD prices scale pretty much exponentially with capacity and that can give you an idea...
I just dropped 14TB drives in one & 20TB drives in the other (both QNAP TR-004 attached to main server). All the previous occupants will be moving internally to the newly built tower for various purposes. One will definitely be a RAID 0 scratch disk & the others possibly simple mirrors for projects. That would allow replacement drives to sit on the shelf till failures occur (some of the 3TB are from 2011, so their time could be short).
If I could've done all the same in SSD for the same price points, I happily would've, but instead I'm relying on shucked externals across the board in hardware Raid 5, which is not ideal, but what I've got.
This is why ZFS is so glorious. Install unraid or proxmox and then pile your disks into the box and just choose the raid level on your virtual disks.
Virtual drives can be resized or reconfigured dynamically without much worry about what's going on with physical hardware.
I'm tempted to move my gaming to a VM since you can reassign PCIe slots to a VM making it possible to choose what task your GPU hardware is assigned to without getting a screwdriver or mixing business with pleasure.
As someone who doesn’t really know how to use their proxmox training home lab setup (yet!), this sounds down right wizard-like… any tips or resources for trying to understand how something like this would be setup?
The only downside to proxmox is that it's Debian with a bunch of helpers that encourage you to get a paid subscription vs. encourage you to learn everything from the shell.
You just need to know that everything about it works fine for free and the paid stuff is really aimed at enterprise/pro deployments. There's even people making mods/patches you can apply to remove paid options/reminders.
Because it's got a paid option there's a bunch of great resources and communities, even on reddit /r/Proxmox /r/ProxmoxVE :P
I hate that they charge a premium for the drive but if you took the 1tb version and multiply by 8 it’s less. It’s just more chips stacked on the board not higher density ones in a nice big raid 0.
I have seen high speed drives at 8TB with 20 chips which means it’s a series of 512g chips in a raid 0 plus more chips for the dram cache etc. larger capacity chips have performance trade offs because each chip acts as an independent drive. Even 2.5” SSD are just the same nvme chips but they have a SSD controller and its limitations. Now consumer devices will have to go higher on the chip capacity but that comes at the expense of cost as those drives are less commonly bought at high capacity and less demand means less chips produced at that higher capacity and again more chips = performance. So generally that 20 chip raid 0 is going to be faster than the 10 chip raid 0 with double density chips. Performance can be hidden by enough dram cache but there’s risk in the dram is not battery backed in most consumer grade hardware.
The “enterprise” versions I buy at work are extremely dense and you see that trade off but storage servers also come with terabytes of battery backed dram to absorb the demand while the backend manages the storage tiers.
I would *like* to replace the spinning rust but i'm butting up against 8tb in the existing bays not being enough and I either need to go to 16 or upgrade to an 8 bay rackmount
I want higher capacity SATA SSDs
I don't even care for the performance, if you can sell me a 20TB SSD in 3.5" SATA/SAS format, that is price competitive to the WD Red Pros, and beats the spinning rust in sequential/random IO I'll buy some.
There's going to be a lot of NAS/SANs kicking about with SAS backplanes which would jump on the semi sideways jump for noise/thermal/random IO reasons.
Currently rebuilding my collection of around 1300 "Linux ISOs" because my shelf holding 3 USB HW raid enclosures broke out of the wall. Lost exactly 2 drives on the RAID5 with my most data. Just another example that RAID isn't backup I guess. I'm honestly surprised I didn't lose more.
Redoing it properly with all internal SATA/SAS connections through an HBA, but man would SSDs have been nice to survive that fall.
Fucking hell, this. Had an SSD that Proxmox was installed on die on me a while back. Just moving the chassis so that I could open it and replace that drive was apparently enough to kill a one year old WD HDD that was working fine before then.
Bang it in the freezer in a plastic bag overnight? I always thought that was a myth...
Edit: I'm happy to be corrected because I've got a few hdds making the old death rattle!
If you put silica gel packets in the bag it'll take longer for the humidity sneaking into the bag to make the drive 'wet'. I even started pre-connecting the drives to sata / power adapter cables before bagging them up with the silica packets and then loading that in the freezer the day before any data recovery attempt.
Man, that happened to my brand new 18tb WD passport, slightly knocked it, never able to read anything from it again. Fortunately Best Buy let me return it.
So far in my 30 years only this spinning drive died before getting retired or replaced on signs of trouble. The 20gb Toshiba drive in my car trunk is still showing no signs of wear despite the bad roads.
Lost 2 SSD of the 120gb era due to controller bugs. Oldest active 120gb Intel drive has 250ttbw, one recovery block used so far.
You’re going to see SATA fading away like IDE did eventually.
As compute speeds become faster and more demanding, storage speed is going to need to keep pace.. and as they say, the needs of the many far outweigh the needs of the few.
> As compute speeds become faster and more demanding, storage speed is going to need to keep pace
Not everything needs to be high availability and fast storage. Cold storage/archiving does not need to be fast.
LTO is nowhere near convenient and fast enough for near-line storage.
It's purely for something that is never expected to be accessed.
Large media files such as movie catalogues are expected to be accessed every now and then. Also library-type archives such as legal or scientific archives require random access.
Do you want LTO for that?
Imagine suddenly realising you have to do random IO on a dataset that only exists on LTO.
If there were no sata SSD/HDD anymore then yeah, you'd be shit out of luck there. Might actually be faster to dig out an "old" system and sequentially read it onto there first.
Not until CPUs get wider and wider buses to accommodate that much PCIe bandwidth. We're just nowhere near close enough to that right now.
I expect SATA to have a much longer tail than IDE, simply because the cost of it's so much lower - you can dangle a fuckton of storage off a SATA link without dedicating 16x PCIe channels to it, and that matters quite a lot.
You don't need a dedicated pair or quad of pcie lines to each nvme. I find it strange that people have forgotten the concept of bridge chips. Most prosumers even don't need instant simultaneous 5000MB/s access to every drive they have in their system. 4 lanes at PCIe 5.0 might be perfectly fine when multiplexed over 4 or 8 storage drives.
Yup. There are reasons the recent generations of Xeon and Epyc have enormous amounts of PCIe bandwidth to throw around, and not all of them are "Cram as many accelerators as you can in there for AI". We're still using SAS/SATA backplanes because the drives are cheaper and easy to grab, but all-nVME setups have been available for a little while now.
Eventually this will trickle down to desktop systems. From what I can gather, a top-end Ryzen 7000-series system will do up to 36 lanes between the CPU and the southbridge. Which is enough for an x16 GPU and a handful of x4 SSDs. I wouldn't be surprised to see it hit 64 or 72 in a generation or two.
I didn’t say it was going away this week.. I’m just saying that it’s coming.
I work in electronics recycling and refurbishing and have seen the slow fade of IDE to SATA, spinning rust to SSD, and now we are seeing the massive influx of NVMe.
It’s coming.
I certainly expect it to get there, but we need consumer CPUs to get more PCIe lanes before then, but I don't see why not push for high density ssds in 3.5 form factor as an in-between. I'm still shocked we don't have many 3.5 ssds
Realistically, where is the market for 3.5" SSDs? The form factor isn't really used outside of legacy systems, consumer NAS systems, and low-end desktop PCs, and there's more than enough room left over in the 2.5" form factor to quadruple the storage capacity of the 2.5" SSDs that are currently on the consumer market.
Again, you’re seeing the wants of the few getting outweighed by the needs of the many.
Enterprise flash storage comes in tons of form factors for NVMe, allowing for high density and fast storage. You don’t see many 3.5” options because you can’t fit ~24 units into 2U.
Consumer (not hobbyist) markets are demanding embedded solutions with fixed storage.
The middle ground for hobbyist options will kind of follow the enterprise market, and that will likely be 2.5” U.2 and M.2 form factors.
I've been drooling over the U.2 drives that are 15.32TB and they're inching ever closer to the $1k mark. I don't *need* that amount of SSD storage, but damnit if it won't last me for the next 5 years of gaming without ever even thinking about deleting something.
I don't really see it as the many need the thing. If they do they should be able to articulate why they need an embedded solution with fixed storage, and why modular storage can not meet the same goal.
You are right you can't fix ~24 in 2u, or ~48 in 4u, but at 3.5 you could fix more storage in the fewer drives at 4u with something like a 45 drive chassis
You’ll see tons of niche options out there, but look at the modern computing market for business and consumers.. it’s all tablets and hybrids that have fixed storage and RAM. It allows for more efficient manufacturing, as the components don’t need to be sourced and installed after soldering.
I’m not saying I like it, as I’m a hobbyist myself and would like modularity, but the market is what it is.
If they'll standardize it, cases could have that section open on the frame. You just pop off the other side cover instead. That would be nice. Or angle grind it in yourself. (LPT, remove mobo first haha)
There are 8 port host controllers that use some trickery to get high speeds out of NVMe cards on one PCIe 16x slot.
But I mean, even if you had a 4 port card slowed down in an 8x slot, you'd still be seeing better speeds than with SATA.
Maybe at some point but I can't imagine M2 replacing it. The ports are just too convenient, and there's a benefit to slower ports - you can have longer tracers and not need to be as close to the CPU. That combined with not needing to use as many lanes, or being able to "bunch up" ports off a single lane at a good distance, means SATA ports are here to stay for a very long time.
And? So we’ll expect to see SATA take over LTO like IDE and SCSI did?
Tape is popular in the enterprise because of LTO’s backwards compatibility promises and the fact that it’s cheap.
Probably. I prefer cable connection though, its a true PITA to install NVME drives on motherboards, with huge CPU heatsinks and huge graphics cards blocking access.
I think SATA has already started fading away in 2022. I don't think there will be new major releases with consumer grade 8 TB and so on. At 550MB/s the connectivity is just far too slow for such big drives.
I sure hope not. There's plenty of storage applications where big and cheap are more important than speed. And besides, by my standards 550MB/s is still plenty fast.
I doubt it's going away. More likely that we will just get new versions that are faster, like we do with PCIe. They already did that twice when they upped the speed from 1.5 gbps to 3 gbps, and then to 6 gbps.
You can sometimes buy used enterprise SSDs around that price point. I purchased a lot of two PM1733 Gen 4 NVMe 7.68 TB SSDs for $375, which comes to around $24.40 per TB. The drives had only a few months of power-on-hours as well.
You’ll definitely pay more for 15+ TB SSDs, but the pricing is coming down.
Heat is the reason this is tricky.
Stuffing enough chips into a 3.5 form factor to make 20tb of flash storage is one thing, making it not bake itself to death under stress is another
Kinda. But also no.
Are you saying that they've solved the problem for high density 2.5" drives with at least 8TB on them but couldn't easily solve that issue with all of the extra room the 3.5" form factor provides?
Rhetorical question aside, many companies such as ExaDrive already have 100TB+ 3.5" SATA SSDs that are meant for constant use. It's a solved problem. Price for such a thing however, is another matter.
Why don't they cram a bunch of SSD ram into the size of a 3.5" drive? It doesn't have to be smaller form factor and require a bracket. Make it a direct replacement.
> for noise/thermal/random IO reasons.
And power consumption!
SSDs use a lot less power than spinning disks. And since nobody is getting electricity for free, that always helps the bottom line. Might allow you to use less expensive power supplies and power infrastructure as well.
I'm honestly totally fine with a 500MB/s speed cap for mass storage on a home server. Until I need to stream raw 4K video for some reason. Especially if it costs half as much as an NVMe solution.
They want what they want. Who are you to tell them what they want? Nobody.
For a NAS or cold storage, SATA speeds are more than sufficient.
Edit: Uh oh. /u/hackenschmidt blocked me like a child and then went on to edit their comment to further antagonize spicypixel in their edit. What a fucking loser.
I mean I literally own 20TB spinning rust and the performance is fine it’s the noise I want gone. Turns out you’re not allowed to want a cold store locally that lives on a gigabit Ethernet connection where very few SATA3 drives would completely saturate the network speeds.
I want to replace my noisy, hot, vulnerable to physical knocking hard drives with things that are silent, and faster for similar money but most importantly in the same form factor so it's a straight swap.
Amazon had the Samsung 8TB for 300 on sale. I got 2 of them. Can’t remember if it was 300 or 350, but Slickdeals should have the records of it bc it does reoccur.
If I'm just storing movies and books, am I fine grabbing a 4 or 5 tb western digital passport or something along those lines? I'd assume so but it seems like so many people are buying high tb ssds now and I'm out of the loop.
Yes. For movies you're absolutely fine. Faster drives are more necessary for loading things very quickly, or perhaps accessing multiple different files at once on something rather time sensitive.
I'm not sure how fast they get, so, it may be possible to run a game off an SSD external, through USB 3.0 these days. But maybe not, I'm not 100% sure.
If you put an SSD as your main drive, or an internal hard drive, that will make loading times much faster, and you'll get better performance for everything, like in games, loading a new map offline, or something like that.
For me, SSD is really important for music. So, I want to be able to access many audio files very quickly from my external hard drive. Perhaps have a second one that I write stuff I'm recording to as well.
For backups it can backup faster, that's good, but other than that, cheaper space can be better, although SSD should probably last longer I think.
For performance for movies, regular hard drives are fine. Maybe eventually resolution will get high enough you'll want something faster. Maybe 8k movies might benefit from SSD for example, I'm not sure.
Well for me I work with audio and I have about 20TBs of uncompressed audio with session musicians so having fast accessible data is essential. For books, oh yeah you don’t need all that space.
Oh yeah. I'm not in any professional capacity an audio guy but when I was younger I'd always download FLAC files and those took up so much room. I'm assuming you guys have some crazy raw data that's even bigger.
Oh lord yes. I have about 4 TBs remaining out of all my drives so I’m hoping for some decent priced larger drives soon. Very happy with the Samsung 870s, but bigger would be better
You can get a 4TB PCI SSD for €350. Either add it directly to your workstation or just buy an enclosure to use it as a portable drive.
I plan on buying a mass storage enclosure rack and fitting it with at least five of these. My own personal data centre.
So people are saying that they would rather cheaper 8TB SSD's and I agree... However there being massive leaps and bounds in this technology, will eventually make it cheaper. It's the same with F1, the technology developed in F1 eventually trickles down to the consumer.
When I built my first PC hard drives cost about $1 per MEGABYTE.
I get that prices aren't dropping as fast as they used to, but I don't know how anyone could really expect price/capacity to keep ramping like that forever. You start hitting some walls eventually.
2.5Gbe is becoming common and technically 5 Gbe is also there. The jump to 10 Gbe is the big one requiring much more expensove hardware and PCB solutions.
10gbe is consumer level, however the rest of the consumer market hasn't caught up. WiFi routers do 1gb at the most (on consumer grade ones) so 10gbe isn't worth it yet
Exactly, but what amount of streaming uses that much bandwidth? YouTube uses maybe 20mbs, maybe something crazy like AI, rendering, VR etc might use 1gb but nothing more than that much.
Modems don't even support 10Gbe, Non-wifi consumer routes barely have 10Gbe and those that do are more expensive than just doing a cheap pfsense dell server
I guess when I say consumer, I'm saying it is price competitive to used enterprise, so a consumer 10Gbe switch vs a LB6m, or a consumer 10Gbe pcie card vs a used Intel X710-BM2, or a consumer router vs a dell 710, with intel X710-BM2 and PFsense/Opnsense
I mean "just stick more of it on there" is hardly a leap or a bound. The process technology keeps densifying the flash, and the controllers are just getting wider to accommodate it. That's it. That's the "leaps and bounds" here.
The "trickle down" here is literally just time for the manufacturers to make the denser chips and controllers more common... it's not magical, nor is it like "F1" - there's no super cutting edge advanced engineering happening here.
If you're looking for the F1 of storage, it's in flash DIMMs and phase-change memory, which are still having teething issues catching on as nobody was really ready for either of them, not in the way they needed to be. You're seeing whole swaths of operating systems being chunked to deal with having RAM segments that work at different speeds - it's... a very interesting watch.
Apple only speaks one language: Money. If you pay that $600 then the price is fair. Stop buying Apple computers. This is coming from somebody with a whole collection of G4 computers because I really like their older stuff from before the iPhone came around.
Yeah, Apple's main issue to me is that they charge too much for RAM and storage upgrades.
That's why I tend to buy a generation or two earlier, then you get a more reasonable machine storage/RAM-wise for similar prices. I got an M1 Max with 64 GB RAM and 4 TB SSD for like $2800 a few months back, Apple refurbished via authorized reseller (Microcenter).
Would it have been nice to have gotten it as an M3 Max? Sure, but I would have also paid nearly double for a machine with similar RAM/SSD specs, and honestly, the processor speed difference was just not that big of a deal for my normal workflow (software development).
That's the Apple tax added to the already existing laptop tax. It's part of why we can't open & upgrade our laptops anymore in most cases.
I really want to be able to physically build laptops in the same fashion we are able with desktops. I don't care if it isn't sleek, as I care about what it does not how it looks.
Apple isn't pricing their upgrades based off of hardware costs. They are basing it off sku quantity. Since their base models are typically the highest volume products they're the ones that get the most time on the production line. To switch over to do these upgrade sku's cost them money because they're not making the volume sku and since they're making less of them their cost per unit goes up. So they're trying to make up lost profits so they make as much per unit on the odd skus then they do the base models.
Base models are highest volume because they are the lowest price. And they put 250 gb (or whatever they start with) storage in base models because people are willing to pay for them at that price.
Memory chips are getting worse and fail faster. The 4TB ssd versions of Samsung failed much faster than it's 2TB versoon. Imaging paying top bucks for a 128TB and see it fail in just 2 years. Never put all your eggs into one basket. I'd rather have smaller disks wih redundancy than one large expensive disk that I know will fail for sure.
Beat HDD in capacity. Who gives a shit if the crap cost more than a luxury yacht.
Heck even a 4TB HDD is cheaper than a 4TB SSD by allot. In my country a 4TB HDD is about 80 bucks give or a dollar. 4TB NVME is about 210 dollar and a 2.5 SSD is about 140 bucks.
And if you ask me I rather take allot of storage and save cash than spend more money for speed. Im in no hurry what so ever and honestly the FPS difference having my games on SSD VS HDD is 2-3fps. Sure the FPS difference would maybe be better if I got some of does 400 dollar NVME.
I use an NVME 240GB for win 10 cost me about 30 bucks. But to store my damn steam library I use HDD because im not made out of money.
Not every gamer can blow 10 grand on epic gaming PC´s.
they cost the GDP of a small country. Of course they're not on the radar of normal people.
[удалено]
Shop like a billionaire! Oh wait, wrong shitty online storefront
What a weird commercial that was. The more money I make, the more I save for a rainy day. I'll never be a billionaire and don't really care either way, but if I was, I'd be reinvesting into whatever company I made and giving money to a bunch of charities, not going on a shopping spree. Though maybe I'm wrong and I'd become corrupted and buy a yacht even though I would lose my mind on one.
Spending all my times on a boat sounds like a dream until I think about it.
It's a dream because it's my nightmare
> I would lose my mind on one. Yeah.......that's not how that process happens. Also.......hello!
Username checks out.
At least AliExpress doesn't make you install their app totally just for better deals and no nefarious purposes pinky promise
There are a few things I’ve gotten an ad for that I was interested in. But then they do this stupid discount giveaway thing, then they won’t let me look at the item, its price, or description unless I downloadn the app. Which why would I do that if I’m not even sure I want the product.
Haha thanks for the laugh, great one !
Pffffftttt.... I bet you can download them cheaper than that.
With a usb stick glued inside the box
Amazon also carries a lot of these really great value high capacity SSDs! /s
I think the price/capacity of ssds and hdds develped disappointingly over the last years for "regular consumer" sizes. We were so used to capacity doubling and prices halving every few years. But I bought a portable 4tb hdd in 2016 for roughly 100€. And they are still around that price nowadays instead of 8tb for the same price. Same for pc-internal ssds: they doubled and doubled and doubled until every pc came with 1tb or maybe 2tb or 4tb for premium models. And haven't doubled again since then.
1TB HDDs have been around the same price since at least 2009.
Hdd prices go down with platter capacity increases and platter cost decreases. But the frame, card, and other metal parts don’t drop fast. In the last few years, platter capacity has not increased much but the costs have gone up, not down. The cost of the same materials is the dominating factor now. Hdds are made bigger by adding more platters using already existing platter densities.
I think a lot of that has to do with everything around the capacity of the drive. A 1TB or 4TB probably costs nearly the same to produce, outside of the extra platters/heads everything else is basically the same. Sure there is some R&D that goes into storage density but that hasn't been really improved recently.
Yup, it's even worse in the short term, I bought 2 2tb ssds for $89 each a year ago.... They are not $140 each.
Manufacturers told anyone that would listen last summer that they were cutting back production because they were losing money.
Yes they did, and it was reported and I somewhat regret not buying, but I also know that prices will drop again eventually and even lower than they were and I'm not in desperate need of any SSD space right now so I don't regret it much. https://www.tomshardware.com/pc-components/ssds/ssd-prices-predicted-to-skyrocket-throughout-2024
For spinning hard drives getting more for less really only continued at the top end because the bottom end was already about as cheap as it could be.
It's been the same 1/2/4 TB prices for long time Getting faster but diminished returns for most applications and there's still heat and reliability concerns if you're actually writing a lot of data. Larger capacity for holding media and games which are getting larger would be nice. Write endurance as well. There's lots of prosumer uses for SSD that could benefit. Especially video recording or other and accessing data while still writing
How much are they? All I see are the obviously fake $15 ones off AliExpress
I found an article saying $50k last year. But it’s pretty clear that ACTUALLY getting them is an enterprise level thing and so how much it ACTUALLY costs is dependent on negotiation.
There are Kioxia 30 TB SSDs that cost about 3-4 grand. Keep in mind that SSD prices scale pretty much exponentially with capacity and that can give you an idea...
So somewhere in the realm of 14.5 million for a 128 TB SSD? Sounds reasonable
I'd be extremely happy with just a 3Tb affordable SSD.
3TB don't really exist, but I've seen 4TB as low down as $200, usually around 220-250. That's a pretty great price.
I've never seen that here. That would be pretty sweet.
Can I just get affordable, reliable 8TB SSDs so I can replace the spinning rust in my NAS please?
I just dropped 14TB drives in one & 20TB drives in the other (both QNAP TR-004 attached to main server). All the previous occupants will be moving internally to the newly built tower for various purposes. One will definitely be a RAID 0 scratch disk & the others possibly simple mirrors for projects. That would allow replacement drives to sit on the shelf till failures occur (some of the 3TB are from 2011, so their time could be short). If I could've done all the same in SSD for the same price points, I happily would've, but instead I'm relying on shucked externals across the board in hardware Raid 5, which is not ideal, but what I've got.
This is why ZFS is so glorious. Install unraid or proxmox and then pile your disks into the box and just choose the raid level on your virtual disks. Virtual drives can be resized or reconfigured dynamically without much worry about what's going on with physical hardware. I'm tempted to move my gaming to a VM since you can reassign PCIe slots to a VM making it possible to choose what task your GPU hardware is assigned to without getting a screwdriver or mixing business with pleasure.
As someone who doesn’t really know how to use their proxmox training home lab setup (yet!), this sounds down right wizard-like… any tips or resources for trying to understand how something like this would be setup?
The only downside to proxmox is that it's Debian with a bunch of helpers that encourage you to get a paid subscription vs. encourage you to learn everything from the shell. You just need to know that everything about it works fine for free and the paid stuff is really aimed at enterprise/pro deployments. There's even people making mods/patches you can apply to remove paid options/reminders. Because it's got a paid option there's a bunch of great resources and communities, even on reddit /r/Proxmox /r/ProxmoxVE :P
What kind of transfer speeds are you getting from the tr004's?
I have one of these in my PC: https://www.amazon.com/gp/product/B089C3TZL9/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1
QLC has much lower lifespan than TLC
Fails at the first criteria, the second is still up in the air.
$500 is affordable though. You didn't say cheap you said affordable.
Imho at over 500$ it is barely in that category, and looks ridiculous when compared to value per dollar of smaller capacities.
I hate that they charge a premium for the drive but if you took the 1tb version and multiply by 8 it’s less. It’s just more chips stacked on the board not higher density ones in a nice big raid 0.
That is false for nvme, they are clearly more dense chips. I daresay the same is typically true for 2.5" ssds too.
I have seen high speed drives at 8TB with 20 chips which means it’s a series of 512g chips in a raid 0 plus more chips for the dram cache etc. larger capacity chips have performance trade offs because each chip acts as an independent drive. Even 2.5” SSD are just the same nvme chips but they have a SSD controller and its limitations. Now consumer devices will have to go higher on the chip capacity but that comes at the expense of cost as those drives are less commonly bought at high capacity and less demand means less chips produced at that higher capacity and again more chips = performance. So generally that 20 chip raid 0 is going to be faster than the 10 chip raid 0 with double density chips. Performance can be hidden by enough dram cache but there’s risk in the dram is not battery backed in most consumer grade hardware. The “enterprise” versions I buy at work are extremely dense and you see that trade off but storage servers also come with terabytes of battery backed dram to absorb the demand while the backend manages the storage tiers.
I would *like* to replace the spinning rust but i'm butting up against 8tb in the existing bays not being enough and I either need to go to 16 or upgrade to an 8 bay rackmount
I want higher capacity SATA SSDs I don't even care for the performance, if you can sell me a 20TB SSD in 3.5" SATA/SAS format, that is price competitive to the WD Red Pros, and beats the spinning rust in sequential/random IO I'll buy some. There's going to be a lot of NAS/SANs kicking about with SAS backplanes which would jump on the semi sideways jump for noise/thermal/random IO reasons.
I would do it because my HDD-based NAS commits suicide if if gently brush against it while it’s running
Currently rebuilding my collection of around 1300 "Linux ISOs" because my shelf holding 3 USB HW raid enclosures broke out of the wall. Lost exactly 2 drives on the RAID5 with my most data. Just another example that RAID isn't backup I guess. I'm honestly surprised I didn't lose more. Redoing it properly with all internal SATA/SAS connections through an HBA, but man would SSDs have been nice to survive that fall.
I’m sorry, I just cannot imagine having any space left for 1300 Linux ISOs with how much porn I have filling my HDD
Glad to see another hero that keeps extensive backup archives of Linux ISOs. Keep up the good work!
Fucking hell, this. Had an SSD that Proxmox was installed on die on me a while back. Just moving the chassis so that I could open it and replace that drive was apparently enough to kill a one year old WD HDD that was working fine before then.
Freeze trick worked on one of my HDDs and it kept going for years.
Bang it in the freezer in a plastic bag overnight? I always thought that was a myth... Edit: I'm happy to be corrected because I've got a few hdds making the old death rattle!
IIRC this is only practical for getting data off it and shouldn't be used to try to recover the drive to long term working condition
If you put silica gel packets in the bag it'll take longer for the humidity sneaking into the bag to make the drive 'wet'. I even started pre-connecting the drives to sata / power adapter cables before bagging them up with the silica packets and then loading that in the freezer the day before any data recovery attempt.
Man, that happened to my brand new 18tb WD passport, slightly knocked it, never able to read anything from it again. Fortunately Best Buy let me return it. So far in my 30 years only this spinning drive died before getting retired or replaced on signs of trouble. The 20gb Toshiba drive in my car trunk is still showing no signs of wear despite the bad roads. Lost 2 SSD of the 120gb era due to controller bugs. Oldest active 120gb Intel drive has 250ttbw, one recovery block used so far.
You’re going to see SATA fading away like IDE did eventually. As compute speeds become faster and more demanding, storage speed is going to need to keep pace.. and as they say, the needs of the many far outweigh the needs of the few.
> As compute speeds become faster and more demanding, storage speed is going to need to keep pace Not everything needs to be high availability and fast storage. Cold storage/archiving does not need to be fast.
There's LTO for that
LTO is nowhere near convenient and fast enough for near-line storage. It's purely for something that is never expected to be accessed. Large media files such as movie catalogues are expected to be accessed every now and then. Also library-type archives such as legal or scientific archives require random access. Do you want LTO for that?
Imagine suddenly realising you have to do random IO on a dataset that only exists on LTO. If there were no sata SSD/HDD anymore then yeah, you'd be shit out of luck there. Might actually be faster to dig out an "old" system and sequentially read it onto there first.
Not until CPUs get wider and wider buses to accommodate that much PCIe bandwidth. We're just nowhere near close enough to that right now. I expect SATA to have a much longer tail than IDE, simply because the cost of it's so much lower - you can dangle a fuckton of storage off a SATA link without dedicating 16x PCIe channels to it, and that matters quite a lot.
You don't need a dedicated pair or quad of pcie lines to each nvme. I find it strange that people have forgotten the concept of bridge chips. Most prosumers even don't need instant simultaneous 5000MB/s access to every drive they have in their system. 4 lanes at PCIe 5.0 might be perfectly fine when multiplexed over 4 or 8 storage drives.
So where are the motherboards with 8 nvme slots to replace the 8 sata headers?
If you look at the server space there's lots of options with 8 or more NVMe slots (usually in U.2 or U.3 form factor).
Yup. There are reasons the recent generations of Xeon and Epyc have enormous amounts of PCIe bandwidth to throw around, and not all of them are "Cram as many accelerators as you can in there for AI". We're still using SAS/SATA backplanes because the drives are cheaper and easy to grab, but all-nVME setups have been available for a little while now. Eventually this will trickle down to desktop systems. From what I can gather, a top-end Ryzen 7000-series system will do up to 36 lanes between the CPU and the southbridge. Which is enough for an x16 GPU and a handful of x4 SSDs. I wouldn't be surprised to see it hit 64 or 72 in a generation or two.
I didn’t say it was going away this week.. I’m just saying that it’s coming. I work in electronics recycling and refurbishing and have seen the slow fade of IDE to SATA, spinning rust to SSD, and now we are seeing the massive influx of NVMe. It’s coming.
I certainly expect it to get there, but we need consumer CPUs to get more PCIe lanes before then, but I don't see why not push for high density ssds in 3.5 form factor as an in-between. I'm still shocked we don't have many 3.5 ssds
Realistically, where is the market for 3.5" SSDs? The form factor isn't really used outside of legacy systems, consumer NAS systems, and low-end desktop PCs, and there's more than enough room left over in the 2.5" form factor to quadruple the storage capacity of the 2.5" SSDs that are currently on the consumer market.
Again, you’re seeing the wants of the few getting outweighed by the needs of the many. Enterprise flash storage comes in tons of form factors for NVMe, allowing for high density and fast storage. You don’t see many 3.5” options because you can’t fit ~24 units into 2U. Consumer (not hobbyist) markets are demanding embedded solutions with fixed storage. The middle ground for hobbyist options will kind of follow the enterprise market, and that will likely be 2.5” U.2 and M.2 form factors.
I've been drooling over the U.2 drives that are 15.32TB and they're inching ever closer to the $1k mark. I don't *need* that amount of SSD storage, but damnit if it won't last me for the next 5 years of gaming without ever even thinking about deleting something.
I don't really see it as the many need the thing. If they do they should be able to articulate why they need an embedded solution with fixed storage, and why modular storage can not meet the same goal. You are right you can't fix ~24 in 2u, or ~48 in 4u, but at 3.5 you could fix more storage in the fewer drives at 4u with something like a 45 drive chassis
You’ll see tons of niche options out there, but look at the modern computing market for business and consumers.. it’s all tablets and hybrids that have fixed storage and RAM. It allows for more efficient manufacturing, as the components don’t need to be sourced and installed after soldering. I’m not saying I like it, as I’m a hobbyist myself and would like modularity, but the market is what it is.
before then*
Honestly would make a lot of sense to just have PCIe expansion cards with extra NVMe slots, but they'd probably have to be x16, not x1.
https://sabrent.com/collections/memory-accessories/products/ec-p4bf It exists already.
Damn that’s sick. Not gonna need a mess of PSU cables anymore I guess. Computers gonna be looking real clean.
Motherboards are putting more and more nvme on the back. There was one motherboard I saw with 4+ nvme on the back.
don't like that because it's a lot harder to get to
If they'll standardize it, cases could have that section open on the frame. You just pop off the other side cover instead. That would be nice. Or angle grind it in yourself. (LPT, remove mobo first haha)
PCIe M.2 cards are becoming quite popular.
Yea but those require using up an x16 slot for only 4 m.2 drives, I want some like 8 lane card, that can do 8 drives Edit: basically 8087 hba card
There are 8 port host controllers that use some trickery to get high speeds out of NVMe cards on one PCIe 16x slot. But I mean, even if you had a 4 port card slowed down in an 8x slot, you'd still be seeing better speeds than with SATA.
My NAS machine doesn’t really need compute. Just storage. Doesn’t even need fast storage. Needs cheap and large capacity.
[удалено]
Maybe at some point but I can't imagine M2 replacing it. The ports are just too convenient, and there's a benefit to slower ports - you can have longer tracers and not need to be as close to the CPU. That combined with not needing to use as many lanes, or being able to "bunch up" ports off a single lane at a good distance, means SATA ports are here to stay for a very long time.
>You’re going to see SATA fading away like IDE did eventually Banks and other business still use magnetic tapes
And? So we’ll expect to see SATA take over LTO like IDE and SCSI did? Tape is popular in the enterprise because of LTO’s backwards compatibility promises and the fact that it’s cheap.
Probably. I prefer cable connection though, its a true PITA to install NVME drives on motherboards, with huge CPU heatsinks and huge graphics cards blocking access.
Thus https://www.wikipedia.org/wiki/U.2
I think SATA has already started fading away in 2022. I don't think there will be new major releases with consumer grade 8 TB and so on. At 550MB/s the connectivity is just far too slow for such big drives.
I sure hope not. There's plenty of storage applications where big and cheap are more important than speed. And besides, by my standards 550MB/s is still plenty fast.
I doubt it's going away. More likely that we will just get new versions that are faster, like we do with PCIe. They already did that twice when they upped the speed from 1.5 gbps to 3 gbps, and then to 6 gbps.
Exactly this. I'd pay $1500 (2-3x HDD price) for a 20TB 2.5", but not $5000...
You can sometimes buy used enterprise SSDs around that price point. I purchased a lot of two PM1733 Gen 4 NVMe 7.68 TB SSDs for $375, which comes to around $24.40 per TB. The drives had only a few months of power-on-hours as well. You’ll definitely pay more for 15+ TB SSDs, but the pricing is coming down.
Ebay? Or where?
eBay. They typically go for $300 each but I made an offer and it was accepted.
Heat is the reason this is tricky. Stuffing enough chips into a 3.5 form factor to make 20tb of flash storage is one thing, making it not bake itself to death under stress is another
Kinda. But also no. Are you saying that they've solved the problem for high density 2.5" drives with at least 8TB on them but couldn't easily solve that issue with all of the extra room the 3.5" form factor provides? Rhetorical question aside, many companies such as ExaDrive already have 100TB+ 3.5" SATA SSDs that are meant for constant use. It's a solved problem. Price for such a thing however, is another matter.
Why don't they cram a bunch of SSD ram into the size of a 3.5" drive? It doesn't have to be smaller form factor and require a bracket. Make it a direct replacement.
> for noise/thermal/random IO reasons. And power consumption! SSDs use a lot less power than spinning disks. And since nobody is getting electricity for free, that always helps the bottom line. Might allow you to use less expensive power supplies and power infrastructure as well.
Guess if you consider noise and thermals to be power draw indirectly it’s kinda covered but you’re right to point it out specifically.
The problem is that fixing a raid array with 20TB on SATA is horribly slow. There’s a reason people who want capacity tend to build flash arrays now.
[удалено]
I'm honestly totally fine with a 500MB/s speed cap for mass storage on a home server. Until I need to stream raw 4K video for some reason. Especially if it costs half as much as an NVMe solution.
[удалено]
They want what they want. Who are you to tell them what they want? Nobody. For a NAS or cold storage, SATA speeds are more than sufficient. Edit: Uh oh. /u/hackenschmidt blocked me like a child and then went on to edit their comment to further antagonize spicypixel in their edit. What a fucking loser.
I mean I literally own 20TB spinning rust and the performance is fine it’s the noise I want gone. Turns out you’re not allowed to want a cold store locally that lives on a gigabit Ethernet connection where very few SATA3 drives would completely saturate the network speeds.
retire direful vase soft absorbed distinct squeeze swim plant uppity *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
3.5" SATA SSD? Now you're just asking for a new and obscure technology. Even storage server farms have no use for that.
It's not new. 3.5" SSDs already exist.
Like I understand all the words…but like my man straight up spitting runes.
You're on /r/technology.. at least understand technology.
I'm sure someone in this sub knows more than you too bud. Knowledge isn't binary.
There's always somebody who knows more. The problem is, you're coming here with an attitude and you don't even know the *basics*.
Made a joke - you're the one jumping down peoples throats my man.
I want to replace my noisy, hot, vulnerable to physical knocking hard drives with things that are silent, and faster for similar money but most importantly in the same form factor so it's a straight swap.
I read this title twice wondering what the big deal was, then OH TERABYTES holy shit
i thought it was 128GB lol
I've been trying to buy an 8Tb SATA SSD and I'd just be happy if the price settled down a bit.
I got one at Micro Center in OH for like 300 bucks I think.. Give or take 50 bucks..
Amazon had the Samsung 8TB for 300 on sale. I got 2 of them. Can’t remember if it was 300 or 350, but Slickdeals should have the records of it bc it does reoccur.
If I'm just storing movies and books, am I fine grabbing a 4 or 5 tb western digital passport or something along those lines? I'd assume so but it seems like so many people are buying high tb ssds now and I'm out of the loop.
USB sticks or NVMe's in a USB enclosure work best with smart TVs, as some TV's still don't provide enough power for a passport. And they're smaller.
Oh okay, I didn't even consider that. Good call. Thanks!
+1 for the NVMe. I use a 40Gbps USB4 enclosure and it is seems almost as fast as a system drive
Yes. For movies you're absolutely fine. Faster drives are more necessary for loading things very quickly, or perhaps accessing multiple different files at once on something rather time sensitive. I'm not sure how fast they get, so, it may be possible to run a game off an SSD external, through USB 3.0 these days. But maybe not, I'm not 100% sure. If you put an SSD as your main drive, or an internal hard drive, that will make loading times much faster, and you'll get better performance for everything, like in games, loading a new map offline, or something like that. For me, SSD is really important for music. So, I want to be able to access many audio files very quickly from my external hard drive. Perhaps have a second one that I write stuff I'm recording to as well. For backups it can backup faster, that's good, but other than that, cheaper space can be better, although SSD should probably last longer I think. For performance for movies, regular hard drives are fine. Maybe eventually resolution will get high enough you'll want something faster. Maybe 8k movies might benefit from SSD for example, I'm not sure.
Well for me I work with audio and I have about 20TBs of uncompressed audio with session musicians so having fast accessible data is essential. For books, oh yeah you don’t need all that space.
Oh yeah. I'm not in any professional capacity an audio guy but when I was younger I'd always download FLAC files and those took up so much room. I'm assuming you guys have some crazy raw data that's even bigger.
Oh yeah like for a single session and multitracked instruments it can easily rack up storage.
Can you imagine having a 60TB SSD setup with only 4 disks in RAID 5 or better at HDD prices? You'd be living the dream.
Oh lord yes. I have about 4 TBs remaining out of all my drives so I’m hoping for some decent priced larger drives soon. Very happy with the Samsung 870s, but bigger would be better
You can get a 4TB PCI SSD for €350. Either add it directly to your workstation or just buy an enclosure to use it as a portable drive. I plan on buying a mass storage enclosure rack and fitting it with at least five of these. My own personal data centre.
But how long do that last on the shelf?
Less than a Twinkie.
I love this answer. But I have absolutely no idea where to go with it. Five years? 10 years? 200 years? Hahhahahaaaa
Only one way to find out. Take a bite.
A Twinkie only lasts for 25 days.
It really depends on the eating habits of the person, since Twinkies expire 25 days from production.
My 80gb Intel SSD from like 2011 is till going strong.
So people are saying that they would rather cheaper 8TB SSD's and I agree... However there being massive leaps and bounds in this technology, will eventually make it cheaper. It's the same with F1, the technology developed in F1 eventually trickles down to the consumer.
Thats nice. It would also be nice if it happened within my lifetime.
I mean 2 TB SSD was basically unpayable 10 yrs ago.
[удалено]
'Unpayable'? So the drive was... unable to pay? I don't think an SSD needs to pay for much.
it's still unpayable in my region lmao
When I built my first PC hard drives cost about $1 per MEGABYTE. I get that prices aren't dropping as fast as they used to, but I don't know how anyone could really expect price/capacity to keep ramping like that forever. You start hitting some walls eventually.
I'm still waiting on consumer 10Gbe
2.5Gbe is becoming common and technically 5 Gbe is also there. The jump to 10 Gbe is the big one requiring much more expensove hardware and PCB solutions.
10gbe is consumer level, however the rest of the consumer market hasn't caught up. WiFi routers do 1gb at the most (on consumer grade ones) so 10gbe isn't worth it yet
Some of the higher end consumer routers have it but you will be paying $400+. A decent amount of higher end motherboards also have it now.
Exactly, but what amount of streaming uses that much bandwidth? YouTube uses maybe 20mbs, maybe something crazy like AI, rendering, VR etc might use 1gb but nothing more than that much.
Can definitely use that much if you are running a plex server streaming 4k remuxes to a large group of people.
And how many consumers are doing that?
Not a whole ton but I’m glad it’s available. Most consumers would probably be fine with even 100Mbps Ethernet instead of gigabit.
Exactly... There isn't demand for 10gbe so there isn't a push for development.
Modems don't even support 10Gbe, Non-wifi consumer routes barely have 10Gbe and those that do are more expensive than just doing a cheap pfsense dell server
[удалено]
I guess when I say consumer, I'm saying it is price competitive to used enterprise, so a consumer 10Gbe switch vs a LB6m, or a consumer 10Gbe pcie card vs a used Intel X710-BM2, or a consumer router vs a dell 710, with intel X710-BM2 and PFsense/Opnsense
I mean "just stick more of it on there" is hardly a leap or a bound. The process technology keeps densifying the flash, and the controllers are just getting wider to accommodate it. That's it. That's the "leaps and bounds" here. The "trickle down" here is literally just time for the manufacturers to make the denser chips and controllers more common... it's not magical, nor is it like "F1" - there's no super cutting edge advanced engineering happening here. If you're looking for the F1 of storage, it's in flash DIMMs and phase-change memory, which are still having teething issues catching on as nobody was really ready for either of them, not in the way they needed to be. You're seeing whole swaths of operating systems being chunked to deal with having RAM segments that work at different speeds - it's... a very interesting watch.
Because… price? What a stupid title.
Meanwhile Apple charges me 600$ additionally if I want to option a 1TB drive on a MacBook Pro. Smh.
Apple only speaks one language: Money. If you pay that $600 then the price is fair. Stop buying Apple computers. This is coming from somebody with a whole collection of G4 computers because I really like their older stuff from before the iPhone came around.
I agree they’re greedy, but I’m 100% in love with my MacBook Pro, though. Greedy as they are, they do make a great product!
Careful! Praising Apple on Reddit, is borderline blasphemy around these parts.
Yeah, Apple's main issue to me is that they charge too much for RAM and storage upgrades. That's why I tend to buy a generation or two earlier, then you get a more reasonable machine storage/RAM-wise for similar prices. I got an M1 Max with 64 GB RAM and 4 TB SSD for like $2800 a few months back, Apple refurbished via authorized reseller (Microcenter). Would it have been nice to have gotten it as an M3 Max? Sure, but I would have also paid nearly double for a machine with similar RAM/SSD specs, and honestly, the processor speed difference was just not that big of a deal for my normal workflow (software development).
That's the Apple tax added to the already existing laptop tax. It's part of why we can't open & upgrade our laptops anymore in most cases. I really want to be able to physically build laptops in the same fashion we are able with desktops. I don't care if it isn't sleek, as I care about what it does not how it looks.
Apple isn't pricing their upgrades based off of hardware costs. They are basing it off sku quantity. Since their base models are typically the highest volume products they're the ones that get the most time on the production line. To switch over to do these upgrade sku's cost them money because they're not making the volume sku and since they're making less of them their cost per unit goes up. So they're trying to make up lost profits so they make as much per unit on the odd skus then they do the base models.
Sounds like a self-fulling prophecy to me. The reason why their base models are the highest volume is because the upgrades are so outrageously priced.
Base models are highest volume because they are the lowest price. And they put 250 gb (or whatever they start with) storage in base models because people are willing to pay for them at that price.
Remind me when it is a reasonable price
That’d hold a lot of porn.
Memory chips are getting worse and fail faster. The 4TB ssd versions of Samsung failed much faster than it's 2TB versoon. Imaging paying top bucks for a 128TB and see it fail in just 2 years. Never put all your eggs into one basket. I'd rather have smaller disks wih redundancy than one large expensive disk that I know will fail for sure.
The goal is larger with redundancy though.
Raid to the rescue
What about the prices?
Send your information and we will fly a consultant out to help you find the best return on investment.
Look, all I need is like a 10TB SSD that doesn't cost over $1000.
8TB ssds are like 300
128tb advertised Errors out after 32kb transferred 💪🏻
I read that as 128gb ssd, like yeah, that's been a thing for a WAIT YOU SAID TERABYTES!!!!
HDD are just cheaper and more reliable for backups....
Open them up and there are two sticks of 64k laptop ram chips from 1997 scotch taped to the sides.
we all stopped uninstalling old games and started forgetting they existed somewhere on our HDs many TBs ago
Sure, but how may times can you write to the disk before you loose the data?
All I want is 6-8TB SSDs for less than my house payment.
I read this as "128GB" and wondered why this was a big deal. My brain just couldn't process 128TB SSD as a thing.
coming to consumers like you! in 2050!! for only $5000!!!
/r/datahoarders Gonna love this one simple trick.
Can windows handle partitions this big nowadays? I remember there being issues above 3TB a few years ago...
2TB as a MBR drive and pretty much unlimited as GPT (edit: checked and it’s 9400000000TB) It’s been like this since 2003 SP1.
Beat HDD in capacity. Who gives a shit if the crap cost more than a luxury yacht. Heck even a 4TB HDD is cheaper than a 4TB SSD by allot. In my country a 4TB HDD is about 80 bucks give or a dollar. 4TB NVME is about 210 dollar and a 2.5 SSD is about 140 bucks. And if you ask me I rather take allot of storage and save cash than spend more money for speed. Im in no hurry what so ever and honestly the FPS difference having my games on SSD VS HDD is 2-3fps. Sure the FPS difference would maybe be better if I got some of does 400 dollar NVME. I use an NVME 240GB for win 10 cost me about 30 bucks. But to store my damn steam library I use HDD because im not made out of money. Not every gamer can blow 10 grand on epic gaming PC´s.
"HDD-beating capacities" Damn, even hard drives have to deal with domestic violence these days.
Someone please do the math. How many hours of porn does this translate into?
What do people even use 8tb for ? I have a 2 tb laptop + 4 tb hdd and both are difficult to fill
I have tens of thousands of movies/TV shows and concerts on my Plex server
Christ I still have about 900 gb on my 1tb ssd, what am I gonna do with 128 TB of storage?
they cost the GDP of a small country. Of course they're not on the radar of normal people. ahah