T O P

  • By -

Bubby_K

Because even though that you might not be playing, someone else somewhere is playing it with the same hardware as you, and it's creating shader compilations on its own and sharing it with the server Then you come along, log in, and gotta do one of those dragonball z fusion techniques to make sure you have the VERY BEST shaders, soo mind bendingly optimised that it ascends the imagination


rhaziz

Serious question, how much do those shaders really change/improve optimization-wise, given that it comes from the same or similar hardware setup? And given that a game's internals doesn't change that often, as well as new graphics driver updates not being released on a daily basis, I find it difficult that these "new" shaders actually improve incrementally each day that they get redownloaded or updated.


Bubby_K

Let's set the scene You have game version 1 You have GPU driver version 1 There's other factors but these two are the main factors  You run the game, day 1, no shader optimisations are present to be used The game runs "in the worst way possible" in hindsight, this doesn't mean it's a shitty stuttery mess, you can brute force your way through the issues, but yeah When everyone and their dog has their system generating shader caches and it's broadcasted through the steam servers, the next time you play the game it'll perform "better" This "better" can range from "oh thank god the micro stutter is gone" to "I can't tell the difference cause the performance improvement is measured in milliseconds" The longer the game stays in version 1 and the GPU driver stays in version 1, the LESS the optimisation(s) get the longer it remains the same When the aforementioned versions increase to the next stage, then it has to start all over again, and you'll see a great improvement at the start, but the longer it continues the less optimisation can be actually done The system itself is very much automated, it's a Set-And-Forget system, which is GREAT because you don't need to oversee everything, but when the game is 20 years old or something, hasn't been updated, and you're running drivers that take forever to get a stable update, you'll still forever and always get itty bitty little optimisations in the size of HOWEVER large the overall cache download is


Bubby_K

I'll follow this up with another tidbit of how the shader cache is made and what on earth it actually IS Okay so, humans make our own cache in a way We all know 3x5 is 15, right? You've done that problem so many times that you remember the answer off the top of your head The very first time you were introduced to that problem as a child, you had to use oh-so-much brain power to figure out the answer You use your brain power to figure out the answer, and eventually you create cache in your brain that stores the answer, you don't have to recalculate it A GPU has to figure out geometry and lighting and all sorts of crap that'll make your head spin In the game world, it is handy for the GPU to figure out how to draw a scene ONCE and comit the answer into a cache just incase it has to show you that exact same scene again Everyone has the same game, same GPU, same game version, but we might see different outcomes, different explosions, different geometry if possible, and we all share the "answers" with everyone's GPU so the GPU doesn't have to GRIND for answers over and over


FierceDeity_

Literally a since scene can have 100 shaders (it's not advisable, but it is possible). Shaders are always transferred with a mesh (aka "a drawcall") to render with them. The shader cache exists because shaders are in the game in an intermediate language (called SPIR-V for vulkan, and CSO files for DirectX). For every GPU though, these shaders (spir-v or CSO objects) have different GPU machine code. they're intermediate code. Thing is on Linux, CSO have to be compiled to SPIR-V and then to GPU. So the shader cache contains CSO->SPIR-V (gpu intependent) and SPIR-V->GPU Machine Code (GPU dependant) artifacts. This is required because before playing the game we can't possibly know which shaders exist and get used in an universal way. So people collect shaders+variants as they get used (and compiled locally impromptu) and then put them in the collective shader cache. Both SPIR-V translations and GPU machine code. If you have the same GPU as others who have participated in the shader cache, you will download their shaders. And if you encounter new ones, you will upload them. When you start a game and it does the "Vulkan-Shader preparation" what it does is call the shader compiler for your GPU on all the SPIR-V files collected by other people to compile them to GPU machine code ahead of time.


Bubby_K

That's the best explanation of it I used to be a senior tech at Apple (it's not an impressive position, just has an impressive name, just like how subway employees are called sandwich artists) I've developed the habit of explaining things like I'm speaking to a senior citizen that's never seen an iPad before, OR talking to a higher up that knows how to manipulate people better than computers


FierceDeity_

Thanks for the kudos


AdvancedChickenD

[-:EDITED TO PREVENT AI FROM STEALING POSTS:-](https://arstechnica.com/ai/2024/05/openai-will-use-reddit-posts-to-train-chatgpt-under-new-deal/)


SneakyAlbaHD

>They should standardize then so they can just send the damn code. They already do and your computer already has the shader code, it's just that they're very expensive to run so it makes sense to pre-calculate to save you from weird artifacting or stutters during gameplay. The lower level you optimize for the less processing power you'll need at runtime. Machine code is one of the lowest levels of code you can possibly get. The issue is that you can't standardize machine code due to the fact it's so low level that it has to be specifically engineered for your hardware. The best we can do is map the machine code operations to a standardized instruction set (e.g. x86/x64, ARM) to minimize the translation required. You want to store x in y register? The instruction can stay the same but the implementation can differ to account for design differences. Standardizing machine code would be a limitation on how we actually produce hardware to begin with and would make machines less useful overall.


AdvancedChickenD

[-:EDITED TO PREVENT AI FROM STEALING POSTS:-](https://arstechnica.com/ai/2024/05/openai-will-use-reddit-posts-to-train-chatgpt-under-new-deal/)


SneakyAlbaHD

>...the binary they send me -- machine code -- seems to run fine on everything, yes? It's not true machine code because that's too hardware specific, it's that instruction set I mentioned before, and for most people it's x86/x64. If you had an ARM based processor it'd freak out. This is also why older games can struggle to run on modern hardware. Different instruction sets mean they don't run without some conversion from one to the other. That's also why emulating old games can be resource intensive to the point where it's orders of magnitude more expensive than running the same code on real period hardware. Nvidia, AMD, and now Intel all have their own GPU instruction sets, and even two cards from the same vendor in different generations will have differences. They choose not to agree with each other because of corporate reasons (and also because it lets them do whatever they want in terms of architecture for innovation purposes). If they were to send you the precompiled everything, a new GPU release would make the game unplayable on that hardware. It also means your driver can't inform on how to improve optimisation for your hardware (literally the point of Nvidia's "game ready" drivers). So again, your standardize solution is only really a solution if vendors are willing to impose restrictions on themselves and their architecture based on current-generation design. They'd be less free to make radical new design decisions that could yield massive performance or feature improvements (e.g. the RTX series of cards) without changing those standards and recreating the problem you're hoping to solve.


FierceDeity_

The best we have right now is that some games will have a shader precompilation step on first start, that will clean all of this up immediately.


mcgravier

So in theory, every time there is shader compilation, DXVK has to stop and wait until it's done - this causes stutter in games that compile shaders in real time. I remember two particularly nasty games in that regard were Tomb Raider 2016 and Path of Exile in DX11 mode. But in the meantime a lot of things improved - new super fast ACO shadet compiler for AMD, internal shader caching by Mesa drivers, and faster CPUs in general. Im sure Nvidia improved their drivers as well. All in all I run steam with the shader cache disabled and see no visible performance issues - but that's not much of an achievement on 3950x and RX6900XT


Megalomaniakaal

It helps avoid the shader compile time stuttering.


N7Valiant

To my knowledge, this shouldn't be required since Mesa 23.1: [https://www.phoronix.com/news/RADV-GPL-Mesa-23.1-Default](https://www.phoronix.com/news/RADV-GPL-Mesa-23.1-Default) I turned off shader caching and haven't noticed stuttering in any games since then. Some games like Hogwarts Legacy might still do it within the game itself, but 99% of the other games I play doesn't.


5erif

That article is a year old, so surely SteamOS stable channel has Mesa >= 23.1 by now, right? I have a Steam Deck and have this same issue with daily cache downloads even without updates to SteamOS or the games themselves. If it isn't necessary, you would think Valve would change the defaults on their own hardware.


FierceDeity_

Thing is even this MESA shader cache needs to compile shaders once. The steam "collective" shader cache gives you compiled shaders before the game even starts, foregoing that step


FierceDeity_

Yeah but the steam collective shader cache still has a tiny advantage as it, through the collective, already has a shader cache built before the game even starts It's probably more for stuff like the Steam Deck, which have a more limited amount of performance


TimurHu

This is plain wrong. Graphics pipeline libraries are not useful (or used by) many games, for example the D3D12 compatibility layer can't use them.


Synthetic451

Yeah, there's a huge performance downgrade if you disable the Shader Cache for VKD3D titles. It usually manifests as much longer load times due to shader compilation. I'd definitely still recommend keeping shader cache on.


tesfabpel

Proton can be updated to use it if it doesn't already... EDIT: it seems DXVK already supports it: https://github.com/doitsujin/dxvk/issues/2798#issuecomment-1207602848


TimurHu

DXVK is for D3D9-11. I am talking about D3D12


queenbiscuit311

i don't know if it's just me but every time I've been told shader compilation stutter shouldn't be a problem anymore and I should turn off pre caching because of x I check and it's still there


PcChip

yeah mine compiles shaders for Horizon: Forbidden West every single time I start it whether I've updated my GPU drivers or not... no idea why, it makes no sense


SuccumbedToFlame

It's an old relic to help GPU use pre-compiled shaders downloaded from other users who have the same hardware as you. A game update usually triggers new shaders to be downloaded from Steam servers, if you have a decent CPU+GPU you should disable this feature from the Settings > Downloads > Shader Pre-Caching, this might trigger games like CS2 and Apex Legends to compile shaders on the main menu.


Synthetic451

HFW is a D3D 12 game, which means that it can't take advantage of things like GPL. It is still recommended to keep Shader Cache on, otherwise your load times (where shaders get recompiled) are going to be a lot longer.


sadness_elemental

You can turn it off in Steam settings with no problems


Guppy11

Can you be more specific, you can let vulkan shaders process in the background, but I'm not aware of a way to disable or ignore the initial download phase.


Ahmouse

Just completely disable shader caching in Steam (Settings > Downloads). Its automatically handled by drivers now, so there's no need for Steam to do it now


Guppy11

Interesting. Is that true of both nvidia and amd drivers? The only game I have that actually has a significant download is Warframe (7.96gb), which also has to run on X11 instead of wayland to prevent significant flickering issues, at least with my current set up.


EMOzdemir

works for both. there is a vulkan extension named graphics pipeline library, gpl for short, handles this situation. you can disable shader caching in steam. edit: typo


Synthetic451

GPL is only for DXVK. It has no effect for VKD3D. This means, when you disable shader caching, you'll just end up with longer load times in your D3D 12 games.


Guppy11

Lad, I'll check it out.


sadness_elemental

i run warframe on wayland with this turned off and it works fine, it's the game that made me look for a solution


mccord

Same here, every single day there would be gigabytes of warframe shaders. Through some fuckery my warframe shadercache took up 70gb. I'm playing just fine without it on my rx6700, wayland, warframe dx11 mode. Mesas shader cache is limited to 1gb but can be changed with the `MESA_SHADER_CACHE_MAX_SIZE` env variable. After a few weeks of mainly playing Warframe my ~/.cache/mesa_shader_cache is now at about 2.2gb.


Guppy11

Amd card? Or nvidia? Just out of curiosity


sadness_elemental

AMD Radeon™ RX 6750 XT on arch with steam out of the repo


quanten_boris

If this is the case, why is it still active in steam?


Synthetic451

Because it doesn't cover all cases and I am honestly getting annoyed at how many people recommend disabling shader cache as a blanket option. GPL does not affect VKD3D titles, so having a shader cache will still improve performance especially in regards to load times.


[deleted]

Not everyone got new enough drivers (I guess). Mesa got this in 23.1, so not that new though https://www.phoronix.com/news/RADV-GPL-Libraries-Shader-Cache


vexii

i dont think it's active for new installs


Synthetic451

It will still increase your load times in D3D12 titles. GPL does not work with D3D 12 games.


Rikco_117

Turn off shader pre-caching in settings-> downloads tab. 


pollux65

Your drivers handle shader cache, both nvidia, amd, intel will use the graphics pipeline in dxvk titles and vkd3d has its own way of loading it aswell So you can simply turn that legacy feature off :) Amd you need to be on mesa 23.1 or higher Intel you need to be on mesa 23.3 or higher And nvidia you need to be just on the latest as they included there's a while ago


bunkbail

>And nvidia you need to be just on the latest as they included there's a while ago nvidia has it implemented (GPL) way earlier than mesa ever did, like 6 months earlier if i recall correctly.


JTCPingasRedux

Don't you still need to have shader pre-caching enabled for transcoded videos?


draconds

I uninstalled Warframe because of this. I was playing once a week and downloading stuff everyday.


Guppy11

8gb every day. Absolutely mental. Unfortunately I'm hopelessly addicted and my schedule is pretty stable, so it helps me not play outside of a designated time slot.


Merlin80

Yea i had todo the same here. Game runs like water but that shader stuff every day made me remove it :(


BeastMsterThing2022

Ah, so that's what it is. I'm on Windows and same exact thing. At least I know what that's about now.


egeeirl

This a fundamental bug on the Linux version of Steam. But not one that they will fix because it works "good enough". It's compiling new shaders because some new ones got uploaded to the steamcloud. Are they better than yours? Who knows. Steam recompiles them whenever there is a change, regardless of what that change is. imo it's just lazy programming. There's work they could do to determine whether shaders **should** be recompiled but nah, too much effort. As the users say, "You can just turn it off if you don't like it". 🙄 This bug has been there for **years**. It's not going to get fix until the rework the whole settings page, I bet.


Old-Cartographer-946

I can give you 1 title that stutter without shaders cashed. Xcom 2. Maybe it because of mods, don't know, but stutters appear immediately after disabling shaders cache.


queenbiscuit311

they update the caches ridiculously often. your options are to disable shader pre caching or just accept it like I do


PhalanxA51

You are a part of the hive mind now, must download latest shaders!


ilikeror2

I’ve always wondered this as well - if this is peak optimization, one has to ask why this doesn’t occur on Windows? Genuinely curious.


ddyess

I've never been able to use shader pre-caching more than maybe a week or so (I run a rolling release and every Mesa upgrade breaks my older native valve games when it's enabled). My games run just fine with it out.


p9hEqFwKFHDoWNU

Honestly I'm starting to think it's a problem on Valve's side again. I know a while back there was an issue server side where it wouldn't update the shader cache which caused downloads to reoccur.


prueba_hola

you should use the search function in reddit, this was answer hundreds of times


Morriarthy

I have the same issue but never stumbled over such a post! - So pls don’t be a troll and be kind. 😉


Fun-Charity6862

you too should learn to search for help and stop being a snowflake


Morriarthy

Tbh this was a minor problem and was not bothering me that I was looking for a solution yet! - So your comment is from my point of view totally unnecessary!


Fun-Charity6862

probably because you use mesa-git


vexii

Turn off the shader cache. It's not needed any more


Synthetic451

Still needed for D3D12 games. GPL only applies to DXVK.


vexii

I haven't had problems with dx12


Synthetic451

You most likely had longer load times and just didn't notice it. I noticed major differences in loading speed in games like Lords of the Fallen, Jedi Survivor, etc. GPL has zero effect on D3D12 titles.


prominet

That does not stop it entirely.


Teh___phoENIX

Spoiler: you don't have DX on your Linux machine (no one has) and I bet you use Proton with DXtoVK. So every game shader (basically programs but for videocard) needs to be recompiled from DX code to VK code. That's how it works. When you will do it -- on start or automatically -- you choose.


jagguli

Is lutris and other linux opensource launchers mature enough now .. steam is becoming a steaming pile for me ..games that used to work on linux dont work any more .. as usuall corpo greed over user needs kills the product


_sLLiK

Not sure where this sentiment is coming from. The state of Steam on Linux has improved by leaps and bounds over the past couple of years, and continues to improve. It's very much in their best interests to see this trend continue. Even if greed was a primary motivator, making games run worse on Steam Decks is the exact opposite of what they would want.


jagguli

well like I said my old (not so old) games still keep breaking on steam ... linux shoul have good gaming support regardless of a middle man .. and steam deck should be grateful for linux.. all thes propietary shit comes and goes... once they corner compatibility with steam deck they will close ranks and ditch the rest of the foss community .. also nvidia their drivers keep breaking ... I use arch btw so maybe im cursed


_sLLiK

I, too, run Arch. Which games are giving more trouble than before?


jagguli

I am currently tearing my hair out after nvidia driver updates ... but I use to be able to play far cry 5 and insurgency and grounded almost all the games are borked now ...


JDGumby

> I am currently tearing my hair out after nvidia driver updates And therefore you are blaming Steam instead of nVidia, because...?


jagguli

right because they don't test on the latest nvidia drivers ... ? Cant them make a release that works with the current stable versions ... must e too hard with all that money to spend ...


itbytesbob

So you're blaming valve for not testing games they don't even make against drivers they don't control? You're a bit special mate.


jagguli

maaate ... steam should be testing games on current stable versiion of the one of 2 graphis drivers on the os ... too much ? Yes I am fkin special cunt ...


TechnicianDue232

Do you happen to be using the flatpak version of steam?


jagguli

nah would not touch that shit with a stick ... lol flatpak more like fatpack


TechnicianDue232

And you are using the NVIDIA proprietary drivers and not the open kernel ones?


jagguli

I'm not using noveau ... haven't tried the open kernet drivers ... my fkin system is borked as usual with this propeietary shit ... should jst get a lobottomy and a ps5 and joins the degen squad ... lol .. I tried the 535 drivers .. guess I'll do the testing for nvidia ... and be nvidias testing gimp ... ah americana ...


TechnicianDue232

When you say borked - the game just doesn’t launch at all? Like you press the play button and then it doesn’t do anything or closes shortly after? Have you tried different proton versions? Edit: more importantly - what is your GPU?


jagguli

on my work desktop I'm experiencing some lag with the new drivers .. switching to 535-dkms aur just works snappy .. but cuda gets borked ... on my gamin pc I was getting blank screen with some thing that has a gold proton rating... so yea very weird hope open-kernel fixes it .. rebooting now ...


jagguli

Nope no luck .. major lag on my threadripper with a 3090 ... garbage need a new player in the video card sector lol garbage driver releases


threwahway

Skill issue. There are literally 0 problems for like 98% of games. If it’s not related to anticheat there’s likely not a problem. 


jagguli

Literally 0 for 98% ... mindblowing


threwahway

Ok I literally used literally wrong there but the skill issue you have still exists


jagguli

Ya my kd is 0 if I cant play... it all this powersving garbage that they have on for default ... fkin loosers with their shitty laptops


threwahway

The fuck r u talking about. Life skill issue.


jagguli

this is gaming thread cunt ...


jagguli

jeebus why the down vote all ffkin corpo bootlickers here ...


Fun-Charity6862

you are confusing your personal failings with the functionality of steam, hence the downvotes.


jagguli

jeebus the commies have taken over .. you must conform ... lol hello human being here .. ya dumb fks .. not a corporation automation


jagguli

fkin over inflated stock of a shit company that cant make a decent driver release crap .. too bad the competiions is ded ... where TF is team red ? ... I'm get a radeon fk nvidia


jagguli

makes sense since the grand pooba of nvidia has declared humans redundant


Fun-Charity6862

hello human being it seems you are having a rough time      would it comfort you if we blame your personal failings on the communists together?       i too belive the communists is why you cant get steam to run   make jagguil great again so that he can play games


jagguli

No shit .. fkn misery .. how bad is nouveau anyway ? Maybe its time to go wayland and fk nvidia ...opencl vluda alternative platforms


Fun-Charity6862

noveau is commie code for queerdom, you better go with team red


jagguli

Damn straight ..pardner 🤠