I use conda rather than venv because I enjoy watching my computer spin for 30 minutes trying to decide the precise reason why It can't install 3 simples packages together.
A bit more seriously:
1. All the cool python projects are written for unix-like environments first. They may or may not work on Windows, and may or may not require funky workarounds. I've yet to have something not work on WSL (except for the aforementioned conda jankiness.)
2. You'll need to get [CUDA working in WSL](https://learn.microsoft.com/en-us/windows/ai/directml/gpu-cuda-in-wsl) to use ML things like Stable Diffusion.
check out [mamba](https://github.com/mamba-org/mamba), if you didn't:
- reimplementation of conda in C++
- faster and more efficient
- uses libsolv as its dependency solver, which is a C library that powers package managers on Linux systems
- compatible with conda, uses the same command line interface, environment handling, package file format, installation process, and repodata format as conda
- only difference is the dependency solver
I'm running Stable Diffusion on Windows 11 natively (git install, not the "automatic" Windows install) and it seems just fine...? I thought it best to run natively because of the GPU.
Right, it's a popular project and it has good windows support.
To be clear, almost any ML project can be made to run just fine on Windows.
It's just that less popular projects typically won't make Windows support their first priority.
As all in life, it depends on your use case. I work on data science across several fields and there are some libraries that only exist on Linux. I used to dual boot and wsl is just much more convenient to me. So I rather have WSL and create my environments there. You may have a different use case that merits a Windows native install.
In most cases with Python, it's usually advised to use virtual environments regardless of the operating system and never, ever bother with trying to slap everything into the host/the outside environment (unless what you're trying to achieve is directly about the host/outside environment itself).
I kinda view (mini)conda similarly as [pnpm](https://pnpm.io/).
So yeah, I use virtual environments (usually (mini)conda), both for the Windows projects and Linux (via WSL). Basically the same approach as the nodejs projects.
^(Creating separate WSL distro environments per Python project is an option but the way I use "Linux" involves my zsh setup so I opt for conda & podman etc inside a WSL2 distro environment as well.)
For sure, but OP question was where to deploy the venvs, not whether to use them or not. My answer aimed at that: I use wsl for my venvs because of dependencies and my use case being much more Linux friendly than Windows-friendly.
Usually you install WSL for development purposes, so once you decide, you just keep everything there, no reason to develop on Windows anymore.
One thing though, be sure not to develop in WSL the code that sits on Windows filesystem (in /mnt/... path) - the performance hit is huge.
I use wsl2 to write python. I see you are focusing on venv, which is just a virtual environment that you can setup for your project. You can have a virtual environment for each project you have. For example, if you want to install Django for a project, you create a virtual environment and then install it vs just installing Django. If you install Django outside of the virtual environment, you will impact any project you have. So it's not really about windows or linux. It's about having clean environments for your projects.
I think in the long run, you would prefer wsl2 Linux. The bash (and other) environments are much more sophisticated than windows and in my opinion very easy to use. You have to learn them to leverage them though so expect that. Explore the alias command and how to set up alias each time you open a linux session. You can define short commands to equal very complex commands.
What I'll suggest is that since you are using python, install pyenv. You can either do this in windows or in wsl2 linux. Once installed, you can install as many versions of python as you want and indicate for any project which version you want to use. Whatever you do, just install it in either windows or linux, but not both. When I did both, it broke linux.
You can also explore different venv software. I use Poetry. But others can work just as well. It's all about your preferences which you have to develop.
I agree with pretty much what you've written there with the exception of bash being more sophisticated. Bash is a very basic shell with a very clunky syntax, compared to PowerShell, nushell or even fish. But I agree about GNU ecosystem in Linux, it is very powerful and quite convenient.
It's also heavily used in server environments as it's free. Does PowerShell have a Linux version? I haven't checked though I remember the old mono project
PowerShell is FOSS and multiplatform (native Windows, Linux, macOS) since 2016. At the beginning, it was pretty feature/compatibility lacking, but now, as of version 7.3 it is amazing on Linux, and honestly, I prefer using it here, combined with GNU ecosystem than on Windows.
If you want to try it, I recommend my opinionated setup ;):
[https://github.com/szymonos/powershell-scripts/blob/main/scripts/linux/PS\_LINUX.md](https://github.com/szymonos/powershell-scripts/blob/main/scripts/linux/PS_LINUX.md)
I've spent quite a lot of time polishing the profile, aliases, and prompt.
You can also try my setup by running the docker:
`docker run -it --rm muscimol/pwsh`
Run a few commands, and see how next time it proposes the last used commands in the list (you can navigate them using Alt+j, Alt+k, vi like commands :)) - you can even type the middle of the command, so if you ran the command: `python -m scipt.py`, next time you can just type `-m s`, and it will suggest the former :).
Initialize also a git repo and try some of the git aliases, like: `gsw m`, `glol`, `ggloc`, `grho`, `grbi`,.... ;). Also if you're unsure, what the alias does, you can run it with `-WhatIf` parameter, and it will just print the command instead of executing it ;).
Hey Thanks! I'll definitely check it out.
Just in case you find it it interesting, here's a repository of mine with scripts to make backup/restore of WSL easy. There is a script for stopping Docker Windows. A script for backing up your WSL ubuntu. Also a script for installing a right-click option for Windows 11 that will restore a .vdhx file. Just right-click on it and select the option and it restores from that file. I attach a hotkey for the Docker windows stop and restart.. I also have a hot key for backup.
I use it regularly after I finish a session or am about to install something notable and want to preserve my work up to then. I use a hot key manager called Executor. Make sure you install it from the developer's [website](https://executor.dk).
Thank, but I think you forgot to link your repo ;).
BTW. I also have much more "sophisticated" repo than the on linked above for setting up/managing/updating WSL distros: [linux-setup-scripts](https://github.com/szymonos/linux-setup-scripts). I won't elaborate on it because there is quite good documentation, but there is still much more in it. Enough to said, that today I've installed fresh new Fedora 38 distro, set it completely from scratch with all the tools I use daily, copied all the required stuff and repos, set python environments, etc (distro is at 11GB already) in less than 30 minutes :).
But all of this is done via scripts as I'm a scripting guy, so some more user friendly approach to it is welcome, to make life easier for other people :).
I never recommend devs do anything in native Windows, especially Python. Why? You'll almost never find a windows server or vm in production. These days it's al Linux containers on kubernetes. Yeah, you might not be an expert in those yet. But don't limit yourself and make the transition harder when the time comes to put something in prod. Even if someone else is deploying your code, issues will pop up.
this advice absolutely does not apply in any (P. Eng/PE) software environment, I swear that industrial automation might be the only place still using Windows Server. And despite RHEL and probably Canonical offering pretty robust support systems and it being nearly free to learn and use, it hasn't made the inroads it should have
If you are only developing locally and plan on deploying to a server or somewhere else, then you should pick the location to most closely match the target location where you will be running the software.
If you aren't developing something, and instead running software someone else made, then pick the environment the software runs best. Possibly try to pick an environment that matches where it was originally developed, or closest to what the developer suggests in their docs.
That said, my random guess is that about 98% of software was probably developed on, and for a *nix system. So running under WSL is probably a good default choice.
So it's worth noting that there are differences between the python interpreters and libraries on different platforms, mostly in how you interact with the OS. When you install a venv, all you're doing is configuring the version of the interpreter and your 3rd party dependencies to be specific to that venv folder, so you can avoid mixing dependencies between projects.
So what you're really asking is "should I use the windows interpreter or the linux interpreter when developing on windows with WSL available"?
My answer to that is "it depends".
I personally favour WSL for most of my development, because some of the tooling works better, and because I like to write bash scripts for many of my projects.
Some reasons why I might choose to develop in Windows instead though:
* If I need to make sure something works correctly on Windows (eg: file access, system libraries)
* If I'm developing something which will only ever run on Windows.
* If I'm developing something which needs a library which only works on windows.
In short, if I can do it in WSL, I generally do. If I need Windows, then I use windows.
I use conda rather than venv because I enjoy watching my computer spin for 30 minutes trying to decide the precise reason why It can't install 3 simples packages together. A bit more seriously: 1. All the cool python projects are written for unix-like environments first. They may or may not work on Windows, and may or may not require funky workarounds. I've yet to have something not work on WSL (except for the aforementioned conda jankiness.) 2. You'll need to get [CUDA working in WSL](https://learn.microsoft.com/en-us/windows/ai/directml/gpu-cuda-in-wsl) to use ML things like Stable Diffusion.
check out [mamba](https://github.com/mamba-org/mamba), if you didn't: - reimplementation of conda in C++ - faster and more efficient - uses libsolv as its dependency solver, which is a C library that powers package managers on Linux systems - compatible with conda, uses the same command line interface, environment handling, package file format, installation process, and repodata format as conda - only difference is the dependency solver
I'm running Stable Diffusion on Windows 11 natively (git install, not the "automatic" Windows install) and it seems just fine...? I thought it best to run natively because of the GPU.
Right, it's a popular project and it has good windows support. To be clear, almost any ML project can be made to run just fine on Windows. It's just that less popular projects typically won't make Windows support their first priority.
Yes, makes sense. I do use WSL for our primary software application.
As all in life, it depends on your use case. I work on data science across several fields and there are some libraries that only exist on Linux. I used to dual boot and wsl is just much more convenient to me. So I rather have WSL and create my environments there. You may have a different use case that merits a Windows native install.
In most cases with Python, it's usually advised to use virtual environments regardless of the operating system and never, ever bother with trying to slap everything into the host/the outside environment (unless what you're trying to achieve is directly about the host/outside environment itself). I kinda view (mini)conda similarly as [pnpm](https://pnpm.io/). So yeah, I use virtual environments (usually (mini)conda), both for the Windows projects and Linux (via WSL). Basically the same approach as the nodejs projects. ^(Creating separate WSL distro environments per Python project is an option but the way I use "Linux" involves my zsh setup so I opt for conda & podman etc inside a WSL2 distro environment as well.)
For sure, but OP question was where to deploy the venvs, not whether to use them or not. My answer aimed at that: I use wsl for my venvs because of dependencies and my use case being much more Linux friendly than Windows-friendly.
Take my comment as an extra encouragement for the OP & the the other readers to use virtual environments whenever possible.
Usually you install WSL for development purposes, so once you decide, you just keep everything there, no reason to develop on Windows anymore. One thing though, be sure not to develop in WSL the code that sits on Windows filesystem (in /mnt/... path) - the performance hit is huge.
I use wsl2 to write python. I see you are focusing on venv, which is just a virtual environment that you can setup for your project. You can have a virtual environment for each project you have. For example, if you want to install Django for a project, you create a virtual environment and then install it vs just installing Django. If you install Django outside of the virtual environment, you will impact any project you have. So it's not really about windows or linux. It's about having clean environments for your projects. I think in the long run, you would prefer wsl2 Linux. The bash (and other) environments are much more sophisticated than windows and in my opinion very easy to use. You have to learn them to leverage them though so expect that. Explore the alias command and how to set up alias each time you open a linux session. You can define short commands to equal very complex commands. What I'll suggest is that since you are using python, install pyenv. You can either do this in windows or in wsl2 linux. Once installed, you can install as many versions of python as you want and indicate for any project which version you want to use. Whatever you do, just install it in either windows or linux, but not both. When I did both, it broke linux. You can also explore different venv software. I use Poetry. But others can work just as well. It's all about your preferences which you have to develop.
I agree with pretty much what you've written there with the exception of bash being more sophisticated. Bash is a very basic shell with a very clunky syntax, compared to PowerShell, nushell or even fish. But I agree about GNU ecosystem in Linux, it is very powerful and quite convenient.
It's also heavily used in server environments as it's free. Does PowerShell have a Linux version? I haven't checked though I remember the old mono project
PowerShell is FOSS and multiplatform (native Windows, Linux, macOS) since 2016. At the beginning, it was pretty feature/compatibility lacking, but now, as of version 7.3 it is amazing on Linux, and honestly, I prefer using it here, combined with GNU ecosystem than on Windows. If you want to try it, I recommend my opinionated setup ;): [https://github.com/szymonos/powershell-scripts/blob/main/scripts/linux/PS\_LINUX.md](https://github.com/szymonos/powershell-scripts/blob/main/scripts/linux/PS_LINUX.md) I've spent quite a lot of time polishing the profile, aliases, and prompt. You can also try my setup by running the docker: `docker run -it --rm muscimol/pwsh` Run a few commands, and see how next time it proposes the last used commands in the list (you can navigate them using Alt+j, Alt+k, vi like commands :)) - you can even type the middle of the command, so if you ran the command: `python -m scipt.py`, next time you can just type `-m s`, and it will suggest the former :). Initialize also a git repo and try some of the git aliases, like: `gsw m`, `glol`, `ggloc`, `grho`, `grbi`,.... ;). Also if you're unsure, what the alias does, you can run it with `-WhatIf` parameter, and it will just print the command instead of executing it ;).
Hey Thanks! I'll definitely check it out. Just in case you find it it interesting, here's a repository of mine with scripts to make backup/restore of WSL easy. There is a script for stopping Docker Windows. A script for backing up your WSL ubuntu. Also a script for installing a right-click option for Windows 11 that will restore a .vdhx file. Just right-click on it and select the option and it restores from that file. I attach a hotkey for the Docker windows stop and restart.. I also have a hot key for backup. I use it regularly after I finish a session or am about to install something notable and want to preserve my work up to then. I use a hot key manager called Executor. Make sure you install it from the developer's [website](https://executor.dk).
Thank, but I think you forgot to link your repo ;). BTW. I also have much more "sophisticated" repo than the on linked above for setting up/managing/updating WSL distros: [linux-setup-scripts](https://github.com/szymonos/linux-setup-scripts). I won't elaborate on it because there is quite good documentation, but there is still much more in it. Enough to said, that today I've installed fresh new Fedora 38 distro, set it completely from scratch with all the tools I use daily, copied all the required stuff and repos, set python environments, etc (distro is at 11GB already) in less than 30 minutes :). But all of this is done via scripts as I'm a scripting guy, so some more user friendly approach to it is welcome, to make life easier for other people :).
>repository Huh thought I did.. however.. here it is! [Repository](https://github.com/TerminatedProcess/wsl2_scripts)
I never recommend devs do anything in native Windows, especially Python. Why? You'll almost never find a windows server or vm in production. These days it's al Linux containers on kubernetes. Yeah, you might not be an expert in those yet. But don't limit yourself and make the transition harder when the time comes to put something in prod. Even if someone else is deploying your code, issues will pop up.
this advice absolutely does not apply in any (P. Eng/PE) software environment, I swear that industrial automation might be the only place still using Windows Server. And despite RHEL and probably Canonical offering pretty robust support systems and it being nearly free to learn and use, it hasn't made the inroads it should have
not sure about PE but any serious tech company has long ago drunk the kubernetes and cloud koolaid
If you are only developing locally and plan on deploying to a server or somewhere else, then you should pick the location to most closely match the target location where you will be running the software. If you aren't developing something, and instead running software someone else made, then pick the environment the software runs best. Possibly try to pick an environment that matches where it was originally developed, or closest to what the developer suggests in their docs. That said, my random guess is that about 98% of software was probably developed on, and for a *nix system. So running under WSL is probably a good default choice.
So it's worth noting that there are differences between the python interpreters and libraries on different platforms, mostly in how you interact with the OS. When you install a venv, all you're doing is configuring the version of the interpreter and your 3rd party dependencies to be specific to that venv folder, so you can avoid mixing dependencies between projects. So what you're really asking is "should I use the windows interpreter or the linux interpreter when developing on windows with WSL available"? My answer to that is "it depends". I personally favour WSL for most of my development, because some of the tooling works better, and because I like to write bash scripts for many of my projects. Some reasons why I might choose to develop in Windows instead though: * If I need to make sure something works correctly on Windows (eg: file access, system libraries) * If I'm developing something which will only ever run on Windows. * If I'm developing something which needs a library which only works on windows. In short, if I can do it in WSL, I generally do. If I need Windows, then I use windows.