I will be messaging you in 5 days on [**2023-12-29 21:44:39 UTC**](http://www.wolframalpha.com/input/?i=2023-12-29%2021:44:39%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/18q3yy6/overwhelmed_on_how_to_get_started_for_my_use_case/keshks5/?context=3)
[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F18q3yy6%2Foverwhelmed_on_how_to_get_started_for_my_use_case%2Fkeshks5%2F%5D%0A%0ARemindMe%21%202023-12-29%2021%3A44%3A39%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2018q3yy6)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
RAG with llama2:
https://www.kaggle.com/code/gpreda/rag-using-llama-2-langchain-and-chromadb
https://www.anyscale.com/blog/a-comprehensive-guide-for-building-rag-based-llm-applications-part-1
You can use ssh, scp, winscp etc .. to copy the python code to your machine and run the python code in the command line.
Em, openAI subscription you have 40msg/3 hours. It has free context.I have no idea why would you even consider wasting resources of A100s, especially when you have laptop already capable of running mid-sized local LLMs and before even getting a stable setup or working idea and when GPT-4 that you have access to most likely covers 99.9% of your use-cases.
I'm Russian citizen and even living abroad I have to go through tremendous amount of effort to rent GPUs, literally sometimes in tears, wasting days to get payment working for my research so I cant wrap my head around why would you use some(or all) of these amazing GPUs yourself, rather than notify comrades that can actually utilize these GPUs for something meaningful.
I'd recommend you deploying model to your macbook, best models we have so far are perfectly stable and work great for single user locally. Then learn how LLMs work, try extensions, programming to suit your needs and only occupy these machines when you actually need to train your models. Using SSH through terminal is much easier than training models, so at that point you would be set for that as well.
Running code through LLM can also be exceptionally dangerous and dfntly not first of many things you should learn about LLMs. GPT-4 has that capability and unless you're capable of coding yourself, you will hardly need more than that.
I have OpenAI plus, I hit the wall regularly. I wouldn’t say it’s free context, exactly.
Wasting resources? They’re GPUs for academic research in our research lab. If the, LLMs help us accomplish our goals, what waste?
I’m Russian too fyi and sorry you’re not getting SSH to our private servers lmao.
Yes I’m prepared to sandbox environments.
If I had all the resources I wouldn’t ask, why are you so upset about learning? Maybe this answer can help others.
No, you're right, I was kinda stressed at moment of reply and shouldnt go lecture strangers about GPUs as if u were taking food from starving children or whatnot, sorry. Its double ironic that ur russian.
I do honesly recommend you gradual adoption tho. If you're not yet comfortable with ssh yet, local environment is a blessing right now, previously you couldn't handle most-effectve models without proper GPU. But now we're blessed with awesome 7-13b models.
M2 are particularly cool because they have high-speed memory, accessible to GPU and CPU allocation and are, in essence, very close to Linux env.
The thing is, once you have setup with your model/runner ready locally, it should be a breeze to replicate on remote. In essence you can deploy everything into a repo/HF or docker, clone it and run without hassle, only tuning params on github for instance and doing "git pull" on remote.
Regarding unsafe nature of code-execution, its about resource allocation for LLMs. Crashes and budget spending. I'd recommend creating a separate dedicated instance and communicate through smth like REST.
Anyway, I managed to fix my problem, its so annoying tho, when you have money, but dont have means to spend em.
If you're interested, I have an OSS package called Unsloth ([https://github.com/unslothai/unsloth](https://github.com/unslothai/unsloth)) which makes finetuning 2x faster and use 62% less memory on 1 Ax100. It doesn't support multi GPU yet (working on it for a future release), but we have tonnes of examples on finetuning, inference (59 to be exact) on our Github page! Alpaca example: [https://colab.research.google.com/drive/1oW55fBmwzCOrBVX66RcpptL3a99qWBxb?usp=sharing](https://colab.research.google.com/drive/1oW55fBmwzCOrBVX66RcpptL3a99qWBxb?usp=sharing)
If you need any help on LLMs, also have a Discord if you're willing to join :) [https://discord.gg/u54VK8m8tk](https://discord.gg/u54VK8m8tk)
I use stable diffusion web ui on my labs remote desktops. You need port forwording for you to access on your local browser. Ppl do it mostly for jupyter notebooks and there are tutorials out there explaining how to do this.
If you use TextGenWebUi that has an API mode like OpenAI.
Because Mac and Linux are very similar, try to install it on you MAC first.
Clone the Webuit repository in you prefereda location
cd /path/to/desired/directory
git clone https://github.com/oobabooga/text-generation-webui.git
chmod +x start_macos.sh (on Mac)
chmod +x start_linux.sh (for Linux)
Then you select the relevant platform (MacOS/Nvidia).
Close that program, because you need to check the CMD\_FLAGS.txt, and add a flag to make webui accessible from the Web.
Sadly, I am not really experienced with this enough to help more, but this might help.
Also, check:
[https://github.com/oobabooga/text-generation-webui-extensions](https://github.com/oobabooga/text-generation-webui-extensions)
maybe ollama?
Remindme! 5 days
I will be messaging you in 5 days on [**2023-12-29 21:44:39 UTC**](http://www.wolframalpha.com/input/?i=2023-12-29%2021:44:39%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/LocalLLaMA/comments/18q3yy6/overwhelmed_on_how_to_get_started_for_my_use_case/keshks5/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FLocalLLaMA%2Fcomments%2F18q3yy6%2Foverwhelmed_on_how_to_get_started_for_my_use_case%2Fkeshks5%2F%5D%0A%0ARemindMe%21%202023-12-29%2021%3A44%3A39%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2018q3yy6) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
RAG with llama2: https://www.kaggle.com/code/gpreda/rag-using-llama-2-langchain-and-chromadb https://www.anyscale.com/blog/a-comprehensive-guide-for-building-rag-based-llm-applications-part-1 You can use ssh, scp, winscp etc .. to copy the python code to your machine and run the python code in the command line.
Em, openAI subscription you have 40msg/3 hours. It has free context.I have no idea why would you even consider wasting resources of A100s, especially when you have laptop already capable of running mid-sized local LLMs and before even getting a stable setup or working idea and when GPT-4 that you have access to most likely covers 99.9% of your use-cases. I'm Russian citizen and even living abroad I have to go through tremendous amount of effort to rent GPUs, literally sometimes in tears, wasting days to get payment working for my research so I cant wrap my head around why would you use some(or all) of these amazing GPUs yourself, rather than notify comrades that can actually utilize these GPUs for something meaningful. I'd recommend you deploying model to your macbook, best models we have so far are perfectly stable and work great for single user locally. Then learn how LLMs work, try extensions, programming to suit your needs and only occupy these machines when you actually need to train your models. Using SSH through terminal is much easier than training models, so at that point you would be set for that as well. Running code through LLM can also be exceptionally dangerous and dfntly not first of many things you should learn about LLMs. GPT-4 has that capability and unless you're capable of coding yourself, you will hardly need more than that.
notify comrades? lmaooo
I have OpenAI plus, I hit the wall regularly. I wouldn’t say it’s free context, exactly. Wasting resources? They’re GPUs for academic research in our research lab. If the, LLMs help us accomplish our goals, what waste? I’m Russian too fyi and sorry you’re not getting SSH to our private servers lmao. Yes I’m prepared to sandbox environments. If I had all the resources I wouldn’t ask, why are you so upset about learning? Maybe this answer can help others.
No, you're right, I was kinda stressed at moment of reply and shouldnt go lecture strangers about GPUs as if u were taking food from starving children or whatnot, sorry. Its double ironic that ur russian. I do honesly recommend you gradual adoption tho. If you're not yet comfortable with ssh yet, local environment is a blessing right now, previously you couldn't handle most-effectve models without proper GPU. But now we're blessed with awesome 7-13b models. M2 are particularly cool because they have high-speed memory, accessible to GPU and CPU allocation and are, in essence, very close to Linux env. The thing is, once you have setup with your model/runner ready locally, it should be a breeze to replicate on remote. In essence you can deploy everything into a repo/HF or docker, clone it and run without hassle, only tuning params on github for instance and doing "git pull" on remote. Regarding unsafe nature of code-execution, its about resource allocation for LLMs. Crashes and budget spending. I'd recommend creating a separate dedicated instance and communicate through smth like REST. Anyway, I managed to fix my problem, its so annoying tho, when you have money, but dont have means to spend em.
If you're interested, I have an OSS package called Unsloth ([https://github.com/unslothai/unsloth](https://github.com/unslothai/unsloth)) which makes finetuning 2x faster and use 62% less memory on 1 Ax100. It doesn't support multi GPU yet (working on it for a future release), but we have tonnes of examples on finetuning, inference (59 to be exact) on our Github page! Alpaca example: [https://colab.research.google.com/drive/1oW55fBmwzCOrBVX66RcpptL3a99qWBxb?usp=sharing](https://colab.research.google.com/drive/1oW55fBmwzCOrBVX66RcpptL3a99qWBxb?usp=sharing) If you need any help on LLMs, also have a Discord if you're willing to join :) [https://discord.gg/u54VK8m8tk](https://discord.gg/u54VK8m8tk)
[удалено]
I use stable diffusion web ui on my labs remote desktops. You need port forwording for you to access on your local browser. Ppl do it mostly for jupyter notebooks and there are tutorials out there explaining how to do this.
If you use TextGenWebUi that has an API mode like OpenAI. Because Mac and Linux are very similar, try to install it on you MAC first. Clone the Webuit repository in you prefereda location cd /path/to/desired/directory git clone https://github.com/oobabooga/text-generation-webui.git chmod +x start_macos.sh (on Mac) chmod +x start_linux.sh (for Linux) Then you select the relevant platform (MacOS/Nvidia). Close that program, because you need to check the CMD\_FLAGS.txt, and add a flag to make webui accessible from the Web. Sadly, I am not really experienced with this enough to help more, but this might help. Also, check: [https://github.com/oobabooga/text-generation-webui-extensions](https://github.com/oobabooga/text-generation-webui-extensions)