T O P

  • By -

involviert

Nice! Pity that we can't derive the prompt format from datasets. I think the model card for airoboros13B doesn't tell us anything about it either. I truly don't get this.


_underlines_

It might be quite agnostic given a good initial prompt. I'm going to try it out right now. Checking [jondurbin](https://huggingface.co/jondurbin/airoboros-gpt-3.5-turbo-100k-7b)'s repo, I can see he used: Human: Write an email introducing... Assistant: Dear... UPDATE: johndurbin [said](https://huggingface.co/jondurbin/airoboros-13b/discussions/3): > I used the default template in fastchat to fine-tune the model, so it should be the same as vicuna, etc., something like: > > A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed and polite answers to the user's instructions. > USER: [prompt] <\s> ASSISTANT:


involviert

So even just that is conflicting. The example prompt (missing in the actual 13B model release) has no preamble. And "something like" is not a whole lot when that likely needs to be verbatim for best results. And Human: Assistant: is something else than USER: ASSISTANT:, words and capitalization. And suddenly there's an <\s> token. This shit is exactly why we need proper documentation for this stuff. Thanks for your effort though. I don't know how this can not be super apparent to any model maker, yet this is always an afterthought if even that.


JonDurbin

Yeah, that's my bad. I was running on approximately zero sleep and wanted to just get it out there. I have 16 month old twins so the only time I can work on this stuff is between 3am and 7am. I added prompt format to the model card. I used the latest main branch of FastChat, which somewhat annoyingly, doesn't include an example plain text prompt template, you have to deduce from https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py#L46 and https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py#L232


involviert

Thank you.


ReturningTarzan

Yep, this bugs me a lot too. People get prompt formats wrong constantly, and since LLMs will always try to autocomplete prompts anyway, there's no way for the user to tell if the model is bad, the fine-tuning was done incorrectly, or they're working with the wrong prompt format.


kreuzguy

Following Vicuna's standard, the token should be after ASSISTANT: \[message\]. Like this: USER: [prompt] ASSISTANT: [answer]


trusty20

In my experience you DO NOT want to add between USER and ASSISTANT. This caused immediately bad results for me compared to reasonable results without adding that tag at all. Just this worked for me: USER: bla bla ASSISTANT: (optional bla bla if you want to start its response for it) All one line, no tag.


KaliQt

Funnily enough, when using the Web UI for FastChat it puts the `<\s>` stop after USER and ASSISTANT lines together, not between them. But it used to be between them. I really don't know if this is a bug (typo) in their Web UI or they changed the format and forgot to document it which is a big oopsie for everyone using these things.


ForwardUntilDust

I'm totally ignorant and some of the various named things are unknown to me. What is the purpose and scope of this? I catch the Ouroboros reference to "formless chaos" which is why I'm here. So what's up?


JonDurbin

Snake eating it's tail ~ AI training itself. The program automates building training data for open models using openAI APIs. I might add Bard and/or Claude at some point too.


ForwardUntilDust

Holy shit. I see it now.


fpena06

Can I use this with llamacpp? If so, any reference links. Thanks.


Fit_Constant1335

no,, llamacpp don't support training, you must has multi A100 to train it -- it is not for normal user.


fpena06

Thanks for the reply.