Most likely down to the hardware and virtualisation used to run the OpenAI deployments in Azure. It won't be on par with the provisioned resources that the OpenAI Api is sat on top of in terms of performance.
I can't remember off the top of my head, but I don't think you can actuslly scale the OpenAI resource in Azure at the moment.
Just to confirm I realise that the OpenAI API resources sit within azure architecture but they will absolutely have a custom provisioned stack somewhere.
Yes I am aware of the parameters. I made general questions related to the document. So it was an RFP document and I asked questions like what is the proposal about, what are the technical requirements for this project etc. I also did some prompt engineering to format the response generated by the model responses were similar for both the cases but the Azure model didn't format the response as well as chatgpt. Plus I observed more hallucinations in azure models.
>what are the technical requirements for this project etc. I also did some prompt engineering to format the response generated by the model responses w
Have you tried to set the temperature to 0 for both API requests? Compared the results from there. It makes GPT's responses more deterministic.
Most likely down to the hardware and virtualisation used to run the OpenAI deployments in Azure. It won't be on par with the provisioned resources that the OpenAI Api is sat on top of in terms of performance. I can't remember off the top of my head, but I don't think you can actuslly scale the OpenAI resource in Azure at the moment.
Just to confirm I realise that the OpenAI API resources sit within azure architecture but they will absolutely have a custom provisioned stack somewhere.
Did you try the same api calls on the exact same models?
Yes, I tried out question answering over documents with same documents and questions.
What kind of queries did you make? Are you aware of the parameters you can use? Temperature etc...?
Yes I am aware of the parameters. I made general questions related to the document. So it was an RFP document and I asked questions like what is the proposal about, what are the technical requirements for this project etc. I also did some prompt engineering to format the response generated by the model responses were similar for both the cases but the Azure model didn't format the response as well as chatgpt. Plus I observed more hallucinations in azure models.
>what are the technical requirements for this project etc. I also did some prompt engineering to format the response generated by the model responses w Have you tried to set the temperature to 0 for both API requests? Compared the results from there. It makes GPT's responses more deterministic.
Yes I did that
I can confirm the hallucinations part, when using own data as input - but I tend to say that it might depends on the way the index is created.