It appears that all requests taking more than 30 seconds to complete result in your API dropping the connection and returning a 503 - Service Unavailable error. All requests that take less than 30 seconds seem to work fine.
However, a prompt with 4k tokens, or more, will inevitably take more than 30 seconds to complete. How do we resolve this issue?
Junte-se à discussão - adicione o comentário abaixo:
Efetue login / inscreva-se para postar novos comentários
It appears that all requests taking more than 30 seconds to complete result in your API dropping the connection and returning a 503 - Service Unavailable error. All requests that take less than 30 seconds seem to work fine.
However, a prompt with 4k tokens, or more, will inevitably take more than 30 seconds to complete. How do we resolve this issue?