It appears that all requests taking more than 30 seconds to complete result in your API dropping the connection and returning a 503 - Service Unavailable error. All requests that take less than 30 seconds seem to work fine.
However, a prompt with 4k tokens, or more, will inevitably take more than 30 seconds to complete. How do we resolve this issue?
It appears that all requests taking more than 30 seconds to complete result in your API dropping the connection and returning a 503 - Service Unavailable error. All requests that take less than 30 seconds seem to work fine.
However, a prompt with 4k tokens, or more, will inevitably take more than 30 seconds to complete. How do we resolve this issue?