Agree that reusing correlation id won't work well.
I want to point out that the issue is larger than just unsupported
request types.
Any error that happens before we figure out the request leads to
silently closing the connection. Requests larger than
socket.request.max.bytes, or IPs with more con
Yeah I second what Magnus is saying, reusing correlation id to mean
something else isn't the right thing. This would actually break the Java
clients as well which are very pedantic about getting the right correlation
in each response.
-Jay
On Fri, Jan 16, 2015 at 2:34 AM, Magnus Edenhill wrote:
Magnus,
I think I buy your points, we can return the empty response with just the
response header, in which case the current JVM clients will throw
Serialization Exception, for other clients like kafka-python they will
throw similar exceptions also while trying to construct the response body.
Guo
It is a bit hacky, with multiple requests in transit on the wire how do
you know which request wasn't supported if the returned correlationId is -1?
Requests are currently responded to in the order they were sent, so the
corrId
is not yet critical, but with will this always be true? One could imagi
The "hacky" method that Dana suggests does not sound too hacky to me
actually.
Since such scenario will only happen when 1) new clients talk to older
server and 2) older clients talk to new server with some APIs deprecated,
and "correlation_id" is always set to meaningful numbers before, old
clien
I totally agree but I still think we shouldn't do it. :-)
That change would cause the reimplementation of ALL existing Kafka clients.
(You can't chose not to implement a new protocol version or else we are
committing to keeping the old version supported both ways on the server
forever).
The probl
There are benefits of moving the error code to the response header.
1) I think it is the right thing to-do from an implementation perspective.
It makes the most sense. You send a request and you get back a response.
The response tells you something is wrong in the header.
2) With such a large cha
Yeah, adding it to the metadata request probably makes sense.
What you describe of making it a per-broker field is technically correct,
since each broker could be on a different software version. But I wonder if
it might not be more usable to just give back a single list of api
versions. This will
Perhaps a bit hacky, but you could also reserve a specific correlationId
(maybe -1)
to represent errors and send back to the client an UnknownAPIResponse like:
Response => -1 UnknownAPIResponse
UnknownAPIResponse => originalCorrelationId errorCode
The benefit here would be that it does not break
Yeah I totally agree--using the closed socket to indicate "not supported"
does work since any network error could lead to that.
Arguably we should have a request level error. We discussed this at the
time we were defining the protocols for 0.8 and the conclusion was not to
do that. The reasoning w
> I think #1 may be tricky in practice. The response format is always
> dictated by the request format so how do I know if the bytes I got back are
> a valid response to the given request or are the UnknownRequestResponse?
On the other hand, from the client developer perspective, having to
figure
Yeah I agree there isn't a good way to auto-detect the presence of a given
API.
I think #1 may be tricky in practice. The response format is always
dictated by the request format so how do I know if the bytes I got back are
a valid response to the given request or are the UnknownRequestResponse?
We ran into similar difficulties, both when trying to get Kafka to use
new APIs when available and when testing the wire protocol.
+1 for all three suggestions.
#1 sounds like the bare minimum, but I'm not sure how much it will
complicate the clients (now we expect either a response or an Unknown
Hi all -- continuing on the compatibility discussion:
I've found that it is very difficult to identify when a server does not
recognize an api
(I'm using kafka-python to submit wire-protocol requests). For example,
when I
send a ConsumerMetadataRequest to an 0.8.1.1 server, I get a closed socket
14 matches
Mail list logo
