egalpin commented on code in PR #13742:
URL: https://github.com/apache/pinot/pull/13742#discussion_r1796222438
##########
pinot-core/src/main/java/org/apache/pinot/core/transport/QueryResponse.java:
##########
@@ -49,12 +50,12 @@ enum Status {
/**
* Returns the current server responses without blocking.
*/
- Map<ServerRoutingInstance, ServerResponse> getCurrentResponses();
+ Map<ServerRoutingInstance, List<ServerResponse>> getCurrentResponses();
Review Comment:
@jackjlli I believe I've found a way to maintain compatibility such that
brokers with this version could roll out and communicate with servers on a
prior version, while still gathering the information required. This is done
through borrow a single digit of the request ID (the least significant digit,
such that it does not increase the probability of a broker ID hash collision)
in order to use that bit to identify if a query pertained to a REALTIME or
OFFLINE table.
Because in this version servers will only receive 1 or 2 queries, always
targeting the same table "base" (i.e. foo of foo_OFFLINE), we can infer from
the table type REALTIME or OFFLINE which request the server response pertains
to. This system will not work in the future when a server will return responses
for many different tables, but as soon as we have server versions deployed
which always return the table name in the dataTable metadata, we can fully
dismantle this requestId hijacking. Using the request ID to convey table type
only needs to exist for the moments (or hours, depending on size of cluster)
where brokers are expecting table name in dataTable metadata, but servers are
not yet running the version which does so.
I feel like this provides a good bridge to be able to move forward with
subsequent Logical Table implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
