rohangarg commented on PR #12867: URL: https://github.com/apache/druid/pull/12867#issuecomment-1219774466
@cheddar and I had a discussion where we talked about alternative ways to get the `queryId` and `sqlQueryId` for user queries. The current way to fetch the ids was from response header which is a last resort rather than a principled way. The response header method isn't very scalable incase we want to add more metrics from the router, since all of them would have to wait for the full request to be completed and then would have to be emitted from the response listeners. The challenges with other approaches to fetch ids are : 1. We need to generate query ids incase the user hasn't provided them. This is needed to keep the metrics from routers and other services consistent for same user query. To check and generate the ids, we need to deserialize JDBC and normal SQL queries to a state where we can see the context map for them. 2. JDBC is a stateful protocol which works on the session state being managed by the broker. So, all the JDBC request won't have the ids in them (config params only allowed in connection-open request). Thus, incase a user is setting id via JDBC, while executing the actual SQL queries we won't be able to know about the ids. 3. Native query id for SQL can't be set from router for a query since the SQL might break into multiple native queries (union queries), and all of them then would have the same native id. As a result, we've decided to do the following : 1. For normal SQL queries, we use a deserialized version of the request and inject it with SQL query-id if the user hasn't overriden it. 2. We don't emit native query-id for any SQL queries to avoid any problems with union queries 3. For JDBC queries, we extract the SQL query-id from the response header since that is best we can do as of now. The above mentioned solution will allow to get the `query/time` metric for all SQL queries, while giving us some room to add more metrics atleast for normal SQL queries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
