yashmayya opened a new pull request, #13628: URL: https://github.com/apache/pinot/pull/13628
- With the v2 / multi-stage query engine becoming more robust and feature complete (https://github.com/apache/pinot/pulls?q=is%3Apr+label%3Amulti-stage+is%3Aclosed), existing users might want to potentially migrate their existing v1 query workload to the multi-stage query engine. - However, there are some differences between the two query engines in terms of supported syntax (some of which is documented in https://docs.pinot.apache.org/reference/troubleshooting/troubleshoot-multi-stage-query-engine). - In case a user wants to migrate from the v1 query engine to the v2 query engine, it would be useful to be able to automatically detect whether the existing query workload would work as is without modification. This patch adds a counter metric (optionally enabled via a broker configuration that defaults to false for performance reasons) that is incremented each time a v1 query is run that fails during compilation with the v2 query engine. If this counter remains 0 during regular query workload execution, it signals that users can potentially migrate their query workload to the multi-stage query engine. - Note that there could be situations where a query compiles successfully but fails during execution. There could also be cases where a query runs successfully but there are differences in the result between the v1 and the v2 query engine - in terms of result types, result set size etc. - In order to address these above concerns, we plan to also add a tool / API in the near future that actually executes a given query against both the query engines and compares the results. This is much more heavyweight and shouldn't be done for every query during regular execution and is intended more for ad-hoc use cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
