siddharthteotia commented on code in PR #8874:
URL: https://github.com/apache/pinot/pull/8874#discussion_r894723635
##########
pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableBuilder.java:
##########
@@ -288,9 +295,18 @@ public void finishRow()
}
public DataTable build() {
Review Comment:
Should we consider decoupling this from existing engine's data table ?
There is a long way to go before we adopt the new multi-stage engine for all
queries (I mean including the queries whose functionality is perfectly
supported by the existing engine).
There is already a need to bump the existing engine's data table to V4 (null
handling feature https://github.com/apache/pinot/pull/8872/).
Say that PR takes it to V4 and the multi-stage engine takes it to V5 in this
PR. Now couple of months later, we find something needs to be supported in the
existing engine and bump that to V6.
V6 is not necessarily an extension of V5 because the latter even though
borrows code and format from existing engine's DataTable, it has specific items
like block type (which V4 and V6 don't) and is very likely to evolve
independently. I think it will still work from upgrade and compatibility
perspective, but the version number management looks less clean.
So I am thinking if the version scheme should be decoupled between the
existing engine's DataTable and the multi-stage engine's data table and may be
the common code can continued to be borrowed via inheritance etc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
