paul-rogers commented on PR #13168: URL: https://github.com/apache/druid/pull/13168#issuecomment-1272688227
The other thing to discuss is the ordered merge steps. As it turns out, the need to do the ordered merge is independent of how we do the sort. It doesn't matter if the cursor did the sort for us, or if we added a custom sort. In both cases, we have to generate a merge comparator that has the sort keys, with ASC/DESC sort sense, in the proper order. Merging requires two implementations: one for the "list" (i.e. list of map) format, the other for the "compact list" (i.e. list of array) format. We can assume that the compact-list rows for any one segment have the same set of columns, and that map-based rows have the same keys. That is, the set of column names will be whatever the query requests. The order of columns in the compact-list form _should_ (we should check) be driven by the order that columns appear in the query. But, we must anticipate that the _types_ of columns from different segments may differ. Druid allows column `c` to be a `long` in one segment, a `string` in another. I believe Druid has some rules for reconciling these type conflicts. This means that the comparator has to know how to compare `10` and `"foo"`. There may already be implementations for this in the existing comparators: we should check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
