gianm commented on code in PR #13892:
URL: https://github.com/apache/druid/pull/13892#discussion_r1132571045
##########
processing/src/main/java/org/apache/druid/segment/UnnestStorageAdapter.java:
##########
@@ -226,9 +247,146 @@ public Metadata getMetadata()
return baseAdapter.getMetadata();
}
- public String getDimensionToUnnest()
+ public VirtualColumn getUnnestColumn()
{
- return dimensionToUnnest;
+ return unnestColumn;
}
-}
+ /**
+ * Split queryFilter into pre- and post-correlate filters.
+ *
+ * @param queryFilter query filter passed to makeCursors
+ * @param queryVirtualColumns query virtual columns passed to makeCursors
+ * @param inputColumn input column to unnest if it's a direct
access; otherwise null
+ * @param inputColumnCapabilites input column capabilities if known;
otherwise null
+ *
+ * @return pair of pre- and post-correlate filters
+ */
+ private Pair<Filter, Filter> computeBaseAndPostCorrelateFilters(
+ @Nullable final Filter queryFilter,
+ final VirtualColumns queryVirtualColumns,
+ @Nullable final String inputColumn,
+ @Nullable final ColumnCapabilities inputColumnCapabilites
+ )
+ {
+ class FilterSplitter
+ {
+ final List<Filter> preFilters = new ArrayList<>();
+ final List<Filter> postFilters = new ArrayList<>();
+
+ void add(@Nullable final Filter filter)
+ {
+ if (filter == null) {
+ return;
+ }
+
+ final Set<String> requiredColumns = filter.getRequiredColumns();
+
+ // Run filter post-correlate if it refers to any virtual columns.
+ if (queryVirtualColumns.getVirtualColumns().length > 0) {
+ for (String column : requiredColumns) {
+ if (queryVirtualColumns.exists(column)) {
+ postFilters.add(filter);
+ return;
+ }
+ }
+ }
+
+ if (requiredColumns.contains(outputColumnName)) {
+ // Try to move filter pre-correlate if possible.
+ final Filter newFilter =
rewriteFilterOnUnnestColumnIfPossible(filter, inputColumn,
inputColumnCapabilites);
+ if (newFilter != null) {
+ preFilters.add(newFilter);
Review Comment:
PR with the fix is at: https://github.com/apache/druid/pull/13919. If you
want to incorporate this into your own work then feel free and I will close my
PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
