voonhous commented on code in PR #18837: URL: https://github.com/apache/hudi/pull/18837#discussion_r3340174824
########## hudi-trino-plugin/src/main/java/io/trino/plugin/hudi/HudiUtil.java: ########## @@ -35,39 +38,53 @@ import io.trino.spi.predicate.Domain; import io.trino.spi.predicate.NullableValue; import io.trino.spi.predicate.TupleDomain; +import io.trino.spi.type.TypeManager; import io.trino.spi.type.VarcharType; import org.apache.avro.Schema; import org.apache.avro.SchemaBuilder; +import org.apache.hudi.common.config.RecordMergeMode; +import org.apache.hudi.common.data.HoodiePairData; import org.apache.hudi.common.fs.FSUtils; import org.apache.hudi.common.model.FileSlice; import org.apache.hudi.common.model.HoodieBaseFile; import org.apache.hudi.common.model.HoodieFileFormat; import org.apache.hudi.common.model.HoodieFileGroupId; import org.apache.hudi.common.model.HoodieLogFile; +import org.apache.hudi.common.schema.HoodieSchema; import org.apache.hudi.common.table.HoodieTableMetaClient; import org.apache.hudi.common.table.TableSchemaResolver; import org.apache.hudi.common.table.view.HoodieTableFileSystemView; +import org.apache.hudi.common.util.CollectionUtils; import org.apache.hudi.common.util.HoodieTimer; +import org.apache.hudi.common.util.Option; +import org.apache.hudi.common.util.collection.Pair; import org.apache.hudi.exception.TableNotFoundException; import org.apache.hudi.metadata.HoodieTableMetadata; import org.apache.hudi.storage.StoragePath; +import org.apache.hudi.util.Lazy; import java.io.IOException; import java.io.UncheckedIOException; import java.util.ArrayList; +import java.util.Collections; import java.util.HashMap; +import java.util.HashSet; import java.util.List; import java.util.Locale; import java.util.Map; +import java.util.Objects; import java.util.Optional; import java.util.Set; import java.util.concurrent.ExecutionException; import java.util.concurrent.TimeUnit; import java.util.stream.Collectors; -import java.util.stream.IntStream; import static io.airlift.slice.SizeOf.estimatedSizeOf; Review Comment: Intentional. The prepend helper only needs to add the minimal meta set the merger requires (record-key + partition-path); it is not meant to project the full five-column set. Rationale: - The event-time / precombine ordering field is appended separately via `tableHandle.getOrderingColumns()` inside `prependHudiMetaAndOrderingColumns`, so ordering is covered without `_hoodie_commit_time` (that field is usually a data column). - For `COMMIT_TIME_ORDERING` (and null merge mode), `getOrderingColumnHandles` returns an empty list - the `HoodieFileGroupReader` orders by instant / log-block internally, so a projected `_hoodie_commit_time` is not required. - `_hoodie_commit_seqno` and `_hoodie_file_name` are never merge inputs; they are only materialized when explicitly selected, which goes through the normal column-projection path rather than this helper. MoR + ordering is exercised end-to-end by the snapshot / `_rt` (base+log merge) tests: `STOCK_TICKS_MOR` (`SELECT symbol, max(ts) ... GROUP BY` after updates), `HUDI_STOCK_TICKS_MOR`, `HUDI_MULTI_FG_PT_V6/V8_MOR`, `HUDI_COMPREHENSIVE_TYPES_V6/V8_MOR`, and the timestamp-keygen MOR tables that carry a `precombine_field`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
