xiangfu0 commented on PR #18140: URL: https://github.com/apache/pinot/pull/18140#issuecomment-5334564450
Closing this monolithic PR as superseded by the smaller, independently reviewed UUID implementation PRs. UUID v1 is now complete on `master`. ## Merged implementation Core layers: - #18869 — logical UUID type foundation - #18870 — ingestion and segment storage - #18871 — `DataSchema`, JSON, and Arrow result rendering - #18872 — server-side predicates, casts, and transforms - #18873 — aggregation, group-by, `HAVING`, distinct, and supported sketch functions - #18874 — multi-stage planner, UUID literal bridge, and runtime support - #18876 — focused end-to-end coverage for multi-stage equality joins and realtime UUID-primary-key upsert - #19091 — UUID scalar functions and multi-stage UDF wrappers - #19155 — UUID literals using the existing binary request-literal encoding Review-driven split-outs also merged: - #19181 — logical UUID `CAST` - #19182 — UUID bloom-filter support - #19183 — UUID `CASE`, `IN`, and comparison transforms Supporting work merged while the stack was reviewed: - #16471 — keep `UUID` and `VARIANT` non-reserved - #18400 — Avro/Parquet record-extractor logical-type contract - #18927 — external `java.util.UUID` / internal 16-byte conversion contract - #18952 — raw `BYTES` min/max correctness - #18969 — MV `BYTES`/`BIG_DECIMAL` `GenericRow` SerDe - #18970 and #18973 — recommender logical-type mapping and UUID sample generation - #19029 — logical scalar rendering in `jsonPathString` - #19073 — Avro schema generation from Pinot logical types Existing UUID byte/string helpers (#11988) and the UUID upsert-primary-key hash option (#12538) remain available. ## Why #18875 did not merge #18875 was closed intentionally. A dedicated UUID segment partition function is not required for the OSS v1 logical-type contract and would introduce a new raw-16-byte producer/Pinot hashing convention without a demonstrated interoperability requirement. Existing partitioning behavior remains unchanged; a specialized partitioning contract can be proposed independently if a concrete OSS use case emerges. ## Documentation The canonical user documentation is merged in `pinot-contrib/pinot-docs`: - pinot-contrib/pinot-docs#917, #936, and #940 — recommender, ingestion/storage, and result rendering - pinot-contrib/pinot-docs#952 and #964 — UUID functions and canonical string rendering - pinot-contrib/pinot-docs#968–#970 — logical casts, literals, comparisons, and predicates - pinot-contrib/pinot-docs#989 and #991 — aggregation and multi-stage query behavior The external documentation is the user guide, so the Apache Pinot root README does not need a duplicate UUID section. ## Final v1 contract - `UUID` is a first-class logical type backed by Pinot's fixed-width 16-byte `BYTES` representation. - Persisted values and engine row/key paths use `byte[16]` or `ByteArray`; `java.util.UUID` is used at typed Java and Calcite boundaries. - UUID query results render as canonical lowercase dashed strings. Plain `BYTES` behavior remains unchanged and renders as hexadecimal. - Single-value and multi-value UUID dimensions are supported, subject to the same operator-specific restrictions as other MV/array types. - Offline/realtime ingestion, Avro UUID interop, dictionary/raw storage, predicates, casts, ordering, grouping, distinct, supported sketches, MSE equality/hash joins, scalar functions, and UUID-backed realtime upsert are covered. - Existing `STRING` or `BYTES` columns are not reinterpreted automatically. Adopting UUID requires a new column/table and reingestion or backfill. - Segment storage and request literals reuse existing byte formats, but `DataSchema` carries new `UUID`/`UUID_ARRAY` logical tokens. Brokers and servers must be upgraded together before UUID result traffic is enabled; mixed-version query paths and rollback with UUID results in flight are unsupported. - Without column-based null handling, the nil UUID is the default null sentinel. The generic dimension-table `BYTES` primary-key lookup defect, which also affects BYTES-backed UUID keys, is tracked separately in #19228. It does not require keeping this obsolete implementation branch open. This branch should not be rebased or merged: it duplicates code already merged through the split PRs and retains stale or deliberately rejected scope and contracts. The design discussion and benchmark results here remain useful historical context, but the merged split stack contains the final correctness fixes, evolved SV/MV support, integration coverage, and documentation. Thanks to everyone who reviewed and helped split this work into mergeable layers. The umbrella tracker #16619 can now be closed as completed. <!-- uuid-v1-pr-18140-closure-summary --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
