LuciferYang opened a new issue, #1353: URL: https://github.com/apache/paimon-rust/issues/1353
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar. ### Motivation `nested_update` and `nested_partial_update` with `fields.<col>.nested-key` find the stored row for each incoming nested row by scanning every stored row and comparing the key columns with Arrow `==`. Merging n nested rows into one cell is O(n²). Java's `FieldNestedUpdateAgg` keys a `Map<BinaryRow, InternalRow>` by the BinaryRow of the key columns. Writing one row whose ARRAY holds 10,000 distinct nested keys takes 4.8 s on main (release build, in-memory FileIO; the write buffer runs the aggregation merge). 2,000 keys take 190 ms. ### Solution Index the stored rows by the serialized BinaryRow of the key columns, like Java's map, and keep the scan for key types that cannot be serialized (ARRAY, MAP, MULTISET, ROW, VECTOR, BLOB) or key values that fail to serialize. ### Willingness to contribute - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
