JingsongLi commented on PR #9147: URL: https://github.com/apache/paimon/pull/9147#issuecomment-5240422088
Could we clarify whether this is intended to be a general public VARIANT API or an internal fast path? I could not find an equivalent VARIANT mutation function in Databricks; its documented VARIANT functions are primarily for extraction, inspection, and explosion. JSON APIs, however, have fairly established mutation semantics: - MySQL `JSON_REPLACE` updates existing paths and treats a missing path as a no-op, while `JSON_SET` also creates missing paths. - BigQuery `JSON_SET(..., create_if_missing => false)` provides similar existing-path-only behavior. - Oracle `JSON_TRANSFORM` is an operation DSL (`SET`, `REPLACE`, `REMOVE`, etc.), rather than a callback-based transform. MySQL can optimize `JSON_SET` and `JSON_REPLACE` into a partial in-place binary JSON update under suitable conditions, which is conceptually close to this implementation. Importantly, that is an implementation optimization behind stable set/replace semantics. Would an API such as `variant_replace`, or `variant_set(..., create_if_missing=False)`, be clearer than `variant_transform`? The current binary patch can remain the fast path for existing fixed-width values. If a path is missing, replace semantics could leave the value unchanged (optionally with a strict mode), while set semantics could fall back to reconstruction because adding a field may require changing metadata and offset tables. This would keep the public contract independent of the current optimization and leave room for supporting more types later. References: [Databricks VARIANT functions](https://docs.databricks.com/aws/en/semi-structured/variant-json-diff), [MySQL JSON modification functions](https://dev.mysql.com/doc/refman/9.1/en/json-modification-functions.html), [MySQL partial JSON updates](https://dev.mysql.com/doc/refman/8.0/en/json.html#json-partial-updates), [BigQuery JSON_SET](https://cloud.google.com/bigquery/docs/reference/standard-sql/json_functions#json_set), [Oracle JSON_TRANSFORM](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/json_transform.html). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
