chenhao-db opened a new pull request, #45708:
URL: https://github.com/apache/spark/pull/45708
### What changes were proposed in this pull request?
This PR adds a new `VariantGet` expression that extracts variant value and
casts it to a concrete type. It is exposed as two SQL expressions,
`variant_get` and `try_variant_get`. If the extracted path doesn't exist in the
source variant value, they should both return null. The difference is at the
cast step: when the cast fails,`variant_get` should throw an exception, and
`try_variant_get` should return null.
The cast behavior is NOT affected by the `spark.sql.ansi.enabled` flag:
`variant_get` always has the ANSI cast semantics, while `try_variant_get`
always has the TRY cast semantics. An example is that casting a variant long to
an int never silently overflows and produces the wrapped int value, while
casting a long to an int may silently overflow in LEGACY mode.
The current path extraction only supports array index access and
case-sensitive object key access.
Usage examples:
```
> SELECT variant_get(parse_json('{"a": 1}'), '$.a', 'int');
1
> SELECT variant_get(parse_json('{"a": 1}'), '$.b', 'int');
NULL
> SELECT variant_get(parse_json('[1, "2"]'), '$[1]', 'string');
2
> SELECT variant_get(parse_json('[1, "2"]'), '$[2]', 'string');
NULL
> SELECT variant_get(parse_json('[1, "hello"]'), '$[1]'); -- when the target
type is not specified, it returns variant by default (i.e., only extracts a
sub-variant without cast)
"hello"
> SELECT try_variant_get(parse_json('[1, "hello"]'), '$[1]', 'int'); --
"hello" cannot be cast into int
NULL
```
### How was this patch tested?
Unit tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
