[
https://issues.apache.org/jira/browse/SPARK-51848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaonan Yang updated SPARK-51848:
---------------------------------
Description:
{code:java}
<ROW>
<scalarCol>$i</scalarCol>
<arrayCol>
<nestedCol>$i</nestedCol>
</arrayCol>
<arrayCol>
<nestedCol>${i + 1}</nestedCol>
</arrayCol>
<structCol>
<a>$i</a>
<b>
<nestedCol2>$i</nestedCol2>
</b>
</structCol>
<nestedVariantCol>
<a>$i</a>
<b>
<nestedCol3>$i</nestedCol3>
</b>
</nestedVariantCol>
</ROW>{code}
When parsing the XML record above with a schema of
{code:java}
scalarCol variant, arrayCol array<variant>, structCol struct<a int, b variant>,
nestedVariantCol variant{code}
The variant fields in nested structures like array and struct can’t be parsed
correctly.
> Variant field parsing fails in nested structures (arrays/structs)
> -----------------------------------------------------------------
>
> Key: SPARK-51848
> URL: https://issues.apache.org/jira/browse/SPARK-51848
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Xiaonan Yang
> Assignee: Xiaonan Yang
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> {code:java}
> <ROW>
> <scalarCol>$i</scalarCol>
> <arrayCol>
> <nestedCol>$i</nestedCol>
> </arrayCol>
> <arrayCol>
> <nestedCol>${i + 1}</nestedCol>
> </arrayCol>
> <structCol>
> <a>$i</a>
> <b>
> <nestedCol2>$i</nestedCol2>
> </b>
> </structCol>
> <nestedVariantCol>
> <a>$i</a>
> <b>
> <nestedCol3>$i</nestedCol3>
> </b>
> </nestedVariantCol>
> </ROW>{code}
> When parsing the XML record above with a schema of
> {code:java}
> scalarCol variant, arrayCol array<variant>, structCol struct<a int, b
> variant>, nestedVariantCol variant{code}
> The variant fields in nested structures like array and struct can’t be parsed
> correctly.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
