[
https://issues.apache.org/jira/browse/SPARK-60713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60713:
-----------------------------------
Labels: pull-request-available (was: )
> Report the actual size limit in VARIANT_CONSTRUCTOR_SIZE_LIMIT
> --------------------------------------------------------------
>
> Key: SPARK-60713
> URL: https://issues.apache.org/jira/browse/SPARK-60713
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Stefan Savic
> Priority: Major
> Labels: pull-request-available
>
> VARIANT_CONSTRUCTOR_SIZE_LIMIT hardcodes 16 MiB in its error message:
> Cannot construct a Variant larger than 16 MiB. The maximum allowed size of a
> Variant value is 16 MiB.
> SPARK-52181 increased the normal Variant size limit from 16 MiB to 128 MiB:
> https://github.com/apache/spark/pull/50913
> In current OSS Spark, VariantUtil.SIZE_LIMIT is 128 MiB during normal
> execution and 16 MiB during tests. The Variant constructor checks the value
> and metadata arrays against this constant, but the error message does not use
> it. When the Variant value or metadata exceeds 128 MiB, the error incorrectly
> says the limit is 16 MiB. The lower test limit masks this mismatch.
> The message should report the actual enforced limit. Replace the hardcoded
> size with a sizeLimit parameter populated from VariantUtil.SIZE_LIMIT and
> update the affected error factories and test expectations. This should
> preserve the existing enforced limits and error class.
> VARIANT_SIZE_LIMIT already accepts a sizeLimit parameter;
> this issue concerns VARIANT_CONSTRUCTOR_SIZE_LIMIT.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
