[
https://issues.apache.org/jira/browse/SPARK-60994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-60994:
------------------------------------
Assignee: Hyukjin Kwon
> Type a decimal literal of the Python client as PySpark does
> -----------------------------------------------------------
>
> Key: SPARK-60994
> URL: https://issues.apache.org/jira/browse/SPARK-60994
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, {{lit(Decimal(...))}} sends a decimal literal of
> {{DecimalType()}}, of precision 10 and scale 0, which the server widens to
> the precision and scale of the value: {{lit(Decimal("1.5"))}} is a
> {{decimal(10,1)}} and {{lit(Decimal(0))}} a {{decimal(10,0)}}. The
> Rust-backed Python client sends the precision and scale of the value, so
> these literals are a {{decimal(2,1)}} and a {{decimal(1,0)}}, and the
> expressions over them have other types: {{lit(Decimal("1.5")) *
> lit(Decimal("2.25"))}} is a {{decimal(6,3)}} rather than a {{decimal(21,3)}},
> and so may overflow where PySpark's does not. The seconds of
> {{make_interval}}, {{make_dt_interval}} and {{try_make_interval}}, which
> default to {{lit(Decimal(0))}}, differ the same way. Type a decimal literal
> as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
