[
https://issues.apache.org/jira/browse/SPARK-40100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18012076#comment-18012076
]
Nicolae Marasoiu commented on SPARK-40100:
------------------------------------------
Hi, on the scope of this task, I have a question,
So creating the foundational DataType infrastructure for Int128. Based on
Spark's architecture, this would involve:
# {*}DataType Contract Implementation{*}: Creating a new DataType that follows
Spark's DataType contract, which requires implementing methods for JSON
representations, default size calculations, and string representations [Data
Types - Spark 3.5.3
Documentation|https://spark.apache.org/docs/3.5.3/sql-ref-datatypes.html]
# {*}Integration Points{*}: The DataType class would need to integrate with:
** SQL parser and catalyst optimizer
** Code generation framework
** Serialization/deserialization systems
** UDF support systems
What about math? Is this part of this story?
I can see an incomplete PR opened, not sure if intention was to ask early
feedback, does that PR look like something to build on? Thanks
> Add DataType class for Int128 type
> ----------------------------------
>
> Key: SPARK-40100
> URL: https://issues.apache.org/jira/browse/SPARK-40100
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Jiaan Geng
> Priority: Major
>
> Extend Catalyst's type system by a new type:
> Int128Type represents the Int128
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
