[
https://issues.apache.org/jira/browse/SPARK-56076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-56076:
-----------------------------------
Labels: pull-request-available (was: )
> Types Framework - Phase 1b - Generic Row Access
> -----------------------------------------------
>
> Key: SPARK-56076
> URL: https://issues.apache.org/jira/browse/SPARK-56076
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: David Milicevic
> Priority: Major
> Labels: pull-request-available
>
> *Description:*
> {*}{*}Add generic row read/write support so that framework types can be
> stored in and retrieved from UnsafeRow without per-type edits to row
> interfaces.
> *What* *this* *includes:*
> * {*}{*}Two new TypeOps methods: getValue(input: SpecializedGetters,
> ordinal: Int): Any and unsafeWrite(writer: UnsafeWriter, ordinal: Int, value:
> Any): Unit
> * PhysicalFrameworkDelegatedType(ops: TypeOps) — a general-purpose delegated
> physical type. Dispatchers match on it and call ops.getValue() /
> ops.unsafeWrite() without knowing the storage format (struct, binary, etc.)
> * TypeOps.getOrNull(dt) helper for clean dispatch in Java files
> (ColumnarRow, ColumnarBatchRow)
> * Integration in ~10 files: TypeOps.scala, TimeTypeOps.scala,
> PhysicalDataType.scala, SpecializedGettersReader.java,
> InterpretedUnsafeProjection.scala, ColumnarRow.java, ColumnarBatchRow.java,
> InternalRow.scala, literals.scala
> * TimeType reference implementation (Long-backed, simple case)
> *Note:*
> {*}{*}unsafeRead was dropped — it's redundant because UnsafeRow implements
> SpecializedGetters, so getValue(unsafeRow, ordinal) already covers the read
> path. PhysicalFrameworkDelegatedType replaces the earlier PhysicalOpaqueType
> concept from the design doc — the name conveys both framework ownership and
> delegated behavior. It takes no structFields parameter; the TypeOps handles
> storage format internally, making it work for struct-backed types,
> binary-backed types, or anything else.
> Struct-writing utility methods in InterpretedUnsafeProjection are widened to
> private[catalyst] so that future TypeOps implementations can reuse the same
> struct-writing machinery.
> *Design doc:*
> Linked in the parent work item.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]