Anish Mahto created SPARK-60097:
-----------------------------------
Summary: SCD1 Ignore-null support; define version map construction
Key: SPARK-60097
URL: https://issues.apache.org/jira/browse/SPARK-60097
Project: Spark
Issue Type: Sub-task
Components: Declarative Pipelines
Affects Versions: 4.4.0
Reporter: Anish Mahto
Concretely define the contract for the _contents_ of the version map; the
format must the keys be presented in, and what the stored sequence represents
when null or not null.
Then implement the logic to construct a version map column expression as per
the contract, given an upsert-representing row. Delete-representing rows do not
require any equivalent operation, as they will always hold null version maps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
