zifeif2 commented on code in PR #53287:
URL: https://github.com/apache/spark/pull/53287#discussion_r2604312622
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/utils/SchemaUtil.scala:
##########
@@ -62,13 +72,7 @@ object SchemaUtil {
.add("value", valueSchema)
.add("partition_id", IntegerType)
} else if (sourceOptions.internalOnlyReadAllColumnFamilies) {
- new StructType()
- // todo [SPARK-54443]: change keySchema to a more specific type after
we
- // can extract partition key from keySchema
- .add("partition_key", keySchema)
- .add("key_bytes", BinaryType)
- .add("value_bytes", BinaryType)
- .add("column_family_name", StringType)
+ getScanAllColumnFamiliesSchema(keySchema)
Review Comment:
I can change it back! I split it originally because the following PRs needed
them. But since the PartitionKey extraction logic is landed, we don't need to
extract this function anymore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
