dbtsai commented on issue #26929: [SPARK-30289][SQL] DSv2's partitioning should not accept nested columns URL: https://github.com/apache/spark/pull/26929#issuecomment-573950500 I was on paternity leave, and sorry for the late reply. @rdblue the intent of this was to disable partition by nested columns because I saw some inconsistencies in the v2 codebase when I tried to create a new v2 filter api, and I thought partition by nested columns is not supported at all. For example, in `org.apache.spark.sql.catalyst.catalog.CatalogTable`, we have `partitionColumnNames: Seq[String]` as partition columns instead of using `NamedReference`, how do we properly support using nested column as partitioning? Are we paring the string that contains `.` as a nested column? Can you give me an example? @cloud-fan do you mean fix `InMemoryTable` so it supports it properly?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
