[ https://issues.apache.org/jira/browse/SPARK-32544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171880#comment-17171880 ]
Hyukjin Kwon commented on SPARK-32544: -------------------------------------- This will be deal with together with DataSource V2. > Bucketing and Partitioning information are not passed on to non FileFormat > datasource writes > -------------------------------------------------------------------------------------------- > > Key: SPARK-32544 > URL: https://issues.apache.org/jira/browse/SPARK-32544 > Project: Spark > Issue Type: Improvement > Components: Input/Output > Affects Versions: 3.0.0 > Reporter: Rahij Ramsharan > Priority: Major > > When writing to a FileFormat datasource, bucket spec and partition columns > are passed into InsertIntoHadoopFsRelationCommand: > [https://github.com/apache/spark/blob/v3.0.0/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala#L474-L475]. > > However, from what I can tell, the RelationProvider API does not have a way > to pass in these: > [https://github.com/apache/spark/blob/v3.0.0/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala#L511-L513]. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org