[ https://issues.apache.org/jira/browse/SPARK-26160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699255#comment-16699255 ]
Hyukjin Kwon commented on SPARK-26160: -------------------------------------- Let's ask it / start the discussion from mailing list. > Make assertNotBucketed call in DataFrameWriter::save optional > ------------------------------------------------------------- > > Key: SPARK-26160 > URL: https://issues.apache.org/jira/browse/SPARK-26160 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.4.0 > Reporter: Ximo Guanter > Priority: Minor > > Currently, the `.save` method in DataFrameWriter will fail if the dataframe > is bucketed and/or sorted. This makes sense, since there is no way of storing > metadata in the current file-based data sources to know whether a file was > bucketed or not. > I have a use case where I would like to implement a new, file-based data > source which could keep track of that kind of metadata, so I would like to be > able to `.save` bucketed dataframes. > > Would a patch to extend the datasource api with an indicator of whether that > source is able to serialize bucketed dataframes be a welcome addition? I'm > happy to work on it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org