[
https://issues.apache.org/jira/browse/SPARK-22104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180988#comment-16180988
]
Anbu Cheeralan commented on SPARK-22104:
----------------------------------------
[~hyukjin.kwon] Yes. I am writing files using multiple processes in the same
directory in append mode. I would like to identify the files in case of
troubleshooting.
Hive supports this option via "hive.output.file.extension" property.
> Add new option to dataframe -> parquet ==> custom extension to file name
> ------------------------------------------------------------------------
>
> Key: SPARK-22104
> URL: https://issues.apache.org/jira/browse/SPARK-22104
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Affects Versions: 2.2.0
> Reporter: Anbu Cheeralan
> Priority: Minor
>
> While writing to dataframe to parquet, I would like to add a new option to
> denote the custom extension name (important_info.parquet) instead of a
> simple .parquet.
> I can create a pull request for this solution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
