[
https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723679#comment-16723679
]
ASF GitHub Bot commented on SPARK-26081:
----------------------------------------
HyukjinKwon opened a new pull request #23341: [SPARK-26081][SQL] Use foreach
instead of misuse of map (for Unit)
URL: https://github.com/apache/spark/pull/23341
## What changes were proposed in this pull request?
This PR proposes to use foreach instead of misuse of map (for Unit). This
could cause some weird errors potentially and it's not a good practice anyway.
See also SPARK-16694
## How was this patch tested?
N/A
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Do not write empty files by text datasources
> --------------------------------------------
>
> Key: SPARK-26081
> URL: https://issues.apache.org/jira/browse/SPARK-26081
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Minor
> Labels: release-notes
> Fix For: 3.0.0
>
>
> Text based datasources like CSV, JSON and Text produces empty files for empty
> partitions. This introduces additional overhead while opening and reading
> such files back. In current implementation of OutputWriter, the output stream
> are created eagerly even no records are written to the stream. So, creation
> can be postponed up to the first write.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
