GitHub user janewangfb opened a pull request:
https://github.com/apache/spark/pull/18975
[SPARK-4131] Support "Writing data into the filesystem from queries"
## What changes were proposed in this pull request?
This PR implements the sql feature:
INSERT OVERWRITE [LOCAL] DIRECTORY directory1
[ROW FORMAT row_format] [STORED AS file_format]
SELECT ... FROM ...
## How was this patch tested?
Added new unittests and also pulled the code to fb-spark so that we could
test writing to hdfs directory.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/janewangfb/spark port_local_directory
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18975.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18975
----
commit 6ca7771a39fe7c6ca5f82acc7d76dff00238de68
Author: Jane Wang <[email protected]>
Date: 2017-08-17T01:32:23Z
add insert overwrite local directory
commit a975536ea3fed0674410ebc65538e134992779b1
Author: Jane Wang <[email protected]>
Date: 2017-08-17T03:59:25Z
Add Unittests
commit a15bf4e0c12a37127788528f1e69b0462a0e287b
Author: Jane Wang <[email protected]>
Date: 2017-08-17T05:23:34Z
fix local path
commit 9f596fda9dccdb0d5541340e8d9f2bf83573066a
Author: Jane Wang <[email protected]>
Date: 2017-08-17T18:25:04Z
Merge branch 'master' into port_local_directory
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
