GitHub user sujith71955 opened a pull request:
https://github.com/apache/spark/pull/19823
[SPARK-22601][SQL] Data load is getting displayed successful on providing
non existing hdfs file path
## What changes were proposed in this pull request?
When user tries to load data with a non existing hdfs file path system is
not validating it and the load command operation is getting successful.
This is misleading to the user. already there is a validation in the
scenario of none existing local file path. This PR has added validation in the
scenario of nonexisting hdfs file path
## How was this patch tested?
UT has been added for verifying the issue, also snapshots has been added
after the verification in a spark yarn cluster
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sujith71955/spark master_LoadComand_Issue
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19823.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19823
----
commit 5b247a8b3065a6f7f3a78380af898d9c98eeec8b
Author: sujith71955 <[email protected]>
Date: 2017-11-24T13:31:22Z
[SPARK-22601][SQL] Data load is getting displayed successful on providing
non existing hdfs file path
## What changes were proposed in this pull request?
When user tries to load data with a non existing hdfs file path system is
not validating it and the load command operation is getting successful.
This is misleading to the user. already there is a validation in the
scenario of local file path. This PR has added validation in the scenario of
hdfs file path
## How was this patch tested?
existing tests are present to verify the impact and manually the scenario
is been verified in hdfs cluster
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
