Jens Rabe created SPARK-23395:

             Summary: Add an option to return an empty DataFrame from an RDD 
generated by a Hadoop file
                 Key: SPARK-23395
             Project: Spark
          Issue Type: Improvement
          Components: Input/Output
    Affects Versions: 2.2.1, 2.2.0
            Reporter: Jens Rabe

When using file-based data from custom formats, Spark's ability to use Hadoop's 
FileInputFormats is very handy. However, when the path they are pointed at 
contains no usable data, they throw an IOException saying "No input paths 
specified in job".

It would be a nice feature if the DataFrame API somehow could capture this and 
return an empty DataFrame instead of failing the job.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to