GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/11023
[SPARK-13137][SQL] NullPoingException in schema inference for CSV when the
first line is empty
https://issues.apache.org/jira/browse/SPARK-13137
This PR adds a filter in schema inference so that it does not emit
NullPointException.
Also, I removed `MAX_COMMENT_LINES_IN_HEADER `but instead used a monad
chaining with `filter()` and `first()`.
Lastly, I simply added a newline rather than adding a new file for this so
that this is covered with the original tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/spark SPARK-13137
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11023.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11023
----
commit 5214e8a281e77da5d4451a29a553352959af84ab
Author: hyukjinkwon <[email protected]>
Date: 2016-02-02T08:39:00Z
NullPoingException in schema inference for CSV when the first line is empty
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
