Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/16927
I could add the 2.6.5 binaries if you want, though the 2.6.4 ones should be
compatible.
I think I just lifted the 2.6.x artifacts out of an HDP build; its only the
2.7.x ones where I built things locally, and I don't think things were
compiling the last time I tried to do a build on that VM.
Regarding your comment about the new IOE, yes, that is coming from the
HADOOP-10775 changes.
Before: meaningless stack trace on startup irrespective of actual use,
different meaningless stack trace on failures.
After: startup failures tucked away until later, actual uses intended to
provide enough information that somebody competent can track down why things
are broken.
Doesn't look like the message you've seen meets that criteria of being
informative.
Following the stack trace, it seems to get triggered in an {{exec(String[]
args)}} if one of the arguments is null
[Shell.java#L1192](https://github.com/apache/hadoop/blob/branch-2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java#L1192).
This implies that something has gone wrong in the invocation, but why I
don't know, and the error message isn't doing anything to help. At the very
least it should indicate which arg is invalid, and join the string with some
form of quotations so that you can see what's up in more detail.
Now, what was the problem? Just that winutil.exe was missing? Because
that's something we should be detecting and reporting with something including
a link to [a wiki page](https://wiki.apache.org/hadoop/WindowsProblems)
Improvements to the hadoop logging/diagnostics welcome here. I'd also
personally like the ability to tell have the file:// client to try to set file
permissions, but there you go.,
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
