Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17150
Hi all, I did not open an JIRA because I am not very sure if it is an issue
and it sounds maybe my environment problem.
However, I believe it is worth leaving a note. The build with Maven is fine
(`./build/mvn clean -DskipTests package` as described in
https://spark.apache.org/docs/latest/building-spark.html#buildmvn) but I am
seeing the error with SBT after this commit. The build itself succeeds but it
fails to run `./bin/spark-shell`. I tried the command lines as below (as
described in http://spark.apache.org/developer-tools.html):
```bash
./build/sbt clean package
```
```bash
./bin/spark-shell
```
```
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/logging/LogFactory
at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:173)
at
org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:340)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:157)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:123)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.logging.LogFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 5 more
```
I could reproduce this on ...
- CentOS Linux release 7.3.1611 (Core) / Java(TM) SE Runtime Environment
(build 1.8.0_101-b13)
- macOS 10.12.3 (16D32) / Java(TM) SE Runtime Environment (build
1.8.0_45-b14)
I could not reproduce this on ...
- Ubuntu 14.04 LTS / Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
I checked the build with Maven is fine on ...
- CentOS Linux release 7.3.1611 (Core) / Java(TM) SE Runtime Environment
(build 1.8.0_101-b13)
- macOS 10.12.3 (16D32) / Java(TM) SE Runtime Environment (build
1.8.0_45-b14)
I checked the build before/after the commit on ...
- macOS 10.12.3 (16D32) / Java(TM) SE Runtime Environment (build
1.8.0_45-b14)
It looks this one is introduced here.
Could anyone double check this and see if I am mistaken? I just wonder if
the same commands are consistently being failed on other environments (rather
than the exact failure cause) ...
- CentOS Linux release 7.3.1611 (Core) / Java(TM) SE Runtime Environment
(build 1.8.0_101-b13)
- macOS 10.12.3 (16D32) / Java(TM) SE Runtime Environment (build
1.8.0_45-b14)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
