Github user LiehuoChen commented on the issue:
https://github.com/apache/spark/pull/21588
Hi HyukjinKwon,
Thanks for all the works to try to make the Jenkin test pass.
I patched this PR to spark 2.4, and anything works fine but failed in
org.apache.spark.deploy.yarn.YarnClusterSuite for following four unit tests:
1). run Spark in yarn-cluster mode
2). run Spark in yarn-cluster mode with different configurations, ensuring
redaction
3). run Spark in yarn-client mode
4). run Spark in yarn-client mode with different configurations, ensuring
redaction
1), 2), failed everytime with really few useful error Msg, like:
`FAILED did not equal FINISHED Exception in thread "main"
org.apache.spark.SparkException: Application application_1542090777201_0002
finished with failed status
[info] at org.apache.spark.deploy.yarn.Client.run(Client.scala:1149)
......
[info] at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
(BaseYarnClusterSuite.scala:201)`
2).4). succeed most of time, but sometimes failed on
` Exception in thread "main" java.io.IOException: Server returned HTTP
response code: 500 for URL:
http://user-c02wq03ghtdg.corp.uber.com:61313/node/containerlogs/container_1541809642345_0002_01_000002/lhc/stdout?start=-4096`
and `Fail to invoke HBaseConfiguration
[info] java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.HBaseConfiguration`
Any you ever see the similars errors before? do you did any other fixes
besides this PR to make the all test pass.
Thanks for your time.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
