Jira: https://issues.apache.org/jira/browse/OOZIE-3112
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/267/
###################################################################################
########################## LAST 100 LINES OF THE CONSOLE
###########################
[...truncated 1.69 MB...]
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[WARN] There are [1] new bugs found in total that would be nice to have fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2245 bytes
[TRACE] Full summary file size is 1378 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3112
Cleaning local git workspace
----------------------------
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch does adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
0 There are [1] new bugs found in total that would be nice to have fixed.
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [client].
0 There are [1] new bugs found in [core] that would be nice to have fixed.
You can find the FindBugs diff here: core/findbugs-new.html
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2075
Tests failed: 5
Tests errors: 0
The patch failed the following testcases:
testYarnClientFilesAndArchivesArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
testDriverClassPathArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
testYarnClientExecutorAndDriverExtraClasspathsArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
testLocalClientArgsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
testQuotedDriverAndExecutorExtraJavaOptionsParsing(org.apache.oozie.action.hadoop.TestSparkArgsExtractor)
Tests failing with errors:
+1 DISTRO
+1 distro tarball builds with the patch
----------------------------
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/267/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0100 3706k 100 3706k 0 0 3515k 0 0:00:01 0:00:01 --:--:-- 6863k
Adding comment to JIRA
Comment added.
test-patch exit code: 1
Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3112
Archiving artifacts
[Fast Archiver] Compressed 1.87 MB of artifacts by 78.7% relative to #257
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###################################################################################
############################## FAILED TESTS (if any)
##############################
5 tests failed.
FAILED:
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testYarnClientFilesAndArchivesArgsParsing
Error Message:
Spark args mismatch expected:<[--master, yarn, --deploy-mode, client, --name,
Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--files, spark-log4j.properties,hive-site.xml,aaa,ccc, --archives, bbb,ddd,
--conf, spark.yarn.jar=null, --verbose, /lib/test.jar, arg0, arg1]> but
was:<[--master, yarn, --deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml,aaa,ccc, --archives, bbb,ddd, --conf,
spark.yarn.jar=null, --verbose, /lib/test.jar, arg0, arg1]>
Stack Trace:
java.lang.AssertionError: Spark args mismatch expected:<[--master, yarn,
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--files, spark-log4j.properties,hive-site.xml,aaa,ccc, --archives, bbb,ddd,
--conf, spark.yarn.jar=null, --verbose, /lib/test.jar, arg0, arg1]> but
was:<[--master, yarn, --deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml,aaa,ccc, --archives, bbb,ddd, --conf,
spark.yarn.jar=null, --verbose, /lib/test.jar, arg0, arg1]>
at
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testYarnClientFilesAndArchivesArgsParsing(TestSparkArgsExtractor.java:160)
FAILED:
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testDriverClassPathArgsParsing
Error Message:
Spark args mismatch expected:<[--master, yarn, --deploy-mode, client, --name,
Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=aaa:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--files, spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null,
--verbose, /lib/test.jar, arg0, arg1]> but was:<[--master, yarn, --deploy-mode,
client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=aaa:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]>
Stack Trace:
java.lang.AssertionError: Spark args mismatch expected:<[--master, yarn,
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=aaa:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--files, spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null,
--verbose, /lib/test.jar, arg0, arg1]> but was:<[--master, yarn, --deploy-mode,
client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=aaa:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.executor.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]>
at
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testDriverClassPathArgsParsing(TestSparkArgsExtractor.java:198)
FAILED:
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testYarnClientExecutorAndDriverExtraClasspathsArgsParsing
Error Message:
Spark args mismatch expected:<[--master, yarn, --deploy-mode, client, --name,
Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy,
--driver-memory, 1042M, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=aaa:$PWD/*, --conf,
spark.driver.extraClassPath=ccc:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--files, spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null,
--verbose, /lib/test.jar, arg0, arg1]> but was:<[--master, yarn, --deploy-mode,
client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --driver-memory, 1042M, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=aaa:$PWD/*, --conf,
spark.driver.extraClassPath=ccc:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]>
Stack Trace:
java.lang.AssertionError: Spark args mismatch expected:<[--master, yarn,
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --driver-memory, 1042M, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=aaa:$PWD/*, --conf,
spark.driver.extraClassPath=ccc:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--files, spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null,
--verbose, /lib/test.jar, arg0, arg1]> but was:<[--master, yarn, --deploy-mode,
client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --driver-memory, 1042M, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=aaa:$PWD/*, --conf,
spark.driver.extraClassPath=ccc:$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]>
at
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testYarnClientExecutorAndDriverExtraClasspathsArgsParsing(TestSparkArgsExtractor.java:116)
FAILED:
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testLocalClientArgsParsing
Error Message:
Spark args mismatch expected:<[--master, local[*], --deploy-mode, client,
--name, Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy,
--driver-memory, 1042M, --conf, spark.executor.extraClassPath=aaa, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
spark.driver.extraClassPath=ccc, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--verbose, /lib/test.jar, arg0, arg1]> but was:<[--master, local[*],
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --driver-memory, 1042M, --conf,
spark.executor.extraClassPath=aaa, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
spark.driver.extraClassPath=ccc, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --verbose,
/lib/test.jar, arg0, arg1]>
Stack Trace:
java.lang.AssertionError: Spark args mismatch expected:<[--master, local[*],
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --driver-memory, 1042M, --conf,
spark.executor.extraClassPath=aaa, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
spark.driver.extraClassPath=ccc, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--verbose, /lib/test.jar, arg0, arg1]> but was:<[--master, local[*],
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --driver-memory, 1042M, --conf,
spark.executor.extraClassPath=aaa, --conf,
user.property.after.spark.executor.extraClassPath=bbb, --conf,
spark.driver.extraClassPath=ccc, --conf,
user.property.after.spark.driver.extraClassPath=ddd, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --conf,
spark.driver.extraJavaOptions=-Dlog4j.configuration=spark-log4j.properties,
--properties-file, spark-defaults-oozie-generated.properties, --verbose,
/lib/test.jar, arg0, arg1]>
at
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testLocalClientArgsParsing(TestSparkArgsExtractor.java:72)
FAILED:
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testQuotedDriverAndExecutorExtraJavaOptionsParsing
Error Message:
Spark args mismatch expected:<[--master, yarn, --deploy-mode, client, --name,
Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.driver.extraJavaOptions=-Xmn2703m -XX:SurvivorRatio=2
-XX:ParallelGCThreads=20 -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]> but was:<[--master, yarn, --deploy-mode, client,
--name, Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy,
--conf, spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.driver.extraJavaOptions=-Xmn2703m -XX:SurvivorRatio=2
-XX:ParallelGCThreads=20 -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --properties-file,
spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]>
Stack Trace:
java.lang.AssertionError: Spark args mismatch expected:<[--master, yarn,
--deploy-mode, client, --name, Spark Copy File, --class,
org.apache.oozie.example.SparkFileCopy, --conf,
spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.driver.extraJavaOptions=-Xmn2703m -XX:SurvivorRatio=2
-XX:ParallelGCThreads=20 -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]> but was:<[--master, yarn, --deploy-mode, client,
--name, Spark Copy File, --class, org.apache.oozie.example.SparkFileCopy,
--conf, spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.driver.extraJavaOptions=-Xmn2703m -XX:SurvivorRatio=2
-XX:ParallelGCThreads=20 -Dlog4j.configuration=spark-log4j.properties, --conf,
spark.executor.extraClassPath=$PWD/*, --conf,
spark.driver.extraClassPath=$PWD/*, --conf,
spark.yarn.security.tokens.hadoopfs.enabled=false, --conf,
spark.yarn.security.tokens.hive.enabled=false, --conf,
spark.yarn.security.tokens.hbase.enabled=false, --conf,
spark.yarn.security.credentials.hadoopfs.enabled=false, --conf,
spark.yarn.security.credentials.hive.enabled=false, --conf,
spark.yarn.security.credentials.hbase.enabled=false, --properties-file,
spark-defaults-oozie-generated.properties, --files,
spark-log4j.properties,hive-site.xml, --conf, spark.yarn.jar=null, --verbose,
/lib/test.jar, arg0, arg1]>
at
org.apache.oozie.action.hadoop.TestSparkArgsExtractor.testQuotedDriverAndExecutorExtraJavaOptionsParsing(TestSparkArgsExtractor.java:237)
