shaneknapp edited a comment on issue #23514: [SPARK-24902][K8s] Add PV integration tests URL: https://github.com/apache/spark/pull/23514#issuecomment-459902910 FINALLY. ;) from `kubectl --namespace=3f2e4014072044fa8736c0284349fdf8 logs spark-test-app-54feffd6d5194096b66e4198cf26cf31`: ``` ++ id -u + myuid=185 ++ id -g + mygid=0 + set +e ++ getent passwd 185 + uidentry= + set -e + '[' -z '' ']' + '[' -w /etc/passwd ']' + echo '185:x:185:0:anonymous uid:/opt/spark:/bin/false' + SPARK_K8S_CMD=driver + case "$SPARK_K8S_CMD" in + shift 1 + SPARK_CLASSPATH=':/opt/spark/jars/*' + env + grep SPARK_JAVA_OPT_ + sort -t_ -k4 -n + sed 's/[^=]*=\(.*\)/\1/g' + readarray -t SPARK_EXECUTOR_JAVA_OPTS + '[' -n '' ']' + '[' -n '' ']' + PYSPARK_ARGS= + '[' -n '' ']' + R_ARGS= + '[' -n '' ']' + '[' '' == 2 ']' + '[' '' == 3 ']' + '[' -z ']' + case "$SPARK_K8S_CMD" in + CMD=("$SPARK_HOME/bin/spark-submit" --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" --deploy-mode client "$@") + exec /sbin/tini -s -- /opt/spark/bin/spark-submit --conf spark.driver.bindAddress=172.17.0.4 --deploy-mode client --properties-file /opt/spark/conf/spark.properties --class org.apache.spark.examples.DFSReadWriteTest spark-internal /opt/spark/pv-tests/tmp1597241055045427577.txt /opt/spark/pv-tests 2019-02-01 23:21:54 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Given path (/opt/spark/pv-tests/tmp1597241055045427577.txt) does not exist DFS Read-Write Test Usage: localFile dfsDir localFile - (string) local file to use in test dfsDir - (string) DFS directory for read/write tests 2019-02-01 23:21:54 INFO ShutdownHookManager:57 - Shutdown hook called 2019-02-01 23:21:54 INFO ShutdownHookManager:57 - Deleting directory /tmp/spark-1f87d035-3901-4fe4-9813-83159dd35473 ``` doesn't seem to be much there, but lemme know if you need anything else.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
