shaneknapp commented on issue #26330: [WIP][SPARK-29672][PYSPARK] DO NOT MERGE -- remove py27 support for jenkins tests URL: https://github.com/apache/spark/pull/26330#issuecomment-551248929 > talk about unintended consequences: this PR is breaking the pretty-printed output on jenkins: > > ``` > java.lang.Thread.run(Thread.java:748)\n'b'13:42:45.815 WARN org.apache.spark.sql.execution.command.DropTableCommand: org.apache.spark.sql.AnalysisException: Table or view not found: view9; line 1 pos 14;\n'b"'SubqueryAlias `default`.`view10`\n"b'+- View (`default`.`view10`, [id#431310L,id1#431311L])\n'b" +- 'Project [*]\n"b" +- 'UnresolvedRelation [view9]\n"b'\n'b'org.apache.spark.sql.AnalysisException: Table or view not found: view9; line 1 pos 14;\n'b"'SubqueryAlias `default`.`view10`\n"b'+- View (`default`.`view10`, [id#431310L,id1#431311L])\n'b" +- 'Project [*]\n"b" +- 'UnresolvedRelation [view9]\n"b'\n'b'\tat org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)\n'b'\tat org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$1(CheckAnalysis.scala:95)\n'b'\tat org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$1$adapted(CheckAnalysis.scala:90)\n'b'\tat org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:154)\n'b'\tat > ``` > > wonder if this is from `from __future__ import print_function`... guess i should pull them out as we won't need them anymore and see if that fixes it. ok, it looks like this is caused by the chain of `dev/run-tests.py` running with python3 calling `sbt` (for various build steps) through the `run_cmd()` function in `dev/sparktestsupport/shellutils.py` library. none of the build steps have the `return_output=True` flag for `run_cmd` set, so the output is just dumped out by the subprocess itself as a python bytestring. since we're not decoding the output, it just dumping those bytestrings to the build log. removing `from __future__ import print_function` did seem to fix some of the output issues, but i'll have to think a little bit about better handling this.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
