[ https://issues.apache.org/jira/browse/SPARK-28130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-28130: ------------------------------------ Assignee: Apache Spark (was: Hyukjin Kwon) > Pretty messages not being printed for skipped PySpark tests when xmlrunner is > available > --------------------------------------------------------------------------------------- > > Key: SPARK-28130 > URL: https://issues.apache.org/jira/browse/SPARK-28130 > Project: Spark > Issue Type: Improvement > Components: PySpark > Affects Versions: 3.0.0 > Reporter: Hyukjin Kwon > Assignee: Apache Spark > Priority: Major > > When {{xmlrunner}} is installed, seems {{xmlrunner}} does not respect > {{vervosity}} level in unittest. > So the output looks as below > {code} > 02:54:42 Running tests... > 02:54:42 > ---------------------------------------------------------------------- > 02:54:42 SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS > 02:54:42 > ---------------------------------------------------------------------- > 02:54:42 Ran 48 tests in 0.002s > 02:54:42 > 02:54:42 OK (skipped=48) > {code} > and it is not caught by our message detection mechanism added by SPARK-24044 > If we manually set the {{vervocity}} level to {{xmlrunner}}, it prints > messages as below: > {code} > 02:54:42 Running tests... > 02:54:42 > ---------------------------------------------------------------------- > 02:54:42 test_datasource_with_udf > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_mixed_udf > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_mixed_udf_and_sql > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_nondeterministic_vectorized_udf > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_nondeterministic_vectorized_udf_in_aggregate > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_pandas_udf_nested_arrays > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_pandas_udf_tokenize > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_register_nondeterministic_vectorized_udf_basic > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_register_vectorized_udf_basic > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_timestamp_dst > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > 02:54:42 test_type_annotation > (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... SKIP > (0.000s) > {code} > Note that last {{SKIP}} is different. Looks we should fix the regular > expression to catch {{SKIP}} case as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org