dongjoon-hyun commented on issue #24958: [SPARK-28153][PYTHON] Use AtomicReference at InputFileBlockHolder (to support input_file_name with Python UDF) URL: https://github.com/apache/spark/pull/24958#issuecomment-516935580 Hi, @HyukjinKwon , @cloud-fan , @brkyvz . It's irrelevant to this PR, but PR build seems to skip many Python UTs. Is it safe to proceed the python PRs like this? ``` Skipped tests in pyspark.sql.tests.test_arrow with python2.7: test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_filtered_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_no_partition_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_null_conversion (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_round_trip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_propagates_spark_exception (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_timestamp_dst (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_timestamp_nat (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_batch_order (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_filtered_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_no_partition_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_null_conversion (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_round_trip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_propagates_spark_exception (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_timestamp_dst (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_timestamp_nat (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_batch_order (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' Skipped tests in pyspark.sql.tests.test_dataframe with python2.7: test_create_dataframe_from_pandas_with_dst (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_create_dataframe_from_pandas_with_timestamp (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_to_pandas (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_to_pandas_avoid_astype (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_query_execution_listener_on_collect_with_arrow (pyspark.sql.tests.test_dataframe.QueryExecutionListenerTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' Skipped tests in pyspark.sql.tests.test_pandas_udf with python2.7: test_pandas_udf_arrow_overflow (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_udf_basic (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_udf_decorator (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_udf_detect_unsafe_type_conversion (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_stopiteration_in_udf (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_udf_wrong_arg (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' Skipped tests in pyspark.sql.tests.test_pandas_udf_grouped_agg with python2.7: test_alias (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_array_type (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_complex_expressions (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_complex_groupby (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_invalid_args (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_manual (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_sql (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_udfs (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_retain_group_columns (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_unsupported_types (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' Skipped tests in pyspark.sql.tests.test_pandas_udf_grouped_map with python2.7: test_array_type_correct (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_coerce (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_column_order (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_complex_groupby (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_datatype_string (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_decorator (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_empty_groupby (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_scalar_udfs_followed_by_grouby_apply (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_positional_assignment_conf (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_register_grouped_map_udf (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_self_join_with_pandas (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_supported_types (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_timestamp_dst (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_udf_with_key (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_unsupported_types (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_wrong_args (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_wrong_return_type (pyspark.sql.tests.test_pandas_udf_grouped_map.GroupedMapPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' Skipped tests in pyspark.sql.tests.test_pandas_udf_scalar with python2.7: test_datasource_with_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_udf_and_sql (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_nondeterministic_vectorized_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_nondeterministic_vectorized_udf_in_aggregate (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_udf_nested_arrays (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_pandas_udf_tokenize (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_register_nondeterministic_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_scalar_iter_udf_close (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_scalar_iter_udf_close_early (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_scalar_iter_udf_init (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_timestamp_dst (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_type_annotation (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_array_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_chained (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_chained_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_check_config (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_datatype_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_dates (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_decorator (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_exception (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_invalid_length (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_nested_struct (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_array (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_binary (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_boolean (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_byte (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_decimal (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_double (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_float (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_int (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_long (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_short (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_null_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_return_scalar (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_return_timestamp_tz (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_string_in_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_struct_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_struct_with_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_timestamps (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_timestamps_respect_session_timezone (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_unsupported_types (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_varargs (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_vectorized_udf_wrong_return_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' Skipped tests in pyspark.sql.tests.test_pandas_udf_window with python2.7: test_array_type (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_bounded_mixed (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_bounded_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_growing_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_invalid_args (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_sql (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_sql_and_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_mixed_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_replace_existing (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_shrinking_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_sliding_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' test_without_partitionBy (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped u'Pandas >= 0.23.2 must be installed; however, your version was 0.19.2.' ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
