See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/3764/display/redirect?page=changes>
Changes: [jbonofre] Fix global match test in localfilesystem ------------------------------------------ [...truncated 1020.57 KB...] test_retry_fork_graph (apache_beam.pipeline_test.DirectRunnerRetryTests) ... ok test_element (apache_beam.pipeline_test.DoFnTest) ... ok test_side_input_no_tag (apache_beam.pipeline_test.DoFnTest) ... ok test_side_input_tagged (apache_beam.pipeline_test.DoFnTest) ... ok test_timestamp_param (apache_beam.pipeline_test.DoFnTest) ... ok test_window_param (apache_beam.pipeline_test.DoFnTest) ... ok test_attribute_setting (apache_beam.pipeline_test.PipelineOptionsTest) ... ok test_defaults (apache_beam.pipeline_test.PipelineOptionsTest) ... ok test_dir (apache_beam.pipeline_test.PipelineOptionsTest) ... ok test_flag_parsing (apache_beam.pipeline_test.PipelineOptionsTest) ... ok test_keyword_parsing (apache_beam.pipeline_test.PipelineOptionsTest) ... ok test_view_as (apache_beam.pipeline_test.PipelineOptionsTest) ... ok test_aggregator_empty_input (apache_beam.pipeline_test.PipelineTest) ... ok test_apply_custom_transform (apache_beam.pipeline_test.PipelineTest) ... ok test_create (apache_beam.pipeline_test.PipelineTest) ... ok test_create_singleton_pcollection (apache_beam.pipeline_test.PipelineTest) ... <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/coders/typecoders.py>:134: UserWarning: Using fallback coder for typehint: List[int]. warnings.warn('Using fallback coder for typehint: %r.' % typehint) ok test_flatmap_builtin (apache_beam.pipeline_test.PipelineTest) ... ok test_memory_usage (apache_beam.pipeline_test.PipelineTest) ... ok test_metrics_in_source (apache_beam.pipeline_test.PipelineTest) ... ok test_pipeline_as_context (apache_beam.pipeline_test.PipelineTest) ... <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/coders/typecoders.py>:134: UserWarning: Using fallback coder for typehint: <type 'exceptions.ValueError'>. warnings.warn('Using fallback coder for typehint: %r.' % typehint) ok test_ptransform_overrides (apache_beam.pipeline_test.PipelineTest) ... ok test_read (apache_beam.pipeline_test.PipelineTest) ... ok test_reuse_cloned_custom_transform_instance (apache_beam.pipeline_test.PipelineTest) ... ok test_reuse_custom_transform_instance (apache_beam.pipeline_test.PipelineTest) ... <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline_test.py>:214: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 cm.exception.message, ok test_transform_no_super_init (apache_beam.pipeline_test.PipelineTest) ... ok test_visit_entire_graph (apache_beam.pipeline_test.PipelineTest) ... ok test_pickling (apache_beam.pipeline_test.RunnerApiTest) ... ok test_simple (apache_beam.pipeline_test.RunnerApiTest) Tests serializing, deserializing, and running a simple pipeline. ... ok test_assingleton_multi_element (apache_beam.pvalue_test.PValueTest) ... ok test_pvalue_expected_arguments (apache_beam.pvalue_test.PValueTest) ... ok test_file_checksum_matchcer_invalid_sleep_time (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/testing/pipeline_verifiers_test.py>:130: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 self.assertEqual(cm.exception.message, ok test_file_checksum_matcher_read_failed (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok test_file_checksum_matcher_service_error (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok test_file_checksum_matcher_sleep_before_verify (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok test_file_checksum_matcher_success (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok test_pipeline_state_matcher_fails (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) Test PipelineStateMatcher fails when using default expected state ... ok test_pipeline_state_matcher_given_state (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) Test PipelineStateMatcher successes when matches given state ... ok test_pipeline_state_matcher_success (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) Test PipelineStateMatcher successes when using default expected state ... ok test_append_extra_options (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok test_append_verifier_in_extra_opt (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok test_create_test_pipeline_options (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok test_empty_option_args_parsing (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok test_get_option (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok test_option_args_parsing (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok test_skip_IT (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... SKIP: IT is skipped because --test-pipeline-options is not specified test_basic_execution (apache_beam.testing.test_stream_test.TestStreamTest) ... ok test_basic_test_stream (apache_beam.testing.test_stream_test.TestStreamTest) ... ok test_gbk_execution (apache_beam.testing.test_stream_test.TestStreamTest) ... ok test_test_stream_errors (apache_beam.testing.test_stream_test.TestStreamTest) ... ok test_delete_files_fails_with_invalid_arg (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok test_delete_files_fails_with_io_error (apache_beam.testing.test_utils_test.TestUtilsTest) ... <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/testing/test_utils_test.py>:52: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 error.exception.message.startswith('Delete operation failed')) ok test_delete_files_succeeds (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok test_temp_dir_removes_files (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok test_temp_file_field_correct (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok test_assert_that_fails (apache_beam.testing.util_test.UtilTest) ... ok test_assert_that_fails_on_empty_expected (apache_beam.testing.util_test.UtilTest) ... ok test_assert_that_fails_on_empty_input (apache_beam.testing.util_test.UtilTest) ... ok test_assert_that_passes (apache_beam.testing.util_test.UtilTest) ... ok test_assert_that_passes_empty_equal_to (apache_beam.testing.util_test.UtilTest) ... ok test_assert_that_passes_empty_is_empty (apache_beam.testing.util_test.UtilTest) ... ok test_windowed_value_assert_fail_unmatched_timestamp (apache_beam.testing.util_test.UtilTest) ... ok test_windowed_value_assert_fail_unmatched_value (apache_beam.testing.util_test.UtilTest) ... ok test_windowed_value_assert_fail_unmatched_window (apache_beam.testing.util_test.UtilTest) ... ok test_windowed_value_passes (apache_beam.testing.util_test.UtilTest) ... ok ====================================================================== ERROR: test_basics (apache_beam.examples.complete.tfidf_test.TfIdfTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/examples/complete/tfidf_test.py";,> line 80, in test_basics '--output', os.path.join(temp_folder, 'result')]) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/examples/complete/tfidf.py";,> line 204, in run output = pcoll | TfIdf() File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/examples/complete/tfidf.py";,> line 85, in expand | 'CountDocsPerWord' >> beam.combiners.Count.PerElement()) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 390, in apply label or transform.label) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 400, in apply return self.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/transforms/ptransform.py";,> line 655, in expand return self._fn(pcoll, *args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/transforms/util.py";,> line 190, in RemoveDuplicates | 'RemoveDuplicates' >> Keys()) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 390, in apply label or transform.label) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 400, in apply return self.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/direct/direct_runner.py";,> line 111, in apply_CombinePerKey transform.fn, transform.args, transform.kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/direct/helper_transforms.py";,> line 43, in expand | beam.ParDo(FinishCombine(self._combine_fn))) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 1345, in expand 'GroupByKey operation "%s"' % self.label) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/coders/typecoders.py";,> line 159, in verify_deterministic raise ValueError(error_msg) ValueError: The key coder "TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]" for GroupByKey operation "GroupByKey" is not deterministic. This may result in incorrect pipeline output. This can be fixed by adding a type hint to the operation preceding the GroupByKey step, and for custom key classes, by writing a deterministic custom Coder. Please see the documentation for more details. -------------------- >> begin captured logging << -------------------- root: INFO: Creating temp file: /tmp/tmpP7YXAI/1.txt root: INFO: Creating temp file: /tmp/tmpP7YXAI/2.txt root: INFO: Creating temp file: /tmp/tmpP7YXAI/3.txt root: INFO: Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. --------------------- >> end captured logging << --------------------- ====================================================================== ERROR: test_tfidf_transform (apache_beam.examples.complete.tfidf_test.TfIdfTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/examples/complete/tfidf_test.py";,> line 65, in test_tfidf_transform | beam.Map(re_key)) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/examples/complete/tfidf.py";,> line 85, in expand | 'CountDocsPerWord' >> beam.combiners.Count.PerElement()) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 390, in apply label or transform.label) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 400, in apply return self.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/transforms/ptransform.py";,> line 655, in expand return self._fn(pcoll, *args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/transforms/util.py";,> line 190, in RemoveDuplicates | 'RemoveDuplicates' >> Keys()) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 390, in apply label or transform.label) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 400, in apply return self.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/direct/direct_runner.py";,> line 111, in apply_CombinePerKey transform.fn, transform.args, transform.kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/direct/helper_transforms.py";,> line 43, in expand | beam.ParDo(FinishCombine(self._combine_fn))) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pvalue.py";,> line 103, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/pipeline.py";,> line 436, in apply pvalueish_result = self.runner.apply(transform, pvalueish) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 168, in apply return m(transform, input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 174, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 1345, in expand 'GroupByKey operation "%s"' % self.label) File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/coders/typecoders.py";,> line 159, in verify_deterministic raise ValueError(error_msg) ValueError: The key coder "TupleCoder[BytesCoder, FastPrimitivesCoder]" for GroupByKey operation "GroupByKey" is not deterministic. This may result in incorrect pipeline output. This can be fixed by adding a type hint to the operation preceding the GroupByKey step, and for custom key classes, by writing a deterministic custom Coder. Please see the documentation for more details. -------------------- >> begin captured logging << -------------------- root: INFO: Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. --------------------- >> end captured logging << --------------------- ====================================================================== FAIL: test_using_slow_impl (apache_beam.coders.slow_coders_test.SlowCoders) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/coders/slow_coders_test.py";,> line 41, in test_using_slow_impl import apache_beam.coders.stream AssertionError: ImportError not raised ---------------------------------------------------------------------- Ran 1449 tests in 426.062s FAILED (failures=1, errors=2, skipped=34) Test failed: <unittest.runner.TextTestResult run=1449 errors=2 failures=1> error: Test failed: <unittest.runner.TextTestResult run=1449 errors=2 failures=1> ERROR: InvocationError: '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/py27gcp/bin/python> setup.py test' ___________________________________ summary ____________________________________ docs: commands succeeded ERROR: lint: commands failed ERROR: py27: commands failed ERROR: py27cython: commands failed ERROR: py27gcp: commands failed Build step 'Execute shell' marked build as failure Not sending mail to unregistered user [email protected]
