See <https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/1005/display/redirect?page=changes>
Changes: [noreply] Clarify description of MLTransform. (#29539) ------------------------------------------ [...truncated 187.87 KB...] Thread 0x00007fc07f4e5700 (most recent call first): File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 474 in read File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 507 in from_io File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 1049 in _thread_receiver File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 296 in run File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 361 in _perform_spawn Current thread 0x00007fc07ff1b740 (most recent call first): File "/usr/lib/python3.10/ssl.py", line 1129 in read File "/usr/lib/python3.10/ssl.py", line 1273 in recv_into File "/usr/lib/python3.10/socket.py", line 705 in readinto File "/usr/lib/python3.10/http/client.py", line 279 in _read_status File "/usr/lib/python3.10/http/client.py", line 318 in begin File "/usr/lib/python3.10/http/client.py", line 1374 in getresponse File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/httplib2/__init__.py";,> line 1396 in _conn_request File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/httplib2/__init__.py";,> line 1444 in _request File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/httplib2/__init__.py";,> line 1724 in request File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/google_auth_httplib2.py";,> line 218 in request File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/apitools/base/py/http_wrapper.py";,> line 397 in _MakeRequestNoRetry File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/apitools/base/py/http_wrapper.py";,> line 348 in MakeRequest File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/apitools/base/py/base_api.py";,> line 728 in _RunMethod File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py";,> line 747 in Get File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,> line 925 in get_job File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 275 in wrapper File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 673 in _update_job File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 730 in state File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,> line 98 in wait_until_in_state File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,> line 74 in run_pipeline File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/pipeline.py";,> line 585 in run File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/pipeline.py";,> line 561 in run File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/pipeline.py";,> line 608 in __exit__ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/examples/cookbook/bigtableio_it_test.py";,> line 185 in test_bigtable_write File "/usr/lib/python3.10/unittest/case.py", line 549 in _callTestMethod File "/usr/lib/python3.10/unittest/case.py", line 591 in run File "/usr/lib/python3.10/unittest/case.py", line 650 in __call__ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/unittest.py";,> line 333 in runtest File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 169 in pytest_runtest_call File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_callers.py";,> line 77 in _multicall File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_manager.py";,> line 115 in _hookexec File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_hooks.py";,> line 493 in __call__ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 262 in <lambda> File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 341 in from_call File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 261 in call_runtest_hook File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 222 in call_and_report File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 133 in runtestprotocol File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/runner.py";,> line 114 in pytest_runtest_protocol File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_callers.py";,> line 77 in _multicall File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_manager.py";,> line 115 in _hookexec File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_hooks.py";,> line 493 in __call__ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/xdist/remote.py";,> line 174 in run_one_test File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/xdist/remote.py";,> line 157 in pytest_runtestloop File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_callers.py";,> line 77 in _multicall File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_manager.py";,> line 115 in _hookexec File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_hooks.py";,> line 493 in __call__ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/main.py";,> line 325 in _main File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/main.py";,> line 271 in wrap_session File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/_pytest/main.py";,> line 318 in pytest_cmdline_main File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_callers.py";,> line 77 in _multicall File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_manager.py";,> line 115 in _hookexec File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/pluggy/_hooks.py";,> line 493 in __call__ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/xdist/remote.py";,> line 355 in <module> File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 1157 in executetask File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 296 in run File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 361 in _perform_spawn File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 343 in integrate_as_primary_thread File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 1142 in serve File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 1640 in serve File "<string>", line 8 in <module> File "<string>", line 1 in <module> Extension modules: zstandard.backend_c, charset_normalizer.md, google._upb._message, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, fastavro._logical_readers, fastavro._schema, fastavro._read, fastavro._logical_writers, fastavro._validation, fastavro._write, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.tslib, pandas._libs.lib, pandas._libs.hashing, pyarrow.lib, pyarrow._hdfsio, pandas._libs.ops, pyarrow._compute, pandas._libs.arrays, pandas._libs.index, pandas._libs.join, pandas._libs.sparse, pandas._libs.reduction, pandas._libs.indexing, pandas._libs.internals, pandas._libs.writers, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.tslibs.strptime, pandas._libs.groupby, pandas._libs.testing, pandas._libs.parsers, pandas._libs.json, grpc._cython.cygrpc, _cffi_backend, bson._cbson, pymongo._cmessage, pyarrow._parquet, pyarrow._fs, pyarrow._hdfs, pyarrow._gcsfs, pyarrow._s3fs, crcmod._crcfunext, regex._regex, shapely.lib, shapely._geos, shapely._geometry_helpers, numpy.linalg.lapack_lite, yaml._yaml, sqlalchemy.cimmutabledict, greenlet._greenlet, sqlalchemy.cprocessors, sqlalchemy.cresultproxy, sklearn.__check_build._check_build, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.linalg._flinalg, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.special._ellip_harm_2, scipy.spatial._ckdtree, scipy._lib.messagestream, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.spatial.transform._rotation, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.optimize._direct, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy._lib._uarray._uarray, scipy.stats._statlib, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._mvn, scipy.stats._rcont.rcont, sklearn.utils._isfinite, sklearn.utils.murmurhash, sklearn.utils._openmp_helpers, sklearn.utils._logistic_sigmoid, sklearn.utils.sparsefuncs_fast, sklearn.preprocessing._csr_polynomial_expansion, sklearn.preprocessing._target_encoder_fast, sklearn.utils._random, sklearn.utils._seq_dataset, sklearn.metrics.cluster._expected_mutual_info_fast, sklearn.metrics._dist_metrics, sklearn.metrics._pairwise_distances_reduction._datasets_pair, sklearn.utils._cython_blas, sklearn.metrics._pairwise_distances_reduction._base, sklearn.metrics._pairwise_distances_reduction._middle_term_computer, sklearn.utils._heap, sklearn.utils._sorting, sklearn.metrics._pairwise_distances_reduction._argkmin, sklearn.metrics._pairwise_distances_reduction._argkmin_classmode, sklearn.utils._vector_sentinel, sklearn.metrics._pairwise_distances_reduction._radius_neighbors, sklearn.metrics._pairwise_fast, sklearn.linear_model._cd_fast, sklearn._loss._loss, sklearn.utils.arrayfuncs, sklearn.svm._liblinear, sklearn.svm._libsvm, sklearn.svm._libsvm_sparse, sklearn.utils._weight_vector, sklearn.linear_model._sgd_fast, sklearn.linear_model._sag_fast (total: 219) [gw0] node down: Not properly terminated [gw0] [31mFAILED[0m apache_beam/examples/cookbook/bigtableio_it_test.py::BigtableIOWriteTest::test_bigtable_write replacing crashed worker gw0 apache_beam/examples/cookbook/bigtableio_it_test.py::BigtableIOWriteTest::test_bigtable_write +++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++ ~~~~~~~~ Stack of Thread-9 (poll_for_job_completion) (139617961498368) ~~~~~~~~~ File "/usr/lib/python3.10/threading.py", line 966, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.10/threading.py", line 1009, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 946, in run self._target(*self._args, **self._kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 171, in poll_for_job_completion time.sleep(sleep_secs) ~~~~~~~~~~~~~~~~~~~~~ Stack of <unknown> (139621375334144) ~~~~~~~~~~~~~~~~~~~~~ File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 361, in _perform_spawn reply.run() File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 296, in run self._result = func(*args, **kwargs) File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 1049, in _thread_receiver msg = Message.from_io(io) File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 507, in from_io header = io.read(9) # type 1, channel 4, payload 4 File "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/execnet/gateway_base.py";,> line 474, in read data = self._read(numbytes - len(buf)) +++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++ [gw8] [31mFAILED[0m apache_beam/examples/cookbook/bigtableio_it_test.py::BigtableIOWriteTest::test_bigtable_write =================================== FAILURES =================================== [31m[1m_____________ apache_beam/examples/cookbook/bigtableio_it_test.py ______________[0m [gw0] linux -- Python 3.10.2 <https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/bin/python3.10> worker 'gw0' crashed while running 'apache_beam/examples/cookbook/bigtableio_it_test.py::BigtableIOWriteTest::test_bigtable_write' [31m[1m___________________ BigtableIOWriteTest.test_bigtable_write ____________________[0m [gw8] linux -- Python 3.10.2 <https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/build/gradleenv/2050596098/bin/python3.10> self = <apache_beam.examples.cookbook.bigtableio_it_test.BigtableIOWriteTest testMethod=test_bigtable_write> @pytest.mark.it_postcommit_sickbay def test_bigtable_write(self): number = self.number pipeline_args = self.test_pipeline.options_list pipeline_options = PipelineOptions(pipeline_args) > with beam.Pipeline(options=pipeline_options) as pipeline: [1m[31mapache_beam/examples/cookbook/bigtableio_it_test.py[0m:185: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [1m[31mapache_beam/pipeline.py[0m:608: in __exit__ self.result = self.run() [1m[31mapache_beam/pipeline.py[0m:561: in run self._options).run(False) [1m[31mapache_beam/pipeline.py[0m:585: in run return self.runner.run_pipeline(self, self._options) [1m[31mapache_beam/runners/dataflow/test_dataflow_runner.py[0m:66: in run_pipeline self.result.wait_until_finish(duration=wait_duration) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <DataflowPipelineResult <Job clientRequestId: '20231128020814306712-3448' createTime: '2023-11-28T02:08:15.582172Z' ...023-11-28T02:08:15.582172Z' steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7efb98439f60> duration = None def wait_until_finish(self, duration=None): if not self.is_in_terminal_state(): if not self.has_job: raise IOError('Failed to get the Dataflow job id.') consoleUrl = ( "Console URL: https://console.cloud.google.com/"; f"dataflow/jobs/<RegionId>/{self.job_id()}" "?project=<ProjectId>") thread = threading.Thread( target=DataflowRunner.poll_for_job_completion, args=(self._runner, self, duration)) # Mark the thread as a daemon thread so a keyboard interrupt on the main # thread will terminate everything. This is also the reason we will not # use thread.join() to wait for the polling thread. thread.daemon = True thread.start() while thread.is_alive(): > time.sleep(5.0) [1m[31mE Failed: Timeout >4500.0s[0m [1m[31mapache_beam/runners/dataflow/dataflow_runner.py[0m:758: Failed ------------------------------ Captured log call ------------------------------- [32mINFO [0m apache_beam.runners.portability.stager:stager.py:322 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/build/apache-beam.tar.gz";> to staging location. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:395 Pipeline has additional dependencies to be installed in SDK worker container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild [32mINFO [0m root:environments.py:313 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.10_sdk:beam-master-20231116 [32mINFO [0m root:environments.py:320 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.10_sdk:beam-master-20231116" for Docker environment [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function pack_combiners at 0x7efbec948040> ==================== [32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function sort_stages at 0x7efbec948820> ==================== [32mINFO [0m apache_beam.internal.gcp.auth:auth.py:134 Setting socket default timeout to 60 seconds. [32mINFO [0m apache_beam.internal.gcp.auth:auth.py:136 socket default timeout is 60.0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:673 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1128020814-305133-ejmnd2bj.1701137294.305324/dataflow_python_sdk.tar... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:683 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1128020814-305133-ejmnd2bj.1701137294.305324/dataflow_python_sdk.tar in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:673 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1128020814-305133-ejmnd2bj.1701137294.305324/pipeline.pb... [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:683 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1128020814-305133-ejmnd2bj.1701137294.305324/pipeline.pb in 0 seconds. [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:854 Create job: <Job clientRequestId: '20231128020814306712-3448' createTime: '2023-11-28T02:08:15.582172Z' currentStateTime: '1970-01-01T00:00:00Z' id: '2023-11-27_18_08_14-3315557514303717945' location: 'us-central1' name: 'beamapp-jenkins-1128020814-305133-ejmnd2bj' projectId: 'apache-beam-testing' stageStates: [] startTime: '2023-11-28T02:08:15.582172Z' steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:856 Created job with id: [2023-11-27_18_08_14-3315557514303717945] [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:857 Submitted job: 2023-11-27_18_08_14-3315557514303717945 [32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:858 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-11-27_18_08_14-3315557514303717945?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58 Console log: [32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59 https://console.cloud.google.com/dataflow/jobs/us-central1/2023-11-27_18_08_14-3315557514303717945?project=apache-beam-testing [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-11-27_18_08_14-3315557514303717945 is in state JOB_STATE_RUNNING [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:08:19.439Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:08:22.589Z: JOB_MESSAGE_BASIC: Executing operation Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:08:22.649Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:08:23.175Z: JOB_MESSAGE_BASIC: Finished operation Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:08:31.909Z: JOB_MESSAGE_BASIC: Executing operation Generate Direct Rows/Create/Impulse+Generate Direct Rows/Create/FlatMap(<lambda at core.py:3774>)+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:08:41.943Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:11:47.021Z: JOB_MESSAGE_BASIC: All workers have finished the startup processes and began to receive work requests. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:11:47.458Z: JOB_MESSAGE_BASIC: Finished operation Generate Direct Rows/Create/Impulse+Generate Direct Rows/Create/FlatMap(<lambda at core.py:3774>)+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/AddRandomKeys+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Reify+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:11:47.524Z: JOB_MESSAGE_BASIC: Executing operation Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:11:48.907Z: JOB_MESSAGE_BASIC: Finished operation Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Close [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T02:11:48.962Z: JOB_MESSAGE_BASIC: Executing operation Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Generate Direct Rows/Create/Map(decode)+Generate Direct Rows/WriteToBigTable/ParDo(_BigTableWriteFn) [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T03:23:06.678Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-11-27_18_08_14-3315557514303717945. [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T03:23:06.697Z: JOB_MESSAGE_BASIC: Finished operation Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Generate Direct Rows/Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys+Generate Direct Rows/Create/Map(decode)+Generate Direct Rows/WriteToBigTable/ParDo(_BigTableWriteFn) [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-11-28T03:23:06.873Z: JOB_MESSAGE_BASIC: Stopping worker pool... [32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-11-27_18_08_14-3315557514303717945 is in state JOB_STATE_CANCELLING - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/pytest_postCommitIT-df-py310.xml> - [36m[1m=========================== short test summary info ============================[0m [31mFAILED[0m apache_beam/examples/cookbook/bigtableio_it_test.py::[1mBigtableIOWriteTest::test_bigtable_write[0m [31mFAILED[0m apache_beam/examples/cookbook/bigtableio_it_test.py::[1mBigtableIOWriteTest::test_bigtable_write[0m - Failed: Timeout >4500.0s [31m================== [31m[1m2 failed[0m, [33m17 skipped[0m[31m in 9209.86s (2:33:29)[0m[31m ==================[0m > Task :sdks:python:test-suites:dataflow:py310:postCommitSickbay FAILED FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Sickbay_Python310/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 181 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py310:postCommitSickbay'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Get more help at https://help.gradle.org. Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation. BUILD FAILED in 2h 39m 58s 9 actionable tasks: 8 executed, 1 from cache Publishing build scan... https://ge.apache.org/s/jrmzcy3v7se6i Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
