See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/869/display/redirect?page=changes>
Changes:
[noreply] Better naming and logging to investigate leaking bigtable resources
------------------------------------------
[...truncated 492.74 KB...]
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m_ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_cells_with_timerange_mutation _[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m_ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_column_family_mutation _[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m____ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_row_mutation _____[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m________ ERROR at setup of
TestWriteToBigtableXlangIT.test_set_mutation ________[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[33m=============================== warnings summary
===============================[0m
../../build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/build/gradleenv/2050596099/lib/python3.11/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17:
DeprecationWarning: The distutils package is deprecated and slated for removal
in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_cells_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_cells_with_timerange_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_column_family_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_row_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_set_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31m= [32m8 passed[0m, [33m19 skipped[0m, [33m7213 deselected[0m, [33m1
warning[0m, [31m[1m5 errors[0m[31m in 2722.18s (0:45:22)[0m[31m =[0m
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava
> FAILED
> Task :sdks:python:test-suites:dataflow:py311:gcpCrossLanguageCleanup
Stopping expansion service pid: 3667152.
Skipping invalid pid: 3667153.
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:54:08.922Z: JOB_MESSAGE_BASIC: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:26.063Z: JOB_MESSAGE_BASIC: All workers have finished the
startup processes and began to receive work requests.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:31.375Z: JOB_MESSAGE_BASIC: Finished operation
ReadFromBigtable/SchemaAwareExternalTransform/beam:schematransform:org.apache.beam:bigtable_read:v1/BigtableIO.Read/Read(BigtableSource)/Impulse+ReadFromBigtable/SchemaAwareExternalTransform/beam:schematransform:org.apache.beam:bigtable_read:v1/BigtableIO.Read/Read(BigtableSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+external_9ReadFromBigtable-SchemaAwareExternalTransform-beam-schematransform-org-apache-beam-bigtable_read-v1-3/PairWithRestriction+external_9ReadFromBigtable-SchemaAwareExternalTransform-beam-schematransform-org-apache-beam-bigtable_read-v1-3/SplitWithSizing
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:31.483Z: JOB_MESSAGE_BASIC: Executing operation
assert_that/Group/CoGroupByKeyImpl/GroupByKey/Create
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:31.997Z: JOB_MESSAGE_BASIC: Finished operation
assert_that/Group/CoGroupByKeyImpl/GroupByKey/Create
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:32.089Z: JOB_MESSAGE_BASIC: Executing operation
external_9ReadFromBigtable-SchemaAwareExternalTransform-beam-schematransform-org-apache-beam-bigtable_read-v1-3/ProcessElementAndRestrictionWithSizing+ReadFromBigtable/SchemaAwareExternalTransform/beam:schematransform:org.apache.beam:bigtable_read:v1/MapElements/Map/ParMultiDo(Anonymous)+ReadFromBigtable/ParDo(_BeamRowToPartialRowData)+Extract
cells+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/CoGroupByKeyImpl/Tag[1]+assert_that/Group/CoGroupByKeyImpl/Flatten/OutputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:32.118Z: JOB_MESSAGE_BASIC: Executing operation
assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at
core.py:3774>)+assert_that/Create/Map(decode)+assert_that/Group/CoGroupByKeyImpl/Tag[0]+assert_that/Group/CoGroupByKeyImpl/Flatten/OutputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:32.485Z: JOB_MESSAGE_BASIC: Finished operation
assert_that/Create/Impulse+assert_that/Create/FlatMap(<lambda at
core.py:3774>)+assert_that/Create/Map(decode)+assert_that/Group/CoGroupByKeyImpl/Tag[0]+assert_that/Group/CoGroupByKeyImpl/Flatten/OutputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:47.722Z: JOB_MESSAGE_BASIC: Finished operation
external_9ReadFromBigtable-SchemaAwareExternalTransform-beam-schematransform-org-apache-beam-bigtable_read-v1-3/ProcessElementAndRestrictionWithSizing+ReadFromBigtable/SchemaAwareExternalTransform/beam:schematransform:org.apache.beam:bigtable_read:v1/MapElements/Map/ParMultiDo(Anonymous)+ReadFromBigtable/ParDo(_BeamRowToPartialRowData)+Extract
cells+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/CoGroupByKeyImpl/Tag[1]+assert_that/Group/CoGroupByKeyImpl/Flatten/OutputIdentity+assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:47.780Z: JOB_MESSAGE_BASIC: Executing operation
assert_that/Group/CoGroupByKeyImpl/GroupByKey/Close
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:48.666Z: JOB_MESSAGE_BASIC: Finished operation
assert_that/Group/CoGroupByKeyImpl/GroupByKey/Close
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:48.722Z: JOB_MESSAGE_BASIC: Executing operation
assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read+assert_that/Group/CoGroupByKeyImpl/MapTuple(collect_values)+assert_that/Group/RestoreTags+assert_that/Unkey+assert_that/Match
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:49.082Z: JOB_MESSAGE_BASIC: Finished operation
assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read+assert_that/Group/CoGroupByKeyImpl/MapTuple(collect_values)+assert_that/Group/RestoreTags+assert_that/Unkey+assert_that/Match
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:56:49.277Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200
2023-10-31T08:59:11.399Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job
2023-10-31_01_53_35-13585569666364917573 is in state JOB_STATE_DONE
[32mINFO [0m
apache_beam.io.gcp.bigtableio_it_test:bigtableio_it_test.py:92 Deleting table
[test-table] and instance [bt-read-tests-1698742393-655ded]
[32mPASSED[0m
apache_beam/io/gcp/bigtableio_it_test.py::TestWriteToBigtableXlangIT::test_delete_cells_mutation
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::TestWriteToBigtableXlangIT::test_delete_cells_with_timerange_mutation
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::TestWriteToBigtableXlangIT::test_delete_column_family_mutation
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::TestWriteToBigtableXlangIT::test_delete_row_mutation
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::TestWriteToBigtableXlangIT::test_set_mutation
[31mERROR[0m
==================================== ERRORS ====================================
[31m[1m___ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_cells_mutation ____[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m_ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_cells_with_timerange_mutation _[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m_ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_column_family_mutation _[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m____ ERROR at setup of
TestWriteToBigtableXlangIT.test_delete_row_mutation _____[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
[31m[1m________ ERROR at setup of
TestWriteToBigtableXlangIT.test_set_mutation ________[0m
cls = <class 'apache_beam.io.gcp.bigtableio_it_test.TestWriteToBigtableXlangIT'>
@classmethod
def setUpClass(cls):
cls.test_pipeline = TestPipeline(is_integration_test=True)
cls.project = cls.test_pipeline.get_option('project')
cls.args = cls.test_pipeline.get_full_options_as_args()
cls.expansion_service = ('localhost:%s' %
os.environ.get('EXPANSION_PORT'))
> timestr = "".join(filter(str.isdigit, str(datetime.datetime.utcnow())))
[1m[31mE AttributeError: type object 'datetime.datetime' has no attribute
'datetime'[0m
[1m[31mapache_beam/io/gcp/bigtableio_it_test.py[0m:163: AttributeError
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/ws/src/sdks/python/pytest_gcpCrossLanguage.xml>
-
[36m[1m=========================== short test summary info
============================[0m
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_cells_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_cells_with_timerange_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_column_family_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_delete_row_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31mERROR[0m
apache_beam/io/gcp/bigtableio_it_test.py::[1mTestWriteToBigtableXlangIT::test_set_mutation[0m
- AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
[31m==== [32m8 passed[0m, [33m19 skipped[0m, [33m7213 deselected[0m,
[31m[1m5 errors[0m[31m in 2770.10s (0:46:10)[0m[31m =====[0m
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava
> FAILED
> Task :sdks:python:test-suites:dataflow:py38:gcpCrossLanguageCleanup
Stopping expansion service pid: 3668059.
Skipping invalid pid: 3668060.
> Task :sdks:python:test-suites:xlang:fnApiJobServerCleanup
Killing process at 3666505
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py311:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:gcpCrossLanguagePythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
For more on this, please refer to
https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings
in the Gradle documentation.
BUILD FAILED in 1h 1m
124 actionable tasks: 89 executed, 33 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/asna5pxffyoy2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
