[
https://issues.apache.org/jira/browse/BEAM-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lukasz Gajowy updated BEAM-8938:
--------------------------------
Description:
Some tests (I'm not sure if this is the exhaustive list but they seem to appear
in the dataflow console repeatedly) that seem to not be killed and eat our
resources:
-
[test_reshuffle_preserves_timestamps|https://github.com/apache/beam/blob/719b8cc5e51dcd3e98425ecae5ec246657d46eca/sdks/python/apache_beam/transforms/util_test.py#L487]
(spotted multiple times in the dataflow console) (Python SDK)
-
[test_flatten_same_pcollections|https://github.com/apache/beam/blob/44d456830442e5f13b7fd3bd684695e2b69e2c0d/sdks/python/apache_beam/transforms/ptransform_test.py#L596]
(Python SDK)
-
[testPairWithIndexWindowedTimestampedBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L158]
(Java SDK)
-
[testPairWithIndexBasicBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L125]
(Java SDK)
Temporary solution is to ignore them. Real solution requires greater
investigation.
Please see the devlist thread for more context:
[https://lists.apache.org/thread.html/01eb33ae9c05d12bb0698f91adc0021662fdfe2978cfdfde28dc56b2%40%3Cdev.beam.apache.org%3E]
was:
Some tests (I'm not sure if this is the exhaustive list but they seem to appear
in the dataflow console repeatedly) that seem to not be killed and eat our
resources:
-
[test_reshuffle_preserves_timestamps|https://github.com/apache/beam/blob/719b8cc5e51dcd3e98425ecae5ec246657d46eca/sdks/python/apache_beam/transforms/util_test.py#L487]
(spotted multiple times in the dataflow console) (Python SDK)
-
[test_flatten_same_pcollections|https://github.com/apache/beam/blob/44d456830442e5f13b7fd3bd684695e2b69e2c0d/sdks/python/apache_beam/transforms/ptransform_test.py#L596]
(Python SDK)
-
[testPairWithIndexWindowedTimestampedBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L158]
(Java SDK)
-
[testPairWithIndexBasicBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L125]
(Java SDK)
Temporary solution is to ignore them. Real solution requires greater
investigation.
Please see the devlist thread for more context:
[https://lists.apache.org/thread.html/01eb33ae9c05d12bb0698f91adc0021662fdfe2978cfdfde28dc56b2%40%3Cdev.beam.apache.org%3E]
> Tests end up leaving stale Dataflow jobs in apache-beam-testing project and
> exhaust GCP resources
> -------------------------------------------------------------------------------------------------
>
> Key: BEAM-8938
> URL: https://issues.apache.org/jira/browse/BEAM-8938
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Lukasz Gajowy
> Priority: Blocker
>
> Some tests (I'm not sure if this is the exhaustive list but they seem to
> appear in the dataflow console repeatedly) that seem to not be killed and eat
> our resources:
> -
> [test_reshuffle_preserves_timestamps|https://github.com/apache/beam/blob/719b8cc5e51dcd3e98425ecae5ec246657d46eca/sdks/python/apache_beam/transforms/util_test.py#L487]
> (spotted multiple times in the dataflow console) (Python SDK)
> -
> [test_flatten_same_pcollections|https://github.com/apache/beam/blob/44d456830442e5f13b7fd3bd684695e2b69e2c0d/sdks/python/apache_beam/transforms/ptransform_test.py#L596]
> (Python SDK)
> -
> [testPairWithIndexWindowedTimestampedBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L158]
> (Java SDK)
> -
> [testPairWithIndexBasicBounded|https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/SplittableDoFnTest.java#L125]
> (Java SDK)
>
> Temporary solution is to ignore them. Real solution requires greater
> investigation.
> Please see the devlist thread for more context:
> [https://lists.apache.org/thread.html/01eb33ae9c05d12bb0698f91adc0021662fdfe2978cfdfde28dc56b2%40%3Cdev.beam.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
