[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=441208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441208
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 10:01
Start Date: 04/Jun/20 10:01
Worklog Time Spent: 10m
Work Description: mxm merged pull request #11777:
URL: https://github.com/apache/beam/pull/11777
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441208)
Time Spent: 2h 20m (was: 2h 10m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=441207=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441207
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 10:01
Start Date: 04/Jun/20 10:01
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-638750806
Since the test has been added and this was the missing piece here, I'm going
to merge.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441207)
Time Spent: 2h 10m (was: 2h)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=440723=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440723
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 03/Jun/20 11:51
Start Date: 03/Jun/20 11:51
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-638146606
Please note that the fix commit is the parent of the commit which introduced
the test. So you'll see the test commit fail and the fix commit pass. GitHub
sorts by timestamps which makes this non-obvious.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 440723)
Time Spent: 2h (was: 1h 50m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=440722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440722
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 03/Jun/20 11:48
Start Date: 03/Jun/20 11:48
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-638145542
Probably not. I finally managed to produce a very trimmed down version of
the pipeline for which the execution still stalls.
Please have a look.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 440722)
Time Spent: 1h 50m (was: 1h 40m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=438403=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-438403
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 28/May/20 19:28
Start Date: 28/May/20 19:28
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-635549457
Are those tests sufficient though if they're passing before this PR?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 438403)
Time Spent: 1h 40m (was: 1.5h)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=437938=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437938
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 27/May/20 18:03
Start Date: 27/May/20 18:03
Worklog Time Spent: 10m
Work Description: rohdesamuel commented on a change in pull request
#11777:
URL: https://github.com/apache/beam/pull/11777#discussion_r431341368
##
File path: sdks/python/apache_beam/transforms/trigger.py
##
@@ -1368,7 +1368,7 @@ def _output(
if timestamp is None:
# If no watermark hold was set, output at end of window.
timestamp = window.max_timestamp()
-elif input_watermark < window.end and self.trigger_fn.has_ontime_pane():
+elif output_watermark < window.end and self.trigger_fn.has_ontime_pane():
Review comment:
Yeah this looks good, thanks for thanks for the fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 437938)
Time Spent: 1.5h (was: 1h 20m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=437751=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437751
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 27/May/20 11:55
Start Date: 27/May/20 11:55
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-634610378
This should be covered by the tests added in the PR which introduced the
changes: https://github.com/apache/beam/pull/10304.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 437751)
Time Spent: 1h 20m (was: 1h 10m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=437431=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437431
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 27/May/20 03:48
Start Date: 27/May/20 03:48
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-634231656
I think this fix should be OK. Could you add a test?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 437431)
Time Spent: 1h 10m (was: 1h)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=437318=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437318
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 27/May/20 03:34
Start Date: 27/May/20 03:34
Worklog Time Spent: 10m
Work Description: mxm removed a comment on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-632572299
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 437318)
Time Spent: 1h (was: 50m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=436452=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436452
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 22/May/20 08:36
Start Date: 22/May/20 08:36
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-632572299
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 436452)
Time Spent: 50m (was: 40m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=436094=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436094
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 21/May/20 16:47
Start Date: 21/May/20 16:47
Worklog Time Spent: 10m
Work Description: mxm commented on a change in pull request #11777:
URL: https://github.com/apache/beam/pull/11777#discussion_r428778698
##
File path: sdks/python/apache_beam/transforms/trigger.py
##
@@ -1368,7 +1368,7 @@ def _output(
if timestamp is None:
# If no watermark hold was set, output at end of window.
timestamp = window.max_timestamp()
-elif input_watermark < window.end and self.trigger_fn.has_ontime_pane():
+elif output_watermark < window.end and self.trigger_fn.has_ontime_pane():
Review comment:
Note that this is the fix in question. Please check @rohdesamuel if that
makes sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 436094)
Time Spent: 40m (was: 0.5h)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=436077=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436077
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 21/May/20 16:10
Start Date: 21/May/20 16:10
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-632181899
Please have a look @rohdesamuel if you consider the fix valid. I'm not very
familiar with the Python SDK triggering code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 436077)
Time Spent: 0.5h (was: 20m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=436076=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436076
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 21/May/20 16:09
Start Date: 21/May/20 16:09
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #11777:
URL: https://github.com/apache/beam/pull/11777#issuecomment-632181118
R: @rohdesamuel
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 436076)
Time Spent: 20m (was: 10m)
> Direct Runner execution stalls with test pipeline
> -
>
> Key: BEAM-10054
> URL: https://issues.apache.org/jira/browse/BEAM-10054
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: P2
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Internally, we have a test pipeline which runs with the DirectRunner. When
> upgrading from 2.18.0 to 2.21.0 the test failed with the following exception:
> {noformat}
> tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
> = None
> def raise_(tp, value=None, tb=None):
> """
> A function that matches the Python 2.x ``raise`` statement. This
> allows re-raising exceptions with the cls value and traceback on
> Python 2 and 3.
> """
> if value is not None and isinstance(tp, Exception):
> raise TypeError("instance exception may not have a separate
> value")
> if value is not None:
> exc = tp(value)
> else:
> exc = tp
> if exc.__traceback__ is not tb:
> raise exc.with_traceback(tb)
> > raise exc
> E Exception: Monitor task detected a pipeline stall.
> {noformat}
> I was able to bisect the error. This commit introduced the failure:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731
> If the following conditions evaluates to False, the pipeline runs correctly:
> https://github.com/apache/beam/commit/ea9b1f350b88c2996cafb4d24351869e82857731#diff-2bb845e226f3a97c0f0f737d0558c5dbR1273
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-10054) Direct Runner execution stalls with test pipeline
[
https://issues.apache.org/jira/browse/BEAM-10054?focusedWorklogId=436073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436073
]
ASF GitHub Bot logged work on BEAM-10054:
-
Author: ASF GitHub Bot
Created on: 21/May/20 16:06
Start Date: 21/May/20 16:06
Worklog Time Spent: 10m
Work Description: mxm opened a new pull request #11777:
URL: https://github.com/apache/beam/pull/11777
We have a test pipeline which runs with the DirectRunner. When upgrading
from 2.18.0 to 2.21.0 the test failed with the following exception:
```
tp = Exception('Monitor task detected a pipeline stall.',), value = None, tb
= None
def raise_(tp, value=None, tb=None):
"""
A function that matches the Python 2.x ``raise`` statement. This
allows re-raising exceptions with the cls value and traceback on
Python 2 and 3.
"""
if value is not None and isinstance(tp, Exception):
raise TypeError("instance exception may not have a separate
value")
if value is not None:
exc = tp(value)
else:
exc = tp
if exc.__traceback__ is not tb:
raise exc.with_traceback(tb)
> raise exc
E Exception: Monitor task detected a pipeline stall.
```
I was able to bisect the error. This commit introduced the failure:
ea9b1f350b88c2996cafb4d24351869e82857731
The fix lets to the pipeline running correctly.
Post-Commit Tests Status (on master branch)
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[