Re: [FYI] Rephrasing the 'lull'/processing stuck logs

2020-01-09 Thread Steve Niemitz
One other nice enhancement around this would be if a transform could indicate that it was executing a "slow" operation. A good example is writing in BigQueryIO, it's very reasonable/normal for a load job to run for more than 5 minutes, and the "stuck" message can be confusing to users. The

[FYI] Rephrasing the 'lull'/processing stuck logs

2020-01-09 Thread Pablo Estrada
Hello Beam users and community, The Beam Python SDK, and Java workers have a utility where they will print a log message whenever there's an execution thread where no state transitions happen for over five minutes. These messages are common in two scenarios: 1. A deadlock happening in the worker