shunping commented on issue #35771: URL: https://github.com/apache/beam/issues/35771#issuecomment-3146935378
Basically the current way of sending split request does not work for bundles full of timers. https://github.com/apache/beam/blob/138640827e00d6f2e139dfcc38a41ab21d4bb79a/sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go#L1019-L1024 That's because, although timer elements are sent through the same data channel as data elements, the split handling in SDK worker does not consider them the same as data elements which are splittable. I think one way to fix that is to change `estimatedInputElements`. https://github.com/apache/beam/blob/138640827e00d6f2e139dfcc38a41ab21d4bb79a/sdks/go/pkg/beam/runners/prism/internal/worker/bundle.go#L254-L274 We send the estimatedInputElements to the SDK worker for split request. Currently, we include both the number of timers and data elements in that number. https://github.com/apache/beam/blob/138640827e00d6f2e139dfcc38a41ab21d4bb79a/sdks/go/pkg/beam/runners/prism/internal/stage.go#L144-L152 Maybe we don't count the number of timers there? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
