[
https://issues.apache.org/jira/browse/BEAM-11252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245995#comment-17245995
]
Tudor Plugaru commented on BEAM-11252:
--------------------------------------
I've done some digging into the code, and reached this piese of code:
{code:java}
return [ pvalue.TaggedOutput( BigQueryWriteFn.FAILED_ROWS,
GlobalWindows.windowed_value((destination, row))) for row in failed_rows ]{code}
[https://github.com/apache/beam/blob/553553df4b286cc6d7fd4af0e0bcf8e1cc5e2785/sdks/python/apache_beam/io/gcp/bigquery.py#L1343]
Can this be the issue since I am using a FixedWindow but here, the failed rows
are emitted back to the GlobalWindow?
{code:java}
WindowedValue(value, timestamp, (GlobalWindow(), ), pane_info){code}
> Writing to BigQuery raises a TypeError "GlobalWindow -> ._IntervalWindowBase"
> -----------------------------------------------------------------------------
>
> Key: BEAM-11252
> URL: https://issues.apache.org/jira/browse/BEAM-11252
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.24.0, 2.25.0
> Reporter: Tudor Plugaru
> Priority: P2
>
> I have a pipeline that reads from a PubSub topic, does some transforms and
> then writes to BigQuery. The exception message is this: {{Cannot convert
> GlobalWindow to apache_beam.utils.windowed_value._IntervalWindowBase}}
> You can find the main part of the pipeline here
> [https://gist.github.com/PlugaruT/4666406bd8792b7b196dc1519c8885a2]
> The first part is basically reading from Pub/Sub and then I apply this
> PTranform.
> And here is the gigantic stack trace:
> [https://gist.github.com/PlugaruT/52bf3834eec95fd5cc5779d3332e1433]
>
> I've tried to downgrade to 2.24.0 version but this is still happening.
> Also, I can't reproduce the exception locally when I run the pipeline with
> DirectRunner. It's happening only when running on Dataflow.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
