[
https://issues.apache.org/jira/browse/BEAM-13905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511922#comment-17511922
]
Brian Hulette commented on BEAM-13905:
--------------------------------------
Sorry I didn't get to this for 2.38.0. I'm going to drop the fix version though
since this doesn't meet the criteria of a release blocker.
> Apache Beam Python: Dataframe Transforms break when the option
> runtime_type_check is enabled.
> ---------------------------------------------------------------------------------------------
>
> Key: BEAM-13905
> URL: https://issues.apache.org/jira/browse/BEAM-13905
> Project: Beam
> Issue Type: Bug
> Components: dsl-dataframe
> Affects Versions: 2.35.0
> Environment: OS: Linux
> Python 3.8.12
> Reporter: Benoit Clennett-Sirois
> Assignee: Brian Hulette
> Priority: P2
> Fix For: 2.38.0
>
>
> We have discovered a potential bug whereas when you execute a pipeline that
> contains
> a DataframeTransform with the "runtime_type_check" option set to True, a
> cryptic
> error is raised by Apache Beam typecheckng.
> Simple example to reproduce the bug:
>
> {code:java}
> from apache_beam.options.pipeline_options import PipelineOptions
> from apache_beam import Pipeline, Create, Row
> from apache_beam.dataframe.transforms import DataframeTransform
> pipeline = Pipeline(options=PipelineOptions(runtime_type_check=True))
> pipeline | Create([Row(val1=1)]) | DataframeTransform(lambda df: df)
> pipeline.run(){code}
> This raises a apache_beam.typehints.decorators.TypeCheckError:
> {code:java}
> File ".....lib/python3.8/site-packages/apache_beam/typehints/typehints.py",
> line 416, in check_constraint
> raise SimpleTypeHintError
> apache_beam.typehints.decorators.TypeCheckError: According to type-hint
> expected output should be of type <class
> 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>.
> Instead, received 'BeamSchema_118086df_671f_4643_a929_ba65de48e7e8(val1=1)',
> an instance of type <class
> 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>.
> [while running 'DataframeTransform/Unbatch
> 'placeholder_DataFrame_140623617251840'/ParDo(_UnbatchNoIndex)'] {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
