damccorm opened a new issue, #21428: URL: https://github.com/apache/beam/issues/21428
We have discovered a potential bug whereas when you execute a pipeline that contains a DataframeTransform with the "runtime_type_check" option set to True, a cryptic error is raised by Apache Beam typecheckng. Simple example to reproduce the bug: ``` from apache_beam.options.pipeline_options import PipelineOptions from apache_beam import Pipeline, Create, Row from apache_beam.dataframe.transforms import DataframeTransform pipeline = Pipeline(options=PipelineOptions(runtime_type_check=True)) pipeline | Create([Row(val1=1)]) | DataframeTransform(lambda df: df) pipeline.run() ``` This raises a apache_beam.typehints.decorators.TypeCheckError: ``` File ".....lib/python3.8/site-packages/apache_beam/typehints/typehints.py", line 416, in check_constraint raise SimpleTypeHintError apache_beam.typehints.decorators.TypeCheckError: According to type-hint expected output should be of type <class 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>. Instead, received 'BeamSchema_118086df_671f_4643_a929_ba65de48e7e8(val1=1)', an instance of type <class 'apache_beam.typehints.schemas.BeamSchema_118086df_671f_4643_a929_ba65de48e7e8'>. [while running 'DataframeTransform/Unbatch 'placeholder_DataFrame_140623617251840'/ParDo(_UnbatchNoIndex)'] ``` Imported from Jira [BEAM-13905](https://issues.apache.org/jira/browse/BEAM-13905). Original Jira may contain additional context. Reported by: benwah. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
