[
https://issues.apache.org/jira/browse/BEAM-11690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272417#comment-17272417
]
Brian Hulette commented on BEAM-11690:
--------------------------------------
There's a question about where this error should be caught. I think I had an
offline conversation with [~robertwb] where I floated the idea of just
rejecting non-portable coders in ExternalTransform, but he pointed out that
it's possible the expanding SDK understands pythonsdk_any.
We could provide the most helpful error if we catch this on the Python SDK
side, in sql.py. I wouldn't want to be too aggressive with type checking there,
as we could end up having two sources of truth for what types are supported by
SQL (one in Java and one in Python). It's probably fine to just detect
pythonsdk_any though. I think it would be appropriate to reject that for any
schema-aware external transform that's using a jar expansion service.
> Python SqlTransform fails with an unhelpful Java error when type inference
> uses the pythonsdk_any fallback
> ----------------------------------------------------------------------------------------------------------
>
> Key: BEAM-11690
> URL: https://issues.apache.org/jira/browse/BEAM-11690
> Project: Beam
> Issue Type: Bug
> Components: cross-language, sdk-py-core
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: P2
>
> See
> https://stackoverflow.com/questions/65903919/local-testing-apache-beam-with-sqltransform-in-python-sdk-i-receive-an-error
> for a good example of this.
> We should throw a more helpful error in this situation:
> - It should mention the problematic field(s)
> - It should suggest a solution (try wrapping in casts if using beam.Row,
> don't use Any in a NamedTuple)
> - It might point to some documentation about supported SQL types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
