[
https://issues.apache.org/jira/browse/BEAM-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990270#comment-16990270
]
Brian Hulette commented on BEAM-8801:
-------------------------------------
Before a Beam SDK sends a pipeline graph to the runner, it does a bunch of work
to construct it (basically calls PTransform#expand on all the transforms). I
meant that we should make the decision about flat schemas at that time, rather
than the current approach where we do it for every element.
I was thinking the approach I would take would be to make PubsubMessageToRow a
PTransform rather than a DoFn. That way we could change the logic in #expand.
[~jingc] are you interested in taking this?
> PubsubMessageToRow should not check useFlatSchema() in processElement
> ---------------------------------------------------------------------
>
> Key: BEAM-8801
> URL: https://issues.apache.org/jira/browse/BEAM-8801
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: Major
>
> Currently we check useFlatSchema() for every element that's processed.
> Instead, we should check it once at pipeline construction time. See
> [comment|https://github.com/apache/beam/pull/10158#discussion_r348805530].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
