[
https://issues.apache.org/jira/browse/BEAM-12795?focusedWorklogId=643015&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643015
]
ASF GitHub Bot logged work on BEAM-12795:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Aug/21 21:23
Start Date: 27/Aug/21 21:23
Worklog Time Spent: 10m
Work Description: reuvenlax commented on pull request #15382:
URL: https://github.com/apache/beam/pull/15382#issuecomment-907484209
I was wondering about that, but Dataflow at least replaces things in runner
expansion so I'm not sure if the update would fail or not. I could do
this only for the case of schema key expansion, but that requires updating
every runner to handle both primitive names, which would be much trickier
to do.
Do we have any. update tests in Beam I could run?
On Fri, Aug 27, 2021 at 2:18 PM Lukasz Cwik ***@***.***>
wrote:
> ***@***.**** commented on this pull request.
>
> Unfortunately I didn't think of this before but won't this break pipeline
> update for everyone since every ParDo's name changed (unless they provide
> the rename map which seems painful)?
> ------------------------------
>
> In sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ParDo.java
> <https://github.com/apache/beam/pull/15382#discussion_r697719882>:
>
> > @@ -950,13 +951,13 @@ public PCollectionTuple expand(PCollection<?
extends InputT> input) {
> // Need to figure out where to validate this.
>
> Is there a reason why you didn't move the rest of this logic into
> MultiOutputPrimitive?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/15382#pullrequestreview-740888320>,
> or unsubscribe
>
<https://github.com/notifications/unsubscribe-auth/AFAYJVLUYWNN55Y3WPMDF4TT676JRANCNFSM5CXWLPKA>
> .
> Triage notifications on the go with GitHub Mobile for iOS
>
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
>
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 643015)
Time Spent: 40m (was: 0.5h)
> KVs should not be needed when using schemas
> -------------------------------------------
>
> Key: BEAM-12795
> URL: https://issues.apache.org/jira/browse/BEAM-12795
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Reuven Lax
> Priority: P2
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When using schemas there should be no need to require a KV, as any. key or
> keys can be act as the K or a V in a KV. Users can use schemas without the
> need for KV in all cases except for state/timers DoFns, which today require
> that the input have type KV. We would like to eliminate this requirement.
> Users should be able to specify a key field extraction in their ParDo, e.g.
> ParDo.of(...).withKeyFields("user.location.city"));
> Or in the DoFn itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
