[
https://issues.apache.org/jira/browse/BEAM-3376?focusedWorklogId=500193&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-500193
]
ASF GitHub Bot logged work on BEAM-3376:
----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Oct/20 17:23
Start Date: 13/Oct/20 17:23
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #13081:
URL: https://github.com/apache/beam/pull/13081#issuecomment-707893036
> Note that one can always override CombineFn.apply(), so this seems more
about setup/teardown, right?
Yes, setup/teardown was primary motivation for writing this up.
> The disadvantage of a special method means that the semantics are unclear
of whether one is allowed to call apply() with an empty iterable... or is one
forced to check if the iterable is empty to choose to call apply or
default_value? I'm thinking about other places CombineFns may be used used, for
example, in CombiningStates.
I can clarify in the api that this is optional, and defaults to empty
apply() call if we decide to proceed as is.
> If the goal is just to avoid calling setup/teardown in the main program
In your opinion, is calling setup/teardown in main program a concern at all?
Perhaps I am overthinking this.
> we can rewrite the InjectDefault inside CombineGlobally to take an
iterable side input and invoke apply([]) manually (with the setup and teardown)
iff the iterable is empty.
So in this case evaluation of the default will happen after job submission?
I admit I don't fully follow the idea, so if writing a prototype of this idea
is faster than explaining how to do it right to myself of Kamil, we'd
appreciate a draft; thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 500193)
Time Spent: 1h 10m (was: 1h)
> Better expose BigQuery quota errors to users
> --------------------------------------------
>
> Key: BEAM-3376
> URL: https://issues.apache.org/jira/browse/BEAM-3376
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Chamikara Madhusanka Jayalath
> Priority: P3
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently users simply see following error.
> (9e02ab71e036fcc3): Workflow failed. Causes: (9e02ab71e036fbad): S27:Save to
> BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/Read+Save to
> BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/GroupByWindow+Save to
> BigQuery/BatchLoads/MultiPartitionsReshuffle/ExpandIterable+Save to
> BigQuery/BatchLoads/MultiPartitionsWriteTables/ParMultiDo(WriteTables)+Save
> to BigQuery/BatchLoads/MultiPartitionsWriteTables/WithKeys/AddKeys/Map+Save
> to
> BigQuery/BatchLoads/MultiPartitionsWriteTables/Window.Into()/Window.Assign+Save
> to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Reify+Save to
> BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Write+Save to
> BigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/ParDo(ToIsmRecordForGlobalWindow)
> failed., (f88fcdac78f4b497): A work item was attempted 4 times without
> success. Each time the worker eventually lost contact with the service. The
> work item was attempted on: ...
> (there might be additional info in worker logs).
> We should update the BigQuery connector to raise an exception with a proper
> error message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
