[
https://issues.apache.org/jira/browse/BEAM-9910?focusedWorklogId=467581&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-467581
]
ASF GitHub Bot logged work on BEAM-9910:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Aug/20 21:32
Start Date: 06/Aug/20 21:32
Worklog Time Spent: 10m
Work Description: allenpradeep commented on pull request #12255:
URL: https://github.com/apache/beam/pull/12255#issuecomment-670202696
> @allenpradeep given that this seems to be a per-element change, have we
done a performance analysis to confirm that there's won't be a significant
performance degradation due to this ?
>
> LGTM other than that.
Hi Cham,
While dealing with a batch of mutations the size of the batch is already
calculated. We are just passing this size as a KV pair through the pipeline and
so there would be no cost here. But if a batch fails, then we end doing a
recalculation of individual mutation as we commit it separately which is the
added cost here. This may not be significant compared to the cost of making a
RPC call per mutation.
However, I'm planning to work with Niel on this once he is back.
In my initial tests, I did not find performance degradation when the batches
did not fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 467581)
Time Spent: 50m (was: 40m)
> Add byte counters to SpannerIO.write
> ------------------------------------
>
> Key: BEAM-9910
> URL: https://issues.apache.org/jira/browse/BEAM-9910
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Allen Pradeep Xavier
> Assignee: Allen Pradeep Xavier
> Priority: P3
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently SpannerIO has counters that count the number of mutationgroups
> written and the number successful writes to spanner.
> A common customer complaint regarding Spanner import is the lack of job
> progress information and a byte counter which shows the successful bytes
> written to spanner would help alleviate some customer pain.
>
> cc [~nielm]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
