[
https://issues.apache.org/jira/browse/BEAM-10131?focusedWorklogId=472462&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-472462
]
ASF GitHub Bot logged work on BEAM-10131:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Aug/20 14:32
Start Date: 19/Aug/20 14:32
Worklog Time Spent: 10m
Work Description: piotr-szuberski edited a comment on pull request #12611:
URL: https://github.com/apache/beam/pull/12611#issuecomment-676422822
> Regarding testing: we could consider adding a spanner instance to
apache-beam-testing for integration testing, I'd suggest raising it on dev@ if
you want to pursue it. I also just came across
https://cloud.google.com/spanner/docs/emulator which could be a good option
too. Its a docker container that starts up an in-memory version of spanner to
test against.
@TheNeuralBit Great advice as always! I tried to find something like this
emulator on dockerhub but without success. I managed to successfully use this
emulator, it has much better support than aws for localstack.
Few comments about this PR:
I am almost certain that the Schema doesn't have to be sent as proto in Read
but I didn't come up with anything else.
Another issue is representing the Mutation - for now it's a Row containing 4
fields: operation, table, rows and key_set. It does quite well but I wonder
whether I can do it better.
I erased SpannerWriteResult and return PDone for now - I don't see the way
to keep it without including spanner dependencies to java.core. Because of that
failure mode is FAIL_FAST and I didn't include it in configuration params.
Transactions are not supported because they require a ptransform to be
transferred. I suppose it's doable though and it could be a good future
improvement.
FYI - I'll be OOO the next week so there is absolutely no haste :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 472462)
Time Spent: 1h (was: 50m)
> Patterns: Show pattern for key redistribution when there is no natural key
> --------------------------------------------------------------------------
>
> Key: BEAM-10131
> URL: https://issues.apache.org/jira/browse/BEAM-10131
> Project: Beam
> Issue Type: New Feature
> Components: website
> Reporter: Reza ardeshir rokni
> Priority: P3
> Labels: pipeline-patterns
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Show how to use WithKeys to generate a key space, for redistribution of work
> when no natural key exists.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
