GitHub user mairbek opened a pull request:
https://github.com/apache/beam/pull/3221
[BEAM-1542] SpannerIO sink updates
Unit and integration test.
Logical mutation size estimation.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`.
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mairbek/beam mergespanner
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3221.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3221
----
commit 8db6693c34712c899a1655d5305f2602a9120e9b
Author: MOLIG004 <[email protected]>
Date: 2017-05-04T16:21:23Z
Initial implementation of SpannerIO.Write
This closes #2166.
commit 491f047726ec2146a634a870efa4a43b9d4d643d
Author: Dan Halperin <[email protected]>
Date: 2017-05-04T17:42:07Z
Minor style, compilation, javadoc fixups
commit 7a2d09a0d2783f074348d9b8a1f5bbab7e0a2871
Author: Dan Halperin <[email protected]>
Date: 2017-05-04T17:48:24Z
SpannerCSVLoader: update for TextIO API Change
commit 77be2762825def0c9a62c30b02b9d0a8ff2a15f2
Author: Dan Halperin <[email protected]>
Date: 2017-05-04T17:48:54Z
Fix spanner dependency management
Also minor cleanup alphabetization in root pom.xml
commit 4049047a38bd5dfa6facee8f3c68086c81873e91
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-12T19:59:27Z
Introduced MutationSizeEstimator.
commit 4392de9be272c045a9854340f887e420f25f0f4a
Author: MOLIG004 <[email protected]>
Date: 2017-05-04T16:21:23Z
Initial implementation of SpannerIO.Write
This closes #2166.
commit b092be8df5cd96012c4d67b3defbd3554707edfe
Author: Dan Halperin <[email protected]>
Date: 2017-05-04T17:42:07Z
Minor style, compilation, javadoc fixups
commit e16339a3e699d7bea30f3f3a703b4c7c3ead0f59
Author: Dan Halperin <[email protected]>
Date: 2017-05-04T17:48:54Z
Fix spanner dependency management
Also minor cleanup alphabetization in root pom.xml
commit 9798534793286aeb3deacdc0e30f63032b6077fe
Author: Dan Halperin <[email protected]>
Date: 2017-05-04T17:48:24Z
Delete SpannerCSVLoader
This is not appropriate for examples. SpannerIO should be well-javadoced
and integration tested.
commit f495a414d6d6ec1a3098fb751f499680d070f7c5
Author: Dan Halperin <[email protected]>
Date: 2017-05-15T17:16:18Z
Refine Spanner API tests
And remove outdated Bigtable comment
commit ce5d04cc354c3d1bd3938eff94aa3474adac1e1b
Author: Dan Halperin <[email protected]>
Date: 2017-05-15T17:42:57Z
SpannerIO.Write cleanup and style fixes
* Rename to Write to match the rest of the SDK.
* Convert to AutoValue, delete toString.
* Drop .writeTo(), instead use .write() as default constructor.
* Temporarily drop withBatchSize, as its existence is not clearly
justified.
commit 81e71f62c8c0ba2eb1f5b7e8240650539785fa52
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-15T18:20:18Z
Merge branch 'takeover-2166' of https://github.com/dhalperi/beam into
mergespanner
# Conflicts:
#
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
#
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/GcpApiSurfaceTest.java
commit 16e4b832628205b1f560e5f37fb105d17f2b56ec
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-15T18:38:44Z
Use a new batch size limit.
commit 5a2214a2623fa6d8d4a7b50852b59dc7789837af
Author: Dan Halperin <[email protected]>
Date: 2017-05-15T17:42:57Z
SpannerIO.Write cleanup and style fixes
* Rename to Write to match the rest of the SDK.
* Convert to AutoValue, delete toString.
* Drop .writeTo(), instead use .write() as default constructor.
* Temporarily drop withBatchSize, as its existence is not clearly
justified.
commit 5fc07fe297f6cf5fdd00bb3385d6a23baaf67104
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-15T21:59:22Z
Added a SpannerIO unit test.
commit 0b7b84dfbe17cbc0747197f34c8512610d01d86e
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-15T23:25:33Z
Renaming and documenting
commit 2b5a10e7b64148db2bbcfadbef81dee7c0e10347
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-15T23:56:23Z
Reorganized test.
Better formatting
commit 938b92c71368e0f2334b264de4e1e218bf96eb41
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-16T00:06:59Z
Merge remote-tracking branch 'dhalperi/takeover-2166' into mergespanner
Resolved merge
commit fd2a76cf8b4fe31be3f7e665e11affe2602cf6a9
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-16T00:09:37Z
Pre PR cleanup
commit e24254c2d1d82638b76347a473a46acfef768845
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-16T01:09:44Z
Sketch out integration test
commit 34c7c6f378d19a2c528954c3d20780921e9461dc
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-16T01:40:21Z
End to end test for Cloud Spanner Sink
commit f30b520159df45592f98ef7371e9a099c37472bd
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-16T06:27:13Z
Make checkstyle happy
commit 060ff58bba93ef01b7c5b4cb38e82ca4f4be23b8
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-24T21:57:24Z
Merge branch 'master' of https://github.com/apache/beam into mergespanner
# Conflicts:
# pom.xml
# sdks/java/io/google-cloud-platform/pom.xml
#
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
commit 1f256efd78af37f9860cbc8fa32a7732adeb23bf
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-24T21:57:35Z
Merge branch 'master' of https://github.com/apache/beam into mergespanner
# Conflicts:
# pom.xml
# sdks/java/io/google-cloud-platform/pom.xml
#
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
commit dabda8d1fe8f8ca7b09c6f3a48262fae24c29dc9
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-24T22:03:09Z
Revert style change
commit cf6e8175d755c7498f06ffd51db749f6ddbc98a4
Author: Mairbek Khadikov <[email protected]>
Date: 2017-05-24T22:12:24Z
Revert style change
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
