GitHub user mshields822 opened a pull request:
https://github.com/apache/incubator-beam/pull/346
[BEAM-53] Wire PubsubUnbounded{Source,Sink} into PubsubIO
This also refines the handling of record ids in the sink to be
random-but-reused-on-failure, using the same trick as we do for the BigQuery
sink.
Still need to re-do the load tests I did a few weeks back with the actual
change.
Note that last time I tested the DataflowPipelineTranslator does not kick
in and replace the new transforms with the correct native transforms. Need to
dig deeper.
R: @dhalperi
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mshields822/incubator-beam pubsub-runner
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/346.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #346
----
commit ce655a6a7480147f7527fa23818e1d546abaa599
Author: Mark Shields <[email protected]>
Date: 2016-04-12T00:36:27Z
Make java unbounded pub/sub source the default.
commit aafcf5f9d0286ec5a6ed0d634df8ff0902897cdc
Author: Mark Shields <[email protected]>
Date: 2016-05-17T23:44:30Z
Refine record id calculation. Prepare for supporting unit tests with
re-using record ids.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
