How does param desiredBundleSizeBytes of BoundedSource#split get determined at runtime?

Hi, we are trying to build a custom BoundedSource based on gRPC call. in class BoundedSource we got method below public abstract

Re: [PROPOSAL] Connectors for memcache and Couchbase

I think Madhusudan's proposal does not involve reading the whole contents of the memcached cluster - it's applied to a PCollection of keys. So I'd suggest to call it MemcachedIO.lookup() rather than MemcachedIO.read(). And it will not involve the questions of splitting - however, it *will*

Re: [BEAM-135] Utilities for "batching" elements in a DoFn

Sorry, just saw https://github.com/apache/beam/pull/2211 On Mon, Jul 10, 2017 at 5:37 PM, Robert Bradshaw wrote: > Any progress on this? > > On Thu, Mar 9, 2017 at 1:43 AM, Etienne Chauchot wrote: >> Hi all, >> >> We had a discussion with Kenn yesterday

Re: [BEAM-135] Utilities for "batching" elements in a DoFn

Any progress on this? On Thu, Mar 9, 2017 at 1:43 AM, Etienne Chauchot wrote: > Hi all, > > We had a discussion with Kenn yesterday about point 1 bellow, I would like > to note it here on the ML: > > Using new method timer.set() instead of timer.setForNowPlus() makes the >

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

Have we heard anything about the remaining issues on https://s.apache.org/beam-2.1.0-burndown? Can we move them all to the following release? On Mon, Jul 10, 2017 at 1:22 PM, Jean-Baptiste Onofré wrote: > Hi all, > > all cherry-pick PRs have been merged on the release-2.1.0

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

Hi all, all cherry-pick PRs have been merged on the release-2.1.0 branch. I'm launching couple of builds and tests. I will cut the RC1 just after. Stay tuned for the vote e-mail ! ;) Regards JB On 07/06/2017 06:43 AM, Jean-Baptiste Onofré wrote: No problem, just define the fix version in

Re: MergeBot is here!

(quick update re #2 above): ~4 minutes after I reopened the ticket, it's fixed. https://github.com/apache/infrastructure-puppet/commit/709944291da5e8aea711cb8578f0594deb45e222 updates the website to the correct address. Infra is once again the best. On Mon, Jul 10, 2017 at 12:38 PM, Jason Kuster

Re: MergeBot is here!

Glad to hear everyone's pretty happy about it! Have a couple answers for your questions. Ted: I believe the MFA stuff (two-factor auth on github) is necessary for getting the additional features on GitHub (reviewer, etc), but may not be necessary for MergeBot. I'll check in with Infra and get

Re: BEAM-934 - Jira permission and pull request

Thanks Kenn. Thanks, Almas On 11 Jul 2017 00:05, "Kenneth Knowles" wrote: > I've added you as a Contributor, which is the role you will need to assign > issues. > > On Mon, Jul 10, 2017 at 11:12 AM, Apache Enthu > wrote: > > > Hi could you

Re: BEAM-934 - Jira permission and pull request

I've added you as a Contributor, which is the role you will need to assign issues. On Mon, Jul 10, 2017 at 11:12 AM, Apache Enthu wrote: > Hi could you please add me (eralmas7) as committer please? > > Thanks, > Almas > > On Mon, Jul 10, 2017 at 8:57 AM, Kenneth Knowles

Re: BEAM-934 - Jira permission and pull request

Hi could you please add me (eralmas7) as committer please? Thanks, Almas On Mon, Jul 10, 2017 at 8:57 AM, Kenneth Knowles wrote: > Just a tiny correction - I think the JIRA role "contributor" for the Beam > can take JIRAs without a committer assigning to them. But

Re: MergeBot is here!

+1 Awesome work! Thank you Jason!!! Mark On Mon, Jul 10, 2017 at 10:05 AM, Robert Bradshaw < rober...@google.com.invalid> wrote: > +1, this is great! I'll second Ismaël's list requests, especially 1 and 3. > > On Mon, Jul 10, 2017 at 2:09 AM, Ismaël Mejía wrote: > >

Re: BEAM-933 - Not reproduceable

I believe you can create a JIRA without any special permissions. Here's a direct link that I think will work: https://issues.apache.org/jira/secure/CreateIssue!default.jspa Kenn On Mon, Jul 10, 2017 at 10:10 AM, Kenneth Knowles wrote: > Well, if it is not reproducible then

Re: BEAM-933 - Not reproduceable

Well, if it is not reproducible then could you issue a pull request deleting that bit of the pom.xml? That would resolve the issue, too. Kenn On Mon, Jul 10, 2017 at 10:01 AM, Apache Enthu wrote: > Thanks Kenneth. Unfortunately i'm still unable to reproduce the issue.

Mixed-Language Pipelines

Hey everyone; I've been working on a design for implementing multi-language pipelines within the Beam SDKs (also known as mix-and-match). This kind of pipeline lets us reuse transforms written in one language in any other language that supports the Runner API and the Fn API. Letting us write a

Re: [PROPOSAL] Connectors for memcache and Couchbase

Splitting on slabs should allow you to split more finely grained then per server since each server itself maintains this information. If you take a look at the memcached protocol, you can see that lru_crawler supports a metadump command which will enumerate all the key for a set of given slabs or

Jenkins build is back to normal : beam_Release_NightlySnapshot #473

See