Re: [Perk] Sharing the love for Flink Forward

Just wanted to chime in here and say that Flink Forward is a great conference. You get to meet lots of people from the Flink community from all over the world, committers as well as end users. There are awesome talks as well. Plus, you get to travel to Berlin which, if you haven't been, I

Re: Bootstrapping Beam's Job Server

Robert, just to be clear about the "callback" proposal. Do you mean that the process startup script listens for an RPC from the Runner to bring up SDK harnesses as needed? I agree this would be helpful to know the required parameter, e.g. you mentioned the Fn Api network configuration. On

Re: Should we allow ValidatesRunner tests to have access to file systems?

I think this should be an integration test if it requires more access than the current ValidatesRunner tests. Although the ValidatesRunner and integration tests are similar, the intent is that the validates runner tests are smaller and more like component tests, and there have been discusions on

jira search in chrome omnibox

In case you want to quickly look up JIRA tickets, e.g., typing 'j', space, 'BEAM-4696'. Search URL: https://issues.apache.org/jira/QuickSearch.jspa?searchString=%s smime.p7s Description: S/MIME Cryptographic Signature

Want to talk Beam at GopherCon?

Hi all! There aren't any Beam talks at GopherCon Denver this year, but that's no reason not to talk Beam! If there are any other Beam community members present this week, at the conference or in Denver I'd love to meet up, and talk about the Go SDK and Beam with you! Possible topics: + What Go

Re: Bootstrapping Beam's Job Server

Understood, so that's a generalized abstraction for creating RPC-based services that manage SDK harnesses. (What we discussed as "external" in the other thread). Would prefer this REST-based, since this makes interfacing with other systems easier. So probably a shell script would already

Re: Process JobBundleFactory for portable runner

On Mon, Aug 27, 2018 at 11:23 AM Maximilian Michels wrote: > > Thanks for your proposal Henning. +1 for explicit environment messages. > I'm not sure how important it is to support cross-platform pipelines. I > can foresee future use but I wouldn't consider it essential. One may want to execute

Re: Bootstrapping Beam's Job Server

I mean that rather than a command line (or docker image) a URL is given that's a GRPC (or REST or ...) endpoint that's invoked to pass what would have been passed by command line arguments (e.g. the FnAPI control plane and logging endpoints). This could be implemented as a script that goes and

Re: Process JobBundleFactory for portable runner

Thanks for your proposal Henning. +1 for explicit environment messages. I'm not sure how important it is to support cross-platform pipelines. I can foresee future use but I wouldn't consider it essential. However, it basically comes for free if we extend the existing environment information

Should we allow ValidatesRunner tests to have access to file systems?

Hello everyone, I am writing a test [1] for the support of @RequiresStableInput annotation in Java SDK [2]. For the test to work, I need to have a ParDo make some side effect (e.g. writing to a file system). However, ValidatesRunner tests in Beam currently cannot depend on external states (cannot

Re: Gradle Races in beam-examples-java, beam-runners-apex

It appears that there is no one working on a fix for the flakes, so I've merged the change to disable parallel tasks on precommit. Andrew On Fri, Aug 24, 2018 at 1:30 PM Andrew Pilloud wrote: > I'm seeing failures due to this on 12 of the last 16 PostCommits. > Precommits take about 22 minutes

[Proposal] Creating a reproducible environment for Beam Jenkins Tests

Hi, I have a proposal for creating a reproducible environment for Jenkins tests by using docker container. The thing is, the environment configurations on Beam Jenkins slaves are sometimes different from developer's machines. Test failures on Jenkins may not be easy to reproduce locally. Also, it

Beam Dependency Check Report (2018-08-27)

High Priority Dependency Updates Of Beam Python SDK: Dependency Name Current Version Latest Version Release Date Of the Current Used Version Release Date Of The Latest Release JIRA Issue google-cloud-bigquery 0.25.0 1.5.0

Re: Test failures list

Thanks Mikhail for the list, it gives very good view of the statuses ! +1 to what Maximilian is saying about ownership and knowledge sharing.Also +1 : flaky tests are priority #1. Etienne Le jeudi 16 août 2018 à 10:59 +0200, Maximilian Michels a écrit : > Thank you Mikhail for looking into test

Re: [PROPOSAL] Prepare Beam 2.7.0 release

Hey everyone, I want to highlight again to those who missed it that if you are aware of any 2.7.0 release blockers, you should add it as a blocker in JIRA with a target version of 2.7.0. It is very helpful to know this information in advance. As of right now, there are no such blockers listed in

JB's back

Hi guys, Maybe you saw that I took some days off last week. I landed back last night, so, just time to unstack my e-mails and I'm back ;) Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [Proposal] Track non-code contributions in Jira

+1 On Sun, Aug 26, 2018, 7:19 PM Griselda Cuevas wrote: > Thanks all for the votes and comments. > > re:Jira - I hear and understand all questions about Jira not being the > absolute best option, however I think the positive points are strong enough > to still use is as primary option. > > re:

Beam failing tests digest

Hi everyone, Heads up for everyone regarding currently open Jira tickets for post-commit tests failures.