Re: Jira tracker permission

I've added you as contributor - welcome -P. On Mon, Jun 3, 2019, 9:16 PM Yichi Zhang wrote: > Hi, beam-dev, > > This is Yichi Zhang from Google, I just started looking into beam projects > and will be actively working on beam sdk, could someone grant me permission > to beam jira issue tracker?

Re: Timer support in Flink

Yeah, people's eyes likely jump to the big "What is being computed?" header first and skip the small font "expand details" (that's what my eyes did anyway!) Even just moving the expand/collapse to be AFTER the header of the table (or down to the next line) and making the font bigger might help a

Re: [DISCUSS] Portability representation of schemas

On Mon, Jun 3, 2019 at 12:27 PM Brian Hulette wrote: > > It has to go into the proto somewhere (since that's the only way the > SDK can get it), but I'm not sure they should be considered integral parts > of the type. > Are you just advocating for an approach where any SDK-specific information >

Jira tracker permission

Hi, beam-dev, This is Yichi Zhang from Google, I just started looking into beam projects and will be actively working on beam sdk, could someone grant me permission to beam jira issue tracker? My jira username is yichi .

Jira issue tracker permission

Hi, beam-dev, This is Yichi Zhang from Google, I just started looking into beam projects and will be actively working on beam sdk, could someone grant me permission to beam jira issue tracker? My jira username is yichi . Looking

Re: [VOTE] Release 2.13.0, release candidate #2

Thanks, Ankur, for driving the release. Do we have a draft of user-friendly summary of release notes with high-level changes somewhere? If so, please tag me on a document or a PR, or post the link in this thread. Thank you! On Mon, Jun 3, 2019 at 5:38 PM Ankur Goenka wrote: > +1 > Thanks for

Re: [DISCUSS] Cookbooks for users with knowledge in other frameworks

Thank you for the feedback so far. It seems like this will be generally helpful :) I guess next step would be, would anyone be interested in working in this area? We can potentially break this down into starter tasks. On Sat, Jun 1, 2019 at 7:00 PM Ankur Goenka wrote: > +1 for the proposal. >

Re: BQ IT tests fail on TestDataflowRunner - Python SDK

Sounds like your input job was somehow incompatible with the Dataflow worker. Running using a clean virtual env should help verify as Ahmet mentioned. On Mon, Jun 3, 2019 at 5:44 PM Ahmet Altay wrote: > Do you have any other changes? Are you trying from head with a clean > virtual environment?

Re: BQ IT tests fail on TestDataflowRunner - Python SDK

Do you have any other changes? Are you trying from head with a clean virtual environment? If you can share a link to dataflow job (in the apache-beam-testing GCP project), we can try to look at additional logs as well. On Mon, Jun 3, 2019 at 1:42 PM Tanay Tummalapalli wrote: > Hi everyone, > >

Re: [VOTE] Release 2.13.0, release candidate #2

+1 Thanks for validating the release and voting. With 0(-1), 6(+1) and 3(+1 binding) votes, I am concluding the voting process. I am going ahead with the release and will keep the community posted with the updates. On Mon, Jun 3, 2019 at 1:57 PM Andrew Pilloud wrote: > +1 Reviewed the Nexmark

Re: [PROPOSAL] Standardize Gradle structure in Python SDK

Hey Mark & others, We've been following the structure proposed in this thread to extend test coverage for Beam Python SDK on Python 3.5, 3.6, 3.7 interpreters, see [1]. This structure allowed us to add 3.x suites without slowing down the pre/postcommit execution time. We can actually see a drop

Re: [Discuss] Ideas for Apache Beam presence in social media

Hello folks, I have created a spreadsheet where people can suggest tweets [1]. It contains a couple of tweets that have been tweeted as examples. Also, there are a couple others that I will ask PMC members to review in the next few days. I have also created a blog post[2] to invite community

Re: [VOTE] Release 2.13.0, release candidate #2

+1 Reviewed the Nexmark java and SQL perfkit graphs, no obvious regressions over the previous release. On Mon, Jun 3, 2019 at 1:15 PM Lukasz Cwik wrote: > Thanks for the clarification. > > On Mon, Jun 3, 2019 at 11:40 AM Ankur Goenka wrote: > >> Yes, i meant i will close the voting at 5pm and

BQ IT tests fail on TestDataflowRunner - Python SDK

Hi everyone, I ran the Integration Tests - BigQueryStreamingInsertTransformIntegrationTests[1] and BigQueryFileLoadsIT[2] on the master branch locally, with the following command: ./scripts/run_integration_test.sh --test_opts

Re: 1 Million Lines of Code (1 MLOC)

You can run loc and tokei with a --files arg to get a breakdown by file. They're just classifying one file as autoconf: https://github.com/apache/beam/blob/master/sdks/python/MANIFEST.in On Mon, Jun 3, 2019 at 1:02 PM Kenneth Knowles wrote: > Where's the autoconf? > > On Mon, Jun 3, 2019 at

Re: [VOTE] Release 2.13.0, release candidate #2

Thanks for the clarification. On Mon, Jun 3, 2019 at 11:40 AM Ankur Goenka wrote: > Yes, i meant i will close the voting at 5pm and start the release process. > > On Mon, Jun 3, 2019, 10:59 AM Lukasz Cwik wrote: > >> Ankur, did you mean to say your going to close the vote today at 5pm? >> (and

Re: 1 Million Lines of Code (1 MLOC)

Where's the autoconf? On Mon, Jun 3, 2019 at 10:21 AM Kyle Weaver wrote: > > time to delete the entire project and start over again > > Agreed, but this time using Rust. (Just think of all the good press we'll > get on Hacker News! ) > > @ruoyun looks like the c++ is a basic `echo` program for

Re: [DISCUSS] Portability representation of schemas

> It has to go into the proto somewhere (since that's the only way the SDK can get it), but I'm not sure they should be considered integral parts of the type. Are you just advocating for an approach where any SDK-specific information is stored outside of the Schema message itself so that Schema

Re: [VOTE] Release 2.13.0, release candidate #2

Yes, i meant i will close the voting at 5pm and start the release process. On Mon, Jun 3, 2019, 10:59 AM Lukasz Cwik wrote: > Ankur, did you mean to say your going to close the vote today at 5pm? (and > then complete the release afterwards) > > On Mon, Jun 3, 2019 at 10:54 AM Ankur Goenka

Re: [DISCUSS] Portability representation of schemas

On Mon, Jun 3, 2019 at 10:53 AM Reuven Lax wrote: > So I feel a bit leery about making the to/from functions a fundamental > part of the portability representation. In my mind, that is very tied to a > specific SDK/language. A SDK (say the Java SDK) wants to allow users to use > a wide variety

Re: [VOTE] Release 2.13.0, release candidate #2

Ankur, did you mean to say your going to close the vote today at 5pm? (and then complete the release afterwards) On Mon, Jun 3, 2019 at 10:54 AM Ankur Goenka wrote: > Thanks for validating and voting. > > We have 4 binding votes. > I will complete the release today 5PM. Please raise any

Re: [VOTE] Release 2.13.0, release candidate #2

Thanks for validating and voting. We have 4 binding votes. I will complete the release today 5PM. Please raise any concerns before that. Thanks, Ankur On Mon, Jun 3, 2019 at 8:36 AM Lukasz Cwik wrote: > Since the gearpump issue has been ongoing since 2.10, I can't consider it > a blocker for

Re: [DISCUSS] Portability representation of schemas

So I feel a bit leery about making the to/from functions a fundamental part of the portability representation. In my mind, that is very tied to a specific SDK/language. A SDK (say the Java SDK) wants to allow users to use a wide variety of native types with schemas, and under the covers uses the

Re: [DISCUSS] Portability representation of schemas

Ah I see, I didn't realize that. Then I suppose we'll need to/from functions somewhere in the logical type conversion to preserve the current behavior. I'm still a little hesitant to make these functions an explicit part of LogicalTypeConversion for another reason. Down the road, schemas could

Re: 1 Million Lines of Code (1 MLOC)

> time to delete the entire project and start over again Agreed, but this time using Rust. (Just think of all the good press we'll get on Hacker News! ) @ruoyun looks like the c++ is a basic `echo` program for an example pipeline?

Re: 1 Million Lines of Code (1 MLOC)

interesting stats. I am very curious in what we can benefit from merely *32* lines of c++ code in a MLOC repository. On Mon, Jun 3, 2019 at 2:10 AM Maximilian Michels wrote: > Interesting stats :) This metric does not take into a account Beam's > dependencies, e.g. libraries and execution

Re: [VOTE] Release 2.13.0, release candidate #2

Since the gearpump issue has been ongoing since 2.10, I can't consider it a blocker for this release and am voting +1. On Mon, Jun 3, 2019 at 7:13 AM Jean-Baptiste Onofré wrote: > +1 (binding) > > Quickly tested on beam-samples. > > Regards > JB > > On 31/05/2019 04:52, Ankur Goenka wrote: > >

Re: [VOTE] Release 2.13.0, release candidate #2

+1 (binding) Quickly tested on beam-samples. Regards JB On 31/05/2019 04:52, Ankur Goenka wrote: > Hi everyone, > > Please review and vote on the release candidate #2 for the version > 2.13.0, as follows: > > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide

Beam Dependency Check Report (2019-06-03)

High Priority Dependency Updates Of Beam Python SDK: Dependency Name Current Version Latest Version Release Date Of the Current Used Version Release Date Of The Latest Release JIRA Issue google-cloud-bigquery 1.6.1 1.13.0

Re: [VOTE] Release 2.13.0, release candidate #2

+1 (binding) Tested Flink Runner local/cluster execution with the included examples and all supported Flink versions. There is an issue with the staging for remote execution but it is not a blocker since an alternative way exists: https://jira.apache.org/jira/browse/BEAM-7478 Reminder:

Re: 1 Million Lines of Code (1 MLOC)

Interesting stats :) This metric does not take into a account Beam's dependencies, e.g. libraries and execution backends. That would increase the LOCs to millions. On 01.06.19 01:46, Alex Amato wrote: Interesting, so if we play with https://github.com/cgag/loc we could break it down further?

Re: Timer support in Flink

Good point. I think I discovered the detailed view when I made changes to the source code. Classic tunnel-vision problem :) On 30.05.19 12:57, Reza Rokni wrote: :-) https://issues.apache.org/jira/browse/BEAM-7456 On Thu, 30 May 2019 at 18:41, Alex Van Boxel > wrote:

Re: [VOTE] Release 2.13.0, release candidate #2

+1 I validated the artifacts and Python 3. On Sat, Jun 1, 2019 at 7:45 PM Ankur Goenka wrote: > > Thanks Ahmet and Luke for validation. > > If no one has objections then I am planning to move ahead without Gearpump > validation as it seems to be broken from past multiple releases. > >