Re: possible bug in AvroUtils

2020-04-29 Thread Reuven Lax
I've copied this failing test into my client, and it passes for me. I can't reproduce the failure. On Wed, Apr 29, 2020 at 6:34 PM Luke Cwik wrote: > +dev +Brian Hulette +Reuven > Lax > > On Wed, Apr 29, 2020 at 4:21 AM Paolo Tomeo wrote: > >> Hi all, >> >> I think the method

Re: contributor permission for Beam Jira tickets

2020-04-29 Thread Darshan Jani
Thanks Luke. Thanks for pointing that out. I am new to Beam contribution community. I would appreciate if you can point me or tag someone of the contributing members who are working on Beam SQL feature, so that we can have a discussion. My view is BEAM-9825 is more about set operations, which

Re: possible bug in AvroUtils

2020-04-29 Thread Luke Cwik
+dev +Brian Hulette +Reuven Lax On Wed, Apr 29, 2020 at 4:21 AM Paolo Tomeo wrote: > Hi all, > > I think the method AvroUtils.toBeamSchema has a not expected side effect. > I found out that, if you invoke it and then you run a pipeline of > GenericRecords containing a timestamp (l tried with

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

2020-04-29 Thread Hannah Jiang
Yes, that's the plan for release images. I had a PR[1] merged several days ago, and need to make change again as the default settings changed. I will add validation tasks with details to release guide as well. The other question is, how easy would it be for a release manager to > accidentally

Re: Python 3.7 docker container fails to build

2020-04-29 Thread Hannah Jiang
There is a PythonDocker Precommit test running for PRs with Python changes. It seems running well.[1] Max, can you please give me a link so I can check more details? Do other images with different Python versions fail as well? 1. https://builds.apache.org/job/beam_PreCommit_PythonDocker_Commit/

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Robert Bradshaw
On Wed, Apr 29, 2020 at 5:08 PM Ahmet Altay wrote: > Nam, this looks better. At least links are working, and the website > visually looks similar and generally in good shape. I think there are still > issues. For example, I do not see any of the images (e.g. the beam logo on > top left is

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Ahmet Altay
Nam, this looks better. At least links are working, and the website visually looks similar and generally in good shape. I think there are still issues. For example, I do not see any of the images (e.g. the beam logo on top left is missing.) On Wed, Apr 29, 2020 at 3:11 PM Brian Hulette wrote: >

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

2020-04-29 Thread Ahmet Altay
On Wed, Apr 29, 2020 at 3:49 PM Valentyn Tymofieiev wrote: > > > On Tue, Apr 28, 2020 at 5:03 PM Robert Bradshaw > wrote: > >> On Tue, Apr 28, 2020 at 4:46 PM Hannah Jiang >> wrote: >> >>> I guess I assumed there was some reason we needed "lightweight images" in our tests (because

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

2020-04-29 Thread Valentyn Tymofieiev
On Tue, Apr 28, 2020 at 5:03 PM Robert Bradshaw wrote: > On Tue, Apr 28, 2020 at 4:46 PM Hannah Jiang > wrote: > >> I guess I assumed there was some reason we needed "lightweight images" in >>> our tests (because licenses take up a lot of space IIRC), but maybe not. >>> Can you elaborate on the

Re: Greetings from Tyson

2020-04-29 Thread Alan Myrvold
Welcome, Tyson! On Wed, Apr 29, 2020 at 3:15 PM Rui Wang wrote: > Welcome! > > -Rui > > On Wed, Apr 29, 2020, 3:13 PM Brian Hulette wrote: > >> Welcome Tyson! >> >> On Wed, Apr 29, 2020 at 2:54 PM Ahmet Altay wrote: >> >>> Welcome! >>> >>> On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang >>>

Re: Jira PR links not being generated?

2020-04-29 Thread Kyle Weaver
Added a comment on https://issues.apache.org/jira/browse/INFRA-19967 On Wed, Apr 29, 2020 at 5:57 PM Ahmet Altay wrote: > Would it be worth filing this as an infra ticket? > > On Mon, Apr 27, 2020 at 4:29 PM Kyle Weaver wrote: > >> Slight correction.. Jira to Github links are back. Github to

Re: Greetings from Tyson

2020-04-29 Thread Rui Wang
Welcome! -Rui On Wed, Apr 29, 2020, 3:13 PM Brian Hulette wrote: > Welcome Tyson! > > On Wed, Apr 29, 2020 at 2:54 PM Ahmet Altay wrote: > >> Welcome! >> >> On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang >> wrote: >> >>> Welcome to the community! >>> >>> >>> On Tue, Apr 28, 2020 at 2:45 PM

Re: Greetings from Tyson

2020-04-29 Thread Brian Hulette
Welcome Tyson! On Wed, Apr 29, 2020 at 2:54 PM Ahmet Altay wrote: > Welcome! > > On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang > wrote: > >> Welcome to the community! >> >> >> On Tue, Apr 28, 2020 at 2:45 PM Tyson Hamilton >> wrote: >> >>> Hello Beam Community, >>> >>> This is just a simple

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Brian Hulette
I left a comment on the PR [1]. I think the reason all of the website content is not being tracked as file renames is because there was a series of commits that created files in the new directory, and then one commit that deleted the old directory. If there were a single commit with all of the

Re: Jira PR links not being generated?

2020-04-29 Thread Ahmet Altay
Would it be worth filing this as an infra ticket? On Mon, Apr 27, 2020 at 4:29 PM Kyle Weaver wrote: > Slight correction.. Jira to Github links are back. Github to Jira links > (which were only recently added) are not being added. > > On Mon, Apr 27, 2020 at 7:20 PM Kyle Weaver wrote: > >>

Re: Automation for Jira

2020-04-29 Thread Ahmet Altay
+1 for the automations. I agree with concerns related to #4. Auto closing issues is not a good experience. A person goes through the work of reporting an issue. This might very well be their first contribution. Automatically closing these issues with no human comments might make the reporter feel

Re: Greetings from Tyson

2020-04-29 Thread Ahmet Altay
Welcome! On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang wrote: > Welcome to the community! > > > On Tue, Apr 28, 2020 at 2:45 PM Tyson Hamilton wrote: > >> Hello Beam Community, >> >> This is just a simple 'Hello' to introduce myself. I'm a Software >> Engineer at Google and have worked with

Re: Python 3.7 docker container fails to build

2020-04-29 Thread Ahmet Altay
+Valentyn Tymofieiev +Hannah Jiang -- in case they have relevant information. On Wed, Apr 29, 2020 at 12:35 PM Maximilian Michels wrote: > Hi, > > has anyone noticed the Python 3.7 Docker container fails to build? I > haven't been able to build the Python 3.7 container, neither locally nor >

Re: Rethinking Python's PortableRunner default job server

2020-04-29 Thread Robert Bradshaw
+1, I was actually thinking about this just the other day. PortableRunner should require job_endpoint to be set, and we can have a nice error message directing the explicit use of FlinkRunner for the old behavior. On Wed, Apr 29, 2020 at 11:50 AM Kyle Weaver wrote: > > Could the error message

Python 3.7 docker container fails to build

2020-04-29 Thread Maximilian Michels
Hi, has anyone noticed the Python 3.7 Docker container fails to build? I haven't been able to build the Python 3.7 container, neither locally nor on Jenkins. I get: 17:48:10 > Task :sdks:python:container:py37:docker 17:49:36 The command '/bin/sh -c pip install -r

Re: Rethinking Python's PortableRunner default job server

2020-04-29 Thread Kyle Weaver
> Could the error message suggest switching to FlinkRunner (and/or other runners that start a job server for you)? Then it seems like the breakage would only be a minor annoyance. Definitely. On Wed, Apr 29, 2020 at 2:49 PM Brian Hulette wrote: > Could the error message suggest switching to

Re: Rethinking Python's PortableRunner default job server

2020-04-29 Thread Brian Hulette
Could the error message suggest switching to FlinkRunner (and/or other runners that start a job server for you)? Then it seems like the breakage would only be a minor annoyance. Brian On Wed, Apr 29, 2020 at 11:32 AM Kyle Weaver wrote: > Hi all, > > Currently, when running a pipeline that has

Re: Automation for Jira

2020-04-29 Thread Tyson Hamilton
+1 for automation. Regarding #4, what about adding the constraint that this rule only applies to issues that are incomplete and require more information from the reporter? Unfortunately it would require a human to triage issues to determine this and apply an appropriate label. Triage should

Rethinking Python's PortableRunner default job server

2020-04-29 Thread Kyle Weaver
Hi all, Currently, when running a pipeline that has the options runner=PortableRunner and job_endpoint unset, the Python SDK spins up a Dockerized Flink job server [1]. This is problematic because the PortableRunner can be used by any portable runner. So for example, a Spark runner user was

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Nam Bui
Hi guys, I'm Nam - from the responsible team of Apache Beam website migration. I am pleased to answer some of the questions here. @aizhamal: Thanks for informing to the community. :) @altay, @robertwb: Yes. there is a problem with the staged version at the moment. We didn't expect some

Re: Automation for Jira

2020-04-29 Thread Brian Hulette
Agree I think this all sounds good except for 4. I like the idea of using automation to help tame the backlog of jiras, but I worry that 4 could lead to a bad experience for users. Say they file a jira and maybe get it assigned, and then watch as it bounces all the way down to closed as obsolete

Re: Automation for Jira

2020-04-29 Thread Robert Bradshaw
+1 to more automation. I'm in favor of all but 4, I think it's quite common for issues to be noticed but not worked on for 60+ days. Most of the time when a developer files an issue they either (1) are working on it right now or (2) are filing it away because it's something they're not working