Reproducible Environment for Jenkins Tests By Using Container

Hello, I am working on creating a reproducible build environment for BEAM. The goal is having a reproducible environment by using docker for Beam build and test on Jenkins and contributors' local machines. More details are in the proposal:

[Proposal] Automation For Beam Dependency Check

ment/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/ Thanks. Best. Yifan Zou

Re: Kubernetes cluster of apache-beam-testing project

Seems like kubectl v1.9.0 was installed on those nodes. Here is the puppet script I found that used to download and install kubectl on beam-jenkins. https://github.com/apache/infrastructure-puppet/blob/95aed4f4f7cdce79a1f0fc8b1e581f13fedf63ff/modules/build_slaves/manifests/kube.pp On Thu, May 24,

Re: Beam Dependency Check Report (2018-06-13)

a's proposal <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit> . For more readings: [Proposal] Beam dependency check automation <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp> by Yifan Zou [P

Re: Precommits broken?

Probably due to the precommit filter applied in #5611 ? On Wed, Jun 13, 2018 at 5:02 PM Andrew Pilloud wrote: > Looks like statuses got posted between me writing this email and sending > it. Still wondering why the python and go jobs appear to be

Re: Beam Dependency Check Report (2018-06-13)

and assign them to owners to review/upgrade dependencies. Regards. Yifan Zou On Wed, Jun 13, 2018 at 10:45 AM Chamikara Jayalath wrote: > > Thanks Yifan. > > On Wed, Jun 13, 2018 at 10:21 AM Ahmet Altay wrote: > >> Thanks Yifan, this is great! >> >> My unsolic

Re: Beam Dependency Check Report (2018-06-13)

Thanks Kenn. These are all direct dependencies. On Wed, Jun 13, 2018 at 4:40 PM Yifan Zou wrote: > Thanks everyone for feedbacks! > > We will embed some text to explain the details of the report and guide > people what to do with it at this point. Cham and I prepare to start >

Re: Beam Dependency Check Report (2018-06-13)

help us maintain Beam more easily and probably >> help us fixing CVE as well. >> >> Etienne >> >> Le mercredi 13 juin 2018 à 07:45 -0700, Yifan Zou a écrit : >> >> Hi, >> >> >> I want to follow up and explain this email. >>

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

+1 Python quick-start and mobile-gaming on core runners passed. On Sun, Jun 17, 2018 at 10:32 AM Pablo Estrada wrote: > Going to https://github.com/apache/beam/tree/v2.5.0-RC2 gives me a 404 > error. May need to push the label? > Best > -P. > > On Sun, Jun 17, 2018 at 1:15 AM Romain Manni-Bucau

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

+1 Verified Python wheels. They are good. https://builds.apache.org/job/beam_PostRelease_Python_Candidate/117/ On Tue, Jun 19, 2018 at 10:32 AM Udi Meiri wrote: > +1 > > On Mon, Jun 18, 2018 at 6:16 PM Boyuan Zhang wrote: > >> Follow up with pervious email( sorry for the inconvenience). >> >>

Re: [Proposal] Automation For Beam Dependency Check

hu, May 24, 2018 at 7:08 AM Yifan Zou wrote: > >> Hello, >> >> I have a proposal to automate Beam dependency check. Since some Beam >> dependent packages are out-of-date, we want to identify them and check for >> dependency updates regularly in the future. G

Re: [VOTE] Code Review Process

+1 On Fri, Jun 1, 2018 at 12:10 PM Robert Bradshaw wrote: > +1 > > On Fri, Jun 1, 2018 at 12:06 PM Chamikara Jayalath > wrote: > >> +1 >> >> Thanks, >> Cham >> >> On Fri, Jun 1, 2018 at 11:36 AM Jason Kuster >> wrote: >> >>> +1 >>> >>> On Fri, Jun 1, 2018 at 11:36 AM Ankur Goenka wrote: >>>

Beam Dependency Ownership

Hi all, We now have the automated detections for Beam dependency updates and sending a weekly report to dev mailing list. In order to address the updates in time, we want to find owners for all dependencies of Beam, and finally, Jira bugs will be automatically created and assigned to the owners

Re: Beam Dependency Ownership

lePlugin.groovy#L325 > > On Wed, Jun 27, 2018 at 8:52 AM Yifan Zou wrote: > >> Hi all, >> >> We now have the automated detections for Beam dependency updates and >> sending a weekly report to dev mailing list. In order to address the >> updates in time,

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

follow up the previous email. The automated validation test verified the wheel below: apache_beam-2.5.0-cp27-cp27mu-manylinux1_x86_64.whl On Tue, Jun 19, 2018 at 5:11 PM Yifan Zou wrote: > +1 > Verified Python wheels. They are good. > https://builds.apach

Re: Filtered Pre-commit triggering is BACK!

Thanks Scott! It's nice to have this feature. On Tue, Jun 26, 2018 at 2:24 PM Pablo Estrada wrote: > This is great. Reducing load on infrastructure should help Beam scale into > a larger project : ) - Thanks Scott! > > On Tue, Jun 26, 2018 at 2:21 PM Scott Wegner wrote: > >> By popular demand

Apache Beam Jenkins Machines Upgrade

, the average waiting time of 10 latest Python PreCommits was 22.2 minutes, and now it drops to 10 seconds! Special thanks to Alan Myrvold and Jason Kuster for helping on the smooth swapping process. Regards. Yifan Zou

[DISCUSS]: Beam 2.3.0 release archetypes missing mobile gaming examples

Greetings, We stopped copying the mobile gaming examples into maven archetypes after merging Java8 examples to "mian" Java examples. So, we're not able to run those pipelines via creating a maven projects since those files are not included. To solve this problem, we could: - 2.3.1 point ct

Re: Cron schedule is failing on the Beam jenkins jobs

s working on triggering > postcommits on new commits and might know more about the mechanisms. > > On Tue, Jul 31, 2018 at 11:46 AM Yifan Zou wrote: > >> I notice that the Cron schedule for the Beam Jenkins jobs are not working >> as expected. For example, the job_Depend

Re: [VOTE] Apache Beam, version 2.6.0, release candidate #1

+1 Tested Python quickstarts and mobile gaming examples against tar and wheel versions. https://builds.apache.org/job/beam_PostRelease_Python_Candidate/123/ On Wed, Aug 1, 2018 at 8:27 AM Andrew Pilloud wrote: > +1 tested the Beam SQL jar from the Maven Central repo, it worked. > > On Wed, Aug

Maintaining Beam Dependencies

Please refer to the Dependencies Guide <https://beam.apache.org/contribute/dependencies/> for more details. Thank you very much. Regards. Yifan Zou

Re: [VOTE] Community Examples Repository

3 for all reasons above. Keeping the examples in the Beam repository is a straightforward way to let the examples being visible and maintainable. And as Lukasz Cwik mentioned, the Quickstart makes a clear and easy approach for users to generate the archetype and run examples. I did not see enough

Re: gradlew :rat broken

I've also encountered that problem before. The nosetests.xml doesn't have the license specified and you have to removed it before running the :rat task locally. On Wed, Aug 15, 2018 at 4:19 PM Pablo Estrada wrote: > The files that are causing the failure seem to have been generated. They > are

[Proposal] Creating a reproducible environment for Beam Jenkins Tests

Hi, I have a proposal for creating a reproducible environment for Jenkins tests by using docker container. The thing is, the environment configurations on Beam Jenkins slaves are sometimes different from developer's machines. Test failures on Jenkins may not be easy to reproduce locally. Also, it

Dependency Ownership Followup

Hi, Thanks all for signing up the spreadsheet and taking the ownership of Beam dependencies. We integrated the owners information into the Beam code base via some yaml files (#5964

Re: Beam Dependency Ownership

tickets of Beam deps in order to track the upgrade process. Thank you. Regards. Yifan On Mon, Jul 9, 2018 at 10:24 AM Yifan Zou wrote: > If you haven't already, please take a look at the Beam SDK Dependency > Ownership > <https://docs.google.com/sp

Re: [DISCUSS] Versioning, Hadoop related dependencies and enterprise users

on the revised policy in Cham's email. Thanks all. Regards. Yifan Zou On Tue, Sep 4, 2018 at 5:35 PM Chamikara Jayalath wrote: > Based on this email thread and offline feedback from several folks, > current concerns regarding dependency upgrade policy and tooling seems to > be followin

Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

important to still be able to > > easily run test locally, without any VM/Docker required. It should be > > activated by profile or so. > > > > Regards > > JB > > > > On 27/08/2018 19:53, Yifan Zou wrote: > > > Hi, > > > >

Re: [DISCUSS] Versioning, Hadoop related dependencies and enterprise users

start alerting again? >> > > I think this makes sense. Setting a "fix version" will be specially useful > for dependency changes that result in API changes that have to be postponed > till next major version of Beam. > > On grouping, I believe we already gr

Re: [PROPOSAL] Beam jira bot exceptions

Thanks for raising this problem. There was a discussion about those issues and we came up with new policies for the Jira bot. At this point, we could close the JIRAs so that the bot won't create duplicate issues with the same version.

Re: [DISCUSS] Versioning, Hadoop related dependencies and enterprise users

since JIRA was closed. Thank you. Regards. Yifan On Wed, Sep 5, 2018 at 2:14 PM Yifan Zou wrote: > +1 on the jira "fix version". > The release frequency of dependencies are various, so that using new > information such as versions from the Jira closing date to reopen the &

[PROPOSAL] JIRA Automation for Beam Dependency Upgrades

Hi all, I have a short proposal on automatically file JIRA tickets that tracks the Beam dependency update process. Any comments are welcome. https://docs.google.com/document/d/1XXTMnofizSQZSorZy4NFlxKfx5f17PGRmws-DngoTms/edit# Regards. Yifan

Re: Automatically create JIRA tickets for failing post-commit tests

+1 to Andrew's concerns. Leaving the tickets unassigned will cause the ticket being ignored and no actions being taken. I can see the challenges on ticket assignment. Like Mikhail mentioned, the plugin does not support dynamic assignments. We have to implement custom script to determine the

Re: Beam Dependency Ownership

we collect the >> data). Otherwise we'll have to put an extra effort to make sure that the >> spreadsheet, BeamModulePlugin, and Python setup.py are in sync. We can >> decide on the exact format of the comment to make sure that automated tool >> can easily parse the commen

Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

> frustrations with Jenkins. > > On Wed, Sep 5, 2018 at 2:30 PM Yifan Zou wrote: > >> Thank you all for making comments on this and I apologize for the late >> reply. >> >> To clarify the concerns of testing locally, it is still able to run tests >> without Dock

Re: Inconsistent jenkins workers

shared for wider review shortly. Yifan Zou On Mon, Apr 16, 2018 at 1:10 PM Robert Bradshaw <rober...@google.com> wrote: > I've been trying to debug why beam_PreCommit_Python_GradleBuild seems to be > failing so often, and it looks like the beam-sdks-python:setupVirtualenv > task succee

Re: Inconsistent jenkins workers

of restrictions on python tests. But for now, I can hardly tell when we could have new Jenkins VMs since the latest OS image is not available yet. Yifan Zou On Mon, Apr 16, 2018 at 3:10 PM Robert Bradshaw <rober...@google.com> wrote: > Thanks. > > In the short term, I could try to l

Re: Inconsistent jenkins workers

ks, so in that time frame we could start looking at upgrading the image > and getting Dockerized builds going. Yifan, are you planning on owning this > area? > > On Mon, Apr 16, 2018 at 4:02 PM Yifan Zou <yifan...@google.com> wrote: > >> Those machines are managed by apa

Re: Docker missing on Beam15

t;> >> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467 >> >> Any help to further investigate or fix would be appreciated! >> >> Thanks, >> Thomas >> >> >> >> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou wrote: &

Re: Docker missing on Beam15

Hi, The docker has been installed on all Jenkins VMs. The image build process was interrupted by a grpc connection issue. *11:02:12* Starting process 'command 'docker''. Working directory:

Re: Docker missing on Beam15

i, > > Can we restart docker as it seems to have fixed the issue for others > https://github.com/moby/moby/issues/31849 ? > > Thanks, > Ankur > > On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou wrote: > >> Hi, >> >> The docker has been installed on all Jenkins

Re: Docker missing on Beam15

t; which should not even execute since we are using Docker >> >> >> On Mon, Oct 22, 2018 at 10:50 PM Yifan Zou wrote: >> >>> I'm not able to reproduce that error in Beam6 (#459 >>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>, &

Re: INSTANCE_GROUP_MANAGERS quota limit exceeded

+1. I believe there were too many dataflow jobs running at that moment that exceeded the compute engine quota of us-central1 ( the current quota is 100). Yifan On Fri, Nov 2, 2018 at 1:49 PM Alan Myrvold wrote: > Not sure, but I expect that each dataflow job needs a managed instance > group on

Re: Python PostCommit failures

The integration test has been fixed by #6567 <https://github.com/apache/beam/pull/6567>. The Python PostCommit Verify is back to normal. Thanks. - Yifan On Thu, Oct 4, 2018 at 12:22 AM Yifan Zou wrote: > We are seeing consecutive Python PostCommit > <https://builds.apache.or

Re: Beam Dependency Check Report (2018-10-01)

The job failed. I am looking into this. On Mon, Oct 1, 2018 at 10:50 AM Mikhail Gryzykhin wrote: > Hi everyone, > > Is this report supposed to be empty? > > Regards, > --Mikhail > > Have feedback ? > > > On Mon, Oct 1, 2018 at 5:07 AM Apache Jenkins Server < >

Python PostCommit failures

We are seeing consecutive Python PostCommit failures due to a broken IT test. I am looking into it (BEAM-5643 ). Yifan

Re: Beam Dependency Check Report (2018-10-01)

The job recovered. The latest dependency report <https://builds.apache.org/job/beam_Dependency_Check/lastSuccessfulBuild/artifact/src/build/dependencyUpdates/beam-dependency-check-report.html> was created and shared with dev@. -Yifan On Mon, Oct 1, 2018 at 10:50 AM Yifan Zou wrote: >

Re: [ANNOUNCEMENT] New Beam chair: Kenneth Knowles

Congratulations Kenn! On Wed, Sep 19, 2018 at 2:36 PM Robert Burke wrote: > Congrats Kenn! :D > > On Wed, Sep 19, 2018, 2:21 PM Ismaël Mejía wrote: > >> Congratulations and welcome Kenn as new chair! >> Thanks Davor for your hard work too. >> >> On Wed, Sep 19, 2018 at 11:14 PM Rui Wang

Re: Cleanup Jenkins old jobs

+1 on this. There are also some jobs for test/experiment purpose which are stale. Good to remove them that makes the UI neat. On Mon, Sep 24, 2018 at 2:56 PM Andrew Pilloud wrote: > This sounds good to me. We have a significant number of jobs have been > renamed. Having both the old and new

Re: Our jenkins beam1 server is down

I am looking on it. On Wed, Jan 16, 2019 at 8:18 AM Ismaël Mejía wrote: > Can somebody PTAL. Sadly the poor jenkins shuffling algorithm is > sending most builds to it so there are issues to validate some PRs. >

Re: Our jenkins beam1 server is down

to my PR: > > Java_Examples_Dataflow, > Portable_Python, > Website_Stage_GCS > > On Wed, Jan 16, 2019 at 9:25 AM Yifan Zou wrote: > >> I am looking on it. >> >> On Wed, Jan 16, 2019 at 8:18 AM Ismaël Mejía wrote: >> >>> Can somebody PTAL. Sadly the poor

Re: Our jenkins beam1 server is down

quot;*10:12:21* ERROR: beam14 is offline; >> cannot locate JDK 1.8 (latest)". >> >> Beam1 is not the only one broken? >> >> On Wed, Jan 16, 2019 at 3:45 PM Yifan Zou wrote: >> >>> The beam1 was still accepting jobs and breaking them after reset this >>&

Re: Our jenkins beam1 server is down

The inventory test on the beam1 passed. The beam1 is back to normal. https://builds.apache.org/job/beam_Inventory_beam1/303/ On Tue, Jan 22, 2019 at 11:41 AM Yifan Zou wrote: > Thanks for reporting the failures. Just disconnect and reconnect beam1. I > am creating a PR that force run

Re: Our jenkins beam1 server is down

-jenkins-slave-group-51fn snmpd[16379]: message repeated 5 times: [ error on subcontainer 'ia_addr' insert (-1)] On Wed, Jan 23, 2019 at 7:19 AM Ismaël Mejía wrote: > Looks like beam9 is now gone. > > On Tue, Jan 22, 2019 at 8:57 PM Yifan Zou wrote: > > > > The inventory tes

Re: Beam snapshots broken

1, 2018 at 7:33 PM Boyuan Zhang wrote: > > > > Seems like, all failed jobs are not owing to the single task failure. > There failed task were executed on beam9, which was rebooted yesterday > because python tests failed continuously. +Yifan Zou may have more useful > content

Re: [VOTE] Donating the Dataflow Worker code to Apache Beam

+1 On Fri, Sep 14, 2018 at 4:20 PM David Morávek wrote: > +1 > > > > On 15 Sep 2018, at 00:59, Anton Kedin wrote: > > +1 > > On Fri, Sep 14, 2018 at 3:22 PM Alan Myrvold wrote: > >> +1 >> >> On Fri, Sep 14, 2018 at 3:16 PM Boyuan Zhang wrote: >> >>> +1 >>> >>> On Fri, Sep 14, 2018 at 3:15 PM

Re: Our jenkins beam1 server is down

The beam1 and 14 are back and building. On Thu, Jan 17, 2019 at 7:04 AM Ismaël Mejía wrote: > Thanks Yifan for taking care. > > On Thu, Jan 17, 2019 at 1:24 AM Yifan Zou wrote: > > > > Yes, beam14 is offline as well. We're on it. > > > > On Wed, Jan 16, 20

Re: Our jenkins beam1 server is down

on_Phrase/88/console > > https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/141/console > > On Tue, Jan 22, 2019 at 10:53 AM Yifan Zou wrote: > >> The beam1 and 14 are back and building. >> >> On Thu, Jan 17, 2019 at 7:04 AM Ismaël Mejía

Re: Beam Dependency Check Report (2018-06-13)

gards, > Ismaël > > > On Thu, Jun 14, 2018 at 11:37 PM Yifan Zou wrote: > >> Thank you Paul for letting us know this issue. We will take care of it >> when upgrading dependencies. >> >> On Thu, Jun 14, 2018 at 7:23 AM Paul Gerver wrote: >> >&g

Re: Python36/37 not installed on Beam2 and Beam12?

You could try to ping them in the slack channel https://the-asf.slack.com/messages/ if it is really urgent. On Wed, Mar 20, 2019 at 5:29 PM Mark Liu wrote: > Hi, > > I saw occasional py36 tox test failure in beam_PreCommit_Python > and beam_Release_NightlySnapshot in cron job >

Re: Frequent failures on beam8

the beam8 is disabled by now. On Mon, Mar 25, 2019 at 2:06 PM Mikhail Gryzykhin wrote: > Yifan is looking into this. > > On Mon, Mar 25, 2019 at 1:55 PM Boyuan Zhang wrote: > >> Hey all, >> >> Could anyone help take a look at beam8 >> ? Seems

Re: [ANNOUNCE] New committer announcement: Mark Liu

Congratulations Mark! On Sun, Mar 24, 2019 at 10:25 PM Connell O'Callaghan wrote: > Well done congratulations Mark!!! > > On Sun, Mar 24, 2019 at 10:17 PM Robert Burke wrote: > >> Congratulations Mark!  >> >> On Sun, Mar 24, 2019, 10:08 PM Valentyn Tymofieiev >> wrote: >> >>>

Changes in Beam Jenkins Agents

Hi, Our Jenkins are in a bad condition. 8 agents are down at this time, and they are not going to be restored because of some bad errors happened on the Puppet server due to the consistent re-provisioning. There are several discussions in the recent weeks between the ASF Infra and us. In general,

Re: Changes in Beam Jenkins Agents

Great! Thank you, Lukasz! On Thu, Apr 4, 2019 at 3:10 AM Łukasz Gajowy wrote: > I verified load tests and IO performance tests jobs. Looking good. Thanks > for doing this! > > Łukasz > > > > czw., 4 kwi 2019 o 03:35 Yifan Zou napisał(a): > >> Hi, >> &

Re: Changes in Beam Jenkins Agents

t; How does one invoke this? > > Kenn > > On Thu, Apr 4, 2019 at 10:04 AM Yifan Zou wrote: > >> Great! Thank you, Lukasz! >> >> On Thu, Apr 4, 2019 at 3:10 AM Łukasz Gajowy >> wrote: >> >>> I verified load tests and IO performance tests jobs. Looking >&g

Re: beam9 bad worker

> > On Mon, Mar 4, 2019 at 2:06 PM Yifan Zou wrote: > >> I am looking into the error and will disconnect the beam9 to stop >> breaking tests. >> >> On Mon, Mar 4, 2019 at 2:00 PM Pablo Estrada wrote: >> >>> I've talked with Yifan, and I believe he's

Re: beam9 bad worker

I am looking into the error and will disconnect the beam9 to stop breaking tests. On Mon, Mar 4, 2019 at 2:00 PM Pablo Estrada wrote: > I've talked with Yifan, and I believe he's looking into it. : ) > Best > -P. > > On Mon, Mar 4, 2019 at 1:55 PM Ankur Goenka wrote: > >> Beam9 is failing all

Investigating Jenkins Agents OOM

Hi, Some of you are aware of our Jenkins nodes fell in trouble very often. We're now getting the beam4, 7 and 13 disconnected and jobs failed due to the similar reason: OOM. I am asking ASF Infra to dump the usage of resources (memory, disk, cpu...) from those machines before we reboot. In the

Re: Jenkins slowness

I saw many non-beam jobs are running on our nodes. It probably be one of the reasons which caused long waiting time. e.g. https://builds.apache.org/computer/beam13/builds On Thu, Feb 7, 2019 at 12:24 PM Udi Meiri wrote: > There is also excessive python test logging tracked here: >

Re: FYI: beam11 bad worker

The beam11 is temporarily offline. Jobs won't be assigned to it at this time. We'll reconfigure the VM and bring it back later. On Mon, Feb 11, 2019 at 10:05 AM Mikhail Gryzykhin wrote: > Hi everyone, > > Small update: > We have a bad jenkins executor that fails all builds. You might experience

Updates on Beam Jenkins

beam.apache.org%3E Thanks. Regards. Yifan Zou

Re: Updates on Beam Jenkins

for nodes. > > I imagine that once we move to dockerizing the testing environment, it > will be much simpler to restart machines that are having trouble? > Thanks again! > -P. > > On Tue, Apr 9, 2019 at 2:23 PM Yifan Zou wrote: > >> Hello, >> >> I have some go

Re: Insufficient CPU quota in apache-beam-testing causes test flakes

We recently created 16 compute instances for the Jenkins. Each one of them has 16 CPUs, means they consume 256 CPU in total. I guess that is why the CPU usage in us-central1 remains high. We're working on the migrating the rest of old Jenkins agents, and the old instances will be removed once

Re: SNAPSHOTS have not been updated since february

and pushes snapshot to maven every day. Current failure is >> like, the jenkin machine cannot publish artifacts into maven owing to some >> weird permission issue. I think +Yifan Zou is >> working on it actively. >> >> On Thu, Apr 18, 2019 at 9:44 AM Ismaël Mejía wrote:

Re: [ANNOUNCE] New PMC Member: Pablo Estrada

Congratulations, Pablo! *From: *Maximilian Michels *Date: *Wed, May 15, 2019 at 2:06 AM *To: * Congrats Pablo! Thank you for your help to grow the Beam community! > > On 15.05.19 10:33, Tim Robertson wrote: > > Congratulations Pablo > > > > On Wed, May 15, 2019 at 10:22 AM Ismaël Mejía >

Re: Jenkins commenting on PRs again

I've asked Infra and they are not sure as well. *From: *Lukasz Cwik *Date: *Tue, May 14, 2019 at 10:09 AM *To: *dev I have seen this in the past, I don't remember how it was resolved. > > Kenn is specifically asking about seeing messages: > asfgit commented 5 minutes

Re: [ANNOUNCE] New committer announcement: Udi Meiri

Thanks Udi and congratulations! On Fri, May 3, 2019 at 2:47 PM Robin Qiu wrote: > Congratulations Udi!!! > > *From: *Ruoyun Huang > *Date: *Fri, May 3, 2019 at 2:39 PM > *To: * > > Congratulations Udi! >> >> On Fri, May 3, 2019 at 2:30 PM Ahmet Altay wrote: >> >>> Congratulations, Udi! >>>

Re: Beam Dependency Check Report (2019-05-06)

The JIRA credentials are missing on the Jenkins. I'm on it. *From: *Apache Jenkins Server *Date: *Mon, May 6, 2019 at 5:06 AM *To: * ERROR: File 'src/build/dependencyUpdates/beam-dependency-check-report.html' > does not exist

Re: Congrats to Beam's first 6 Google Open Source Peer Bonus recipients!

Congratulations! Well deserved! On Thu, May 2, 2019 at 9:37 AM Rui Wang wrote: > Congratulations! > > > -Rui > > On Thu, May 2, 2019 at 8:23 AM Michael Luckey wrote: > >> Congrats! Well deserved! >> >> On Thu, May 2, 2019 at 3:29 PM Alexey Romanenko >> wrote: >> >>> Congrats! >>> >>> On 2 May

Re: apache-beam-jenkins-15 out of disk

Something were eating the disk. Disconnected the worker so jobs could be allocated to other nodes. Will look deeper. Filesystem Size Used Avail Use% Mounted on /dev/sda1 485G 485G 96K 100% / On Thu, Jun 27, 2019 at 10:54 AM Yifan Zou wrote: > I'm on it. > > On Thu, Jun

Re: apache-beam-jenkins-15 out of disk

images and the beam15 is back now. Opened https://issues.apache.org/jira/browse/BEAM-7650. Ankur, I assigned the issue to you. Feel free to reassign it if needed. Thank you. Yifan On Thu, Jun 27, 2019 at 11:29 AM Yifan Zou wrote: > Something were eating the disk. Disconnected the worker so j

Re: apache-beam-jenkins-15 out of disk

; one >>>> after another then the 2nd one will replace the 1st one and cause >>>> flakyness. >>>> >>>> Is their a way to dynamically create and destroy docker repository on a >>>> machine and clean all the relevant data? >>>>

Re: apache-beam-jenkins-15 out of disk

I reimaged the beam15. The worker is re-enabled. Let us know if anything weird happens on any agent. Thanks. Yifan On Mon, Jul 1, 2019 at 10:00 AM Yifan Zou wrote: > https://issues.apache.org/jira/browse/BEAM-7650 tracks the docker issue. > > On Sun, Jun 30, 2019 at 2:35 PM Mark L

Re: apache-beam-jenkins-15 out of disk

same problem will happen in the future. Is there a jira tracking > this issue? > > On Fri, Jun 28, 2019 at 4:56 PM Yifan Zou wrote: > >> Sorry for the inconvenience. I disabled the worker. I'll need more time >> to restore it. >> >> On Fri, Jun 28, 2019 at 3:56

Re: SNAPSHOTS have not been updated since february

hot/421/>. I'll move the job assignment back once our nodes are fully setup. Thanks Yifan On Fri, Apr 19, 2019 at 2:05 AM Ismaël Mejía wrote: > Thanks everyone for the quick answer and thanks Yifan for taking care. > > On Thu, Apr 18, 2019 at 7:15 PM Yifan Zou wrote: > > >

Re: SNAPSHOTS have not been updated since february

g working as expected +1 > > I wonder still if we may need some extra verification task to detect > when those have not been updated (which can happen for both INFRA or > other reasons). > > On Tue, Apr 23, 2019 at 4:44 AM Yifan Zou wrote: > > > > The Infra is reviewing

Re: Hello from Hannah Jiang

Welcome！ On Thu, Apr 25, 2019 at 7:34 PM Connell O'Callaghan wrote: > Welcome Hannah!!! > > On Thu, Apr 25, 2019, 5:42 PM Reza Rokni wrote: > >> Welcome! >> >> On Fri, 26 Apr 2019 at 04:36, Hannah Jiang >> wrote: >> >>> Thanks Cyrus! >>> >>> On Thu, Apr 25, 2019 at 1:34 PM Cyrus Maden wrote:

Re: [ANNOUNCE] New committer: Robert Burke

Congratulations!! On Tue, Jul 16, 2019 at 10:34 AM Rui Wang wrote: > Congrats! > > > -Rui > > On Tue, Jul 16, 2019 at 10:32 AM Udi Meiri wrote: > >> Congrats Robert B.! >> >> On Tue, Jul 16, 2019 at 10:23 AM Ahmet Altay wrote: >> >>> Hi, >>> >>> Please join me and the rest of the Beam PMC in

Re: apache-beam-jenkins-15 out of disk

Jun 28, 2019 at 12:33 AM Yifan Zou wrote: > >> I did the prune on beam15. The disk was free but all jobs fails with >> other weird problems. Looks like docker prune overkills, but I don't have >> evidence. Will look further in AM. >> >> On Thu, Jun 27, 2019 at 11:20

Re: [Update] Beam 2.15 Release Progress

All blockers resolved and cherry-picks are done. Will start building the release candidates. On Wed, Aug 7, 2019 at 2:58 PM Yifan Zou wrote: > Thanks Udi. > > On Wed, Aug 7, 2019 at 2:58 PM Udi Meiri wrote: > >> https://github.com/apache/beam/pull/9240 has been merged >>

Re: [VOTE] Release 2.15.0, release candidate #1

like fix version was changed to 2.16.0) > > Thanks, > Cham > > On Wed, Aug 14, 2019 at 1:42 PM Yifan Zou wrote: > >> Hi everyone, >> Please review and vote on the release candidate #1 for the version >> 2.15.0, as follows: >> [ ] +1, Approve the release &g

[VOTE] Release 2.15.0, release candidate #1

is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Yifan Zou [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12345489 [2] https://dist.apache.org/repos/dist/dev/beam/2.15.0/ [3] https://dist.apache.org/repos/dist/release/beam/KEYS

Re: [VOTE] Release 2.15.0, release candidate #2

Mark and Kyle found a py35 portable test which is flaky: https://issues.apache.org/jira/browse/BEAM-7993. I plan to finalize the release this week. Would that be a blocker? Could we include the fix in 2.16? Thanks. On Mon, Aug 19, 2019 at 4:38 PM Yifan Zou wrote: > I've run m

Re: [VOTE] Release 2.15.0, release candidate #2

>>> From: Ahmet Altay >>> Date: Mon, Aug 19, 2019 at 10:27 AM >>> Subject: Re: [VOTE] Release 2.15.0, release candidate #2 >>> To: dev >>> >>> >>> +1, verified a few python workloads and updated the validation sheet. >>> &g

[RESULT] [VOTE] Release 2.15.0, release candidate #2

Hi all, I'm happy to announce that we have unanimously approved this release. There are 4 approving votes, 3 of which are binding (in order): * Ahmet (al...@google.com); * Pablo (pabl...@google.com); * Lukasz (lc...@google.com); There are no disapproving votes. Thanks everyone! Next step is

Re: How to test a new precommit in PR?

Run the seed job then trigger your tests by using phrase. On Tue, Aug 20, 2019 at 2:39 PM Rui Wang wrote: > Hi Community, > > I am trying to add a new precommit task (see [1] and [2]), and the PR is > pending. Does anyone know how to test the added precommit directly in the > PR before merging

Re: [VOTE] Release 2.15.0, release candidate #2

normal (but not forever) to start on Jenkins, in which case this >>>>> issue >>>>> is nothing new to this release. (The halting problem!) If it's the second, >>>>> it's safe to wait to fix it in the next release. >>>>> >>>&g

Re: [ANNOUNCE] New committer: Valentyn Tymofieiev

Congratulations, Valentyn! Well deserved! On Mon, Aug 26, 2019 at 3:31 PM Aizhamal Nurmamat kyzy wrote: > Congratulations! and thank you for your contributions, Valentyn! > > On Mon, Aug 26, 2019 at 3:26 PM Thomas Weise wrote: > >> Congrats! >> >> >> On Mon, Aug 26, 2019 at 3:22 PM Heejong Lee

[ANNOUNCE] Beam 2.15.0 Released!

. Yifan Zou

Re: [Update] Beam 2.15 Release Progress

out which one and cherry-pick it into the release. >> >> Regards, >> Anton >> >> On Tue, Aug 6, 2019 at 4:52 PM Yifan Zou wrote: >> >>> Hi, >>> >>> There is a perf regression on SQL Query3 on dataflow runner. This was >>

  1   2   >