Greetings from Tyson

Hello Beam Community, This is just a simple 'Hello' to introduce myself. I'm a Software Engineer at Google and have worked with data processing languages and runtime systems on and off during my career. I now have the pleasure of dedicating more time towards working with you lovely folks on

Re: contributor permission for Beam Jira tickets

Hi Darshan, I'll loop in some people for a discussion. On 2020/04/30 04:29:13, Darshan Jani wrote: > Thanks Luke. > > Thanks for pointing that out. > I am new to Beam contribution community. > I would appreciate if you can point me or tag someone of the contributing > members who are working

Re: JIRA priorities explaination

Proposal sounds good to me! The tool tips will be fantastic. On Fri, May 1, 2020 at 3:03 PM Robert Bradshaw wrote: > On Fri, May 1, 2020 at 2:34 PM Kenneth Knowles wrote: > > > > Coming back to this thread (again!) > > > > I wrote up https://beam.apache.org/contribute/jira-priorities/ and >

Re: Automation for Jira

+1 for automation. Regarding #4, what about adding the constraint that this rule only applies to issues that are incomplete and require more information from the reporter? Unfortunately it would require a human to triage issues to determine this and apply an appropriate label. Triage should

Re: New Grafana dashboards

The dashboards look great! Thank you. It would be nice if the 'Useful Links' section included links to Apache Beam related material like the cwiki documentation. On Wed, May 13, 2020 at 4:50 AM Kamil Wasilewski < kamil.wasilew...@polidea.com> wrote: > Hello everyone, > > I'm pleased to announce

Re: Event Calendar?

+1 a calendar would be nice. On Tue, May 19, 2020 at 3:51 PM Austin Bennett wrote: > Hi All, > > As we have events more often that are more accessible (digital), wondering > whether others see a value of adding a calendar to the website? > > Perhaps related, is it worth updating >

Re: [ANNOUNCE] New committer: Robin Qiu

Congratulations! On Tue, May 19, 2020 at 6:10 AM Omar Ismail wrote: > Congrats! > > On Tue, May 19, 2020 at 5:00 AM Gleb Kanterov wrote: > >> Congratulations! >> >> On Tue, May 19, 2020 at 7:31 AM Aizhamal Nurmamat kyzy < >> aizha...@apache.org> wrote: >> >>> Congratulations, Robin! Thank you

Code Coverage Tracking

Hello, I noticed on the github page, there is a badge that reports 100% coverage. This seems suspect, and sure enough after a couple clicks coverall shows that only one file is tracked. A more interesting page is the Builds page [1] that shows the impact of specific PRs. It would be really

Apache Beam Fixit 2020-06-08

Why Google has a Fixit week to improve Engineering Excellence and we want to use it to improve Apache Beam! What For the week of 2020-06-08 you will see an increased focus from Google engineers on improving Engineering Excellence. We’re asking the community to join in and commit as much, or as

Re: joining Beam JIRA

Hey Ted! Welcome =) On Thu, May 21, 2020 at 10:34 AM Ahmet Altay wrote: > Done. And welcome! > > On Thu, May 21, 2020 at 10:32 AM Ted Romer wrote: > >> Hi, this is Ted Romer at Google ... I'd like to get permission to join >> the JIRA project so that I can assign >>

Re: Infra Improvements

t; I'm happy to create an "infra" component but is there a synonym, or a >> small collection of other terms, we could use? >> >> Kenn >> >> On Fri, Sep 11, 2020 at 3:39 PM Tyson Hamilton >> wrote: >> >>> >>> >>

Re: [External] Re: DISCUSS: Sorted MapState API

Hi Reuven, I noticed that there was an implementation of the in-memory OrderedListState introduced [1]. Where can I find out more regarding the plan and design? Is there a design doc? I'd like to know more details about the implementation to see if it fits my use case. I was hoping it would

Re: Infra Improvements

ions, Jenkins inventory jobs, email reports, artifact publishing, etc. so IMO having a general 'infra' would be handy. Another option is to have dedicated components for each of these things but the proliferation may become a problem. > > On Fri, Sep 11, 2020 at 3:11 PM Tyson Hamilto

Infra Improvements

Hi All, Below is a summary of the improvements made to Beam’s infrastructure in the last few months. The completion of this work is thanks to Damian (damgad) and Tobiasz (TobKed) from Polidea. Thank you!!! - Jenkins tests are not always triggered / Switch to new jenkins infra -

Re: Jenkins CI down

In the future, if this happens again, please create an Apache INFRA ticket like the following and update the dev list: https://issues.apache.org/jira/browse/INFRA-20954 On Wed, Oct 7, 2020 at 2:20 AM Ismaël Mejía wrote: > Yes seems to have been broken for some time, but working now. Thanks. >

Re: Behavior change for Gradle build target

Hm. It sounds like it is possibly related to the gradle upgrade to 6? I had a similar issue that I fixed for the Java nightly snapshot build in PR#12947 (https://github.com/apache/beam/pull/12947). It may require similar changes to the gradle.build file to generate the jar. On Tue, Sep 29, 2020

Re: Behavior change for Gradle build target

tributions/expansion-service-2.26.0-SNAPSHOT.tar ./sdks/java/io/expansion-service/build/distributions/expansion-service-2.26.0-SNAPSHOT.zip On Tue, Sep 29, 2020 at 12:59 PM Tyson Hamilton wrote: > Hm. It sounds like it is possibly related to the gradle upgrade to 6? I > had a similar is

Re: Shutting down Perfkit Explorer

m_PostCommit_Python36/2959/ >> >> >> >> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski < >> kamil.wasilew...@polidea.com> wrote: >> >>> Thanks. The application has been disabled. >>> >>> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay wr

Re: [DISCUSS] Move Avro dependency out of core Beam

On Mon, Sep 21, 2020 at 7:56 AM Cristian Constantinescu wrote: > All the proposed solutions seem reasonable. I'm not sure if one has an > edge over the other. I guess it depends on how cautiously the community > would like to move. > > Maybe it's just my impression, but it seems to me that there

Re: Code Coverage Tracking

Huh.. I just realized it is closed already, now I'm confused. On 2020/05/26 15:08:21, Tyson Hamilton wrote: > Yes, this would be a good choice for the fixit. The tracking issue exists > already too: > > https://issues.apache.org/jira/browse/BEAM-1132 > > On 2020/05/26

Re: Code Coverage Tracking

e coverage data is per module. You > will need to gather it together to get a holistic view. There are some > gradle plugins to do this. > > All of the above would be a good choice for the fixit week, perhaps? > > Kenn > > On Wed, May 20, 2020 at 9:41 AM Tyson Hamilton wrote

Re: [DISCUSS] Dealing with @Ignored tests

A lot of these are good fixit tasks (shameless plug [1]), * archeology on the @Ignored tests to normalize them with proper labelling * checkstyle validation for @Ignored tests * spring cleaning on open JIRA issues Having a Jira issue for the first 2 of these and adding the beam-fixit label

Re: [External] Re: DISCUSS: Sorted MapState API

it your use case? If not, I wonder if MapState is closer to > what you are looking for? > > Reuven > > On Tue, Sep 15, 2020 at 2:33 PM Tyson Hamilton wrote: > >> Hi Reuven, >> >> I noticed that there was an implementation of the in-memory >> OrderedList

Re: Shutting down Perfkit Explorer

+1 to removing, thank you Kamil. On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski < kamil.wasilew...@polidea.com> wrote: > Hello everyone, > > Beam support for Python 2 is coming to an end. Consequently, we should > make sure no Python 2 applications are running as a part of Beam's >

Re: Website Revamp Update - Week 1 (and how to get involved)

Hello, I was unable to attend the sprint meeting but had a couple comments. - The Beam mascot isn't on the page anywhere, is this intentional? It would be fun to include it somewhere, it's so cute and fast. - The Beam logo doesn't appear until the footer. Maybe there should be one higher

Re: Problem with updating Hadoop and Hcatalog dependencies

On 2020/10/22 07:35:47, Piotr Szuberski wrote: > I'm trying to update Hadoop dependencies to the recent 3.3.0 and I've > encountered a problem - the Hadoop related checks seem to work without any > further changes but Hcatalog requires to be bumped to 3.x.y versions as well > (2.x.y

Re: Java Jenkins tests not running

Yes, there is a backup of ~14 builds in queue. On Fri, Aug 7, 2020 at 10:54 AM Andrew Pilloud wrote: > Now it is going, just slow to start. There must be a backlog built up from > the outage. > > On Fri, Aug 7, 2020 at 10:44 AM Andrew Pilloud > wrote: > >> It appears Jenkins is not working

Re: Git commit history: "fixup" commits

+1 to automating this! On Wed, Aug 5, 2020 at 8:38 AM Alexey Romanenko wrote: > Thanks Robert and Udi for links. Perhaps we should integrate such check if > there are no objections from community. I’ll take a look on this more > closely. > > On 4 Aug 2020, at 19:31, Udi Meiri wrote: > >

Re: Chronically flaky tests

is is something we should address. With the new jira >>>> automation at least assignees should get an email notification after 30 >>>> days because of a jira comment like [1], but that's too long to let a test >>>> continue to flake. Could Beam Jira Bot ping every N

Re: Failing Python builds & AppEngine application

It was me! I disabled the App in the premise that it only hosted the old perf graphs that were replaced with Grafana. Thanks for fixing the issue. Is there anything else on the app, or is there more migration to Grafana required, or cleanup unfinished? On Thu, Aug 6, 2020, 4:30 AM Damian

Re: [ANNOUNCE] New committer: Aizhamal Nurmamat kyzy

Congrats! On Thu, Jul 2, 2020 at 5:15 AM David Sabater Dinter wrote: > Congratulations Aizhamal! > > On Thu, 2 Jul 2020 at 09:53, Katarzyna Kucharczyk > wrote: > >> Congratulations Aizhamal! :) >> >> On Wed, Jul 1, 2020 at 5:29 AM Thomas Weise wrote: >> >>> Congrats! >>> >>> >>> On Tue, Jun

Re: Is there an easy way to figure out why my build failed?

Is there a reason we don't always run with the --stacktrace flag? Having this enabled all the time could be useful and wouldn't add any additional output for successful runs (right?). On Tue, Jun 30, 2020 at 1:23 PM Luke Cwik wrote: > This rabbitmq:test stuckness has been a common pain point

Re: Canceling Jenkins builds when the update to PR makes prior build irrelevant

e to run the java >>>>>> jobs >>>>>> by trigger phrase if needed. There are also performance tests (Nexmark >>>>>> for >>>>>> example) that have different jobs to ensure PR runs don't end up >>>>>> published &

Re: Versioning published Java containers

What do we consider 'ready'? Maybe the only required outstanding bugs are supporting the direct runner (BEAM-10085), core tests (BEAM-10081), IO tests (BEAM-10084) to start with? Notably this would exclude failing tests like those for GCP core, GCPIOs, Dataflow runner, Spark runner, Flink

Re:

Welcome! On Fri, Jul 10, 2020, 10:38 AM Rui Wang wrote: > Welcome! > > > -Rui > > On Fri, Jul 10, 2020 at 10:33 AM Kenneth Knowles wrote: > >> Welcome to dev@ ! >> >> On Fri, Jul 10, 2020 at 2:14 AM Maximilian Michels >> wrote: >> >>> Welcome Emily! Looking forward to your questions. >>> >>>

Re: Jira Board to track most Beam's tickets

This is very cool! I love the bubble visualization. Is there a way to have something similar for resolved issues? I'd also be interested in seeing other breakdowns of open/resolved issues, like burndowns for example. On Tue, Jun 16, 2020 at 3:34 PM Griselda Cuevas wrote: > Hi folks, > > I made

Re: Canceling Jenkins builds when the update to PR makes prior build irrelevant

+1 the ability to cancel in-flight jobs is worth deduplicating _Phrase and _Commit. I don't see a benefit for having both. On Tue, Jun 23, 2020 at 9:02 AM Luke Cwik wrote: > I think this is a great improvement to prevent the Jenkins queue from > growing too large and has been suggested in the

Re: Composable DoFn IOs Connection Reuse

Nice doc by the way, it's concise. Thanks for sharing and I'm excited to see this feature, particularly the PCollection variant that would have been useful for the Cloud AI transforms recently introduced. On Fri, Jun 26, 2020 at 3:25 PM Tyson Hamilton wrote: > > > On Fri, Jun 26, 2020

Re: Composable DoFn IOs Connection Reuse

On Fri, Jun 26, 2020 at 12:24 PM Siyuan Chen wrote: > Thanks Luke! > > Hi, I'm Siyuan and I'm working on the Google Dataflow team. We are faced > with similar issues with some sinks commonly used by Dataflow such as the > streaming BigQuery sink. Basically the elements are grouped and batched so

Re: [ANNOUNCE] New PMC Member: Alexey Romanenko

Congrats! On Tue, Jun 16, 2020 at 9:40 AM Udi Meiri wrote: > Congratulations! > > On Tue, Jun 16, 2020 at 9:33 AM rahul patwari > wrote: > >> Congrats Alexey! >> >> On Tue, Jun 16, 2020 at 9:27 PM Ismaël Mejía wrote: >> >>> Please join me and the rest of Beam PMC in welcoming Alexey Romanenko

Re: Beam Jenkins Migration

Currently no. We're already experiencing a backlog of builds so the additional load would be a problem. I've opened two related issues that I think need completion before allowing non-committers to trigger tests: Load sharing improvements: https://issues.apache.org/jira/browse/BEAM-10281 Admin

cwiki edit access

Hello, Would someone kindly provide me (tysonjh) with edit access to the wiki? I don't have any updates planned now but as I go through pages I'd like the ability to fix broken links, add information, or update stale content. Thanks, -Tyson

Re: Remove EOL'd Runners

o me. > >> > >> On Tue, Jun 9, 2020 at 8:45 AM Maximilian Michels wrote: > >> > >>> Thanks of the heads-up, Tyson! It's a sensible decision to remove > >>> unsupported runners. > >>> > >>> -Max > >>>

Re: Remove EOL'd Runners

Weise wrote: >>>> >>>>> +1 >>>>> >>>>> >>>>> On Tue, Jun 9, 2020 at 9:41 AM Robert Bradshaw >>>>> wrote: >>>>> >>>>>> Makes sense to me. >>>>>> >>>

Re: Difficulties with triggering PRs Jenkins jobs

There is an active effort in coordination with the Apache Infrastructure team to move Beam to a new dedicated Jenkins cluster. This will resolve the issues described and add some new functionality as well within the next month. On Mon, Jun 8, 2020 at 6:29 AM Alexey Romanenko wrote: > Yes, the

Remove EOL'd Runners

Hi All, As part of the Fixit [1] I'd like to remove EOL'd runners, Apex and Gearpump, as described in BEAM- [2]. This will be a big PR I think and didn't want anyone to be surprised. There is already some agreement in the linked Jira issue. If there are no objections I'll get started later

Re: Beam Jenkins Migration

Very exciting! Thanks for the advanced notice Damian. On Fri, Jun 12, 2020 at 7:58 AM Damian Gadomski wrote: > Hello, > > During the last few days, I was preparing for the Beam Jenkins migration > from builds.apache.org to ci-beam.apache.org. The new Jenkins Master will > be dedicated only for

Re: No space left on device - beam-jenkins 1 and 7

11:57 AM Tyson Hamilton > wrote: > >> Ah I see, thanks Kenn. I found some advice from the Apache infra wiki >> that also suggests using a tmpdir inside the workspace [1]: >> >> Procedures Projects can take to clean up disk space >> >> Projects can help th

Re: No space left on device - beam-jenkins 1 and 7

Bleck. I just realized that it is 'offline' so that won't work. I'll clean up manually on the machine using the cron command. On Fri, Jul 24, 2020 at 10:56 AM Tyson Hamilton wrote: > Something isn't working with the current set up because node 15 appears to > be out of space and is cur

Re: No space left on device - beam-jenkins 1 and 7

Everything I've been looking at is in the /tmp dir. Where are the workspaces, or what are the named? On Fri, Jul 24, 2020 at 11:03 AM Udi Meiri wrote: > I'm curious to what you find. Was it /tmp or the workspaces using up the > space? > > On Fri, Jul 24, 2020 at 10:57 AM Ty

Re: No space left on device - beam-jenkins 1 and 7

:19 AM Tyson Hamilton wrote: > Everything I've been looking at is in the /tmp dir. Where are the > workspaces, or what are the named? > > > > > On Fri, Jul 24, 2020 at 11:03 AM Udi Meiri wrote: > >> I'm curious to what you find. Was it /tmp or the workspaces using

Re: No space left on device - beam-jenkins 1 and 7

2326 which could >> help by cleaning up workspaces after a run (just started a seed job). >> >> On Fri, Jul 24, 2020 at 12:13 PM Tyson Hamilton >> wrote: >> >>> 664Mbeam_PreCommit_JavaPortabilityApi_Commit >>> 656Mbeam_PreCommit_JavaPortabi

Re: No space left on device - beam-jenkins 1 and 7

beam_SQLBigQueryIO_Batch_Performance_Test_Java 175Gtotal On Fri, Jul 24, 2020 at 12:04 PM Tyson Hamilton wrote: > Ya looks like something in the workspaces is taking up room: > > @apache-ci-beam-jenkins-8:/home/jenkins$ sudo du -shc . > 191G. > 191Gtotal > > &

Re: No space left on device - beam-jenkins 1 and 7

Ya looks like something in the workspaces is taking up room: @apache-ci-beam-jenkins-8:/home/jenkins$ sudo du -shc . 191G. 191Gtotal On Fri, Jul 24, 2020 at 11:44 AM Tyson Hamilton wrote: > Node 8 is also full. The partition that /tmp is on is here: > > Filesystem S

Re: Chronically flaky tests

Adding retries can be beneficial in two ways, unblocking a PR, and collecting metrics about the flakes. If we also had a flaky test leaderboard that showed which tests are the most flaky, then we could take action on them. Encouraging someone from the community to fix the flaky test is another

Re: No space left on device - beam-jenkins 1 and 7

FYI there was a job introduced to do this in Jenkins: beam_Clean_tmp_directory Currently it needs to be run manually. I'm seeing some out of disk related errors in precommit tests currently, perhaps we should schedule this job with cron? On 2020/03/11 19:31:13, Heejong Lee wrote: > Still

Re: Monitoring performance for releases

Max, The runner dimension are present when hovering over a particular graph. For some more info, the load test configurations can be found here [1]. I didn't get a chance to look into them but there are tests for all the runners there, possibly not for every loadtest. [1]:

Re: No space left on device - beam-jenkins 1 and 7

(assuming they're already doing it?) > >> > >> On Fri, Jul 24, 2020 at 12:31 PM Udi Meiri wrote: > >> > >>> The additional slots (@3 directories) take up even more space now than > >>> before. > >>> > >

Re: No space left on device - beam-jenkins 1 and 7

gt; https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Inventory.groovy#L51). >>> That way, we can see all the cron jobs as part of the source tree, adjust >>> frequencies and clean up codes with PRs. I do not know how internal cron >>> scripts are cre

Re: No space left on device - beam-jenkins 1 and 7

Mon, Jul 20, 2020 at 7:55 PM Tyson Hamilton wrote: > >> Jobs are hermetic as far as I can tell and use unique subdirectories >> inside of /tmp. Here is a quick look into two examples: >> >> @apache-ci-beam-jenkins-4:/tmp$ sudo du -ah --time . | sort -rhk 1,1 | &g

Re: [VOTE] Release 2.26.0, release candidate #1

+1 from me. I validated Nexmark performance tests. On Tue, Dec 8, 2020 at 7:53 PM Robert Burke wrote: > I'm +1 on RC1 based on the 7 tests I know I can check successfully. I'll > be trying more tomorrow, but remember that release validation requires the > community to validate it meets our

Re: Proposal: ToStringFn

Can you open up comment access please? On Wed, Oct 28, 2020 at 3:40 PM Sam Rohde wrote: > +Lukasz Cwik > > On Tue, Oct 27, 2020 at 12:04 PM Sam Rohde wrote: > >> Hi All, >> >> I'm working on a project in Dataflow that requires the runner to >> translate an element to a human-readable form. To

Re: Jenkins Java Version Confusion/Failures

html/rb4c2834b9874b9f4a74c528de9055958483d2bc6e62c3464bc5c053f%40%3Cbuilds.apache.org%3E [2]: https://issues.apache.org/jira/browse/INFRA-20858 On 2020/10/28 18:24:48, Tyson Hamilton wrote: > Hi All, > > This is an early notice that I'm seeing an issue in Jenkins related to JDK > versions. I've created an INFRA issue [1] to get sup

Re: Updating elasticsearch version to 7.9.2 - problem with HadoopFormatIOElasticTest that uses ES emulator

IMO it really comes down to stability & runtime differences. If there are no significant changes to either of these then keeping it as a precommit and using test containers is fine. Where are the assumptions in the IT test, in HadoopFormatIOElasticTest? On Mon, Oct 12, 2020 at 10:10 AM Piotr

Re: [VOTE] Release 2.25.0, release candidate #2

+1 I went through the Nexmark queries and validated the results. On Thu, Oct 22, 2020 at 4:43 PM Pablo Estrada wrote: > +1 (binding) > Validated Java quickstart for Direct, Dataflow, Spark runners. > Tried out a few interactive queries on InteractiveRunner on Ipython. > Best > -P. > > On Thu,

Re: [ANNOUNCE] New PMC Member: Chamikara Jayalath

Woo! Congrats Cham! On Thu, Jan 21, 2021 at 3:02 PM Robert Burke wrote: > Congratulations! That's fantastic news. > > On Thu, Jan 21, 2021, 2:59 PM Reza Rokni wrote: > >> Congratulations! >> >> On Fri, Jan 22, 2021 at 6:58 AM Ankur Goenka wrote: >> >>> Congrats Cham! >>> >>> On Thu, Jan 21,

Re: [ANNOUNCE] New committer: Piotr Szuberski

Congrats Piotr! Well deserved. On Fri, Jan 22, 2021 at 9:18 AM Ismaël Mejía wrote: > Congratulations Piotr ! Thanks for all your work ! > > On Fri, Jan 22, 2021 at 5:33 PM Alexey Romanenko > wrote: > > > > Hi everyone, > > > > Please join me and the rest of the Beam PMC in welcoming a new >

Re: Problems building latest beam source

I saw this issue on another OSS repo ( https://github.com/apache/hudi/issues/2479), they just removed the spring.io repository but I'm not sure where their dep would come from then. We could swap the spring.io repo for the pentaho nexus one public.nexus.pentaho.org (here probably in Beam

Re: FileIO with custom sharding function

Adding sharding to the model may require a wider discussion than FileIO alone. I'm not entirely sure how wide, or if this has been proposed before, but IMO it warrants a design doc or proposal. A couple high level questions I can think of are, - What runners support sharding? * There will

Re: [Proposal] Go SDK Exits Experimental

WOW! Big news. I'm supportive of leaving experimental status after Go Modules are completed and the LICENSE issue is resolved. I don't think that lacking streaming support is a blocker. The other thing I checked to see was if there were metrics available on metrics.beam.apache.org, specifically

Re: pb2 file generation

+Alex Amato may be able to help with this. On Mon, May 17, 2021 at 7:35 AM Miguel Hernández Sandoval < rogelio.hernan...@wizeline.com> wrote: > Hi team, > I am working on BEAM-11984[1]. I want to add some labels to > metrics_pb2_urns.py. However, I don't know the process to generate/update >

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

I'd like to help, but I don't know how to determine whether this upgrade is going to cause problems or not. Are there tests I should look at, or some validation I should perform? On Wed, May 19, 2021 at 11:29 AM Ismaël Mejía wrote: > Kind reminder, the vote is ongoing > > On Mon, May 17, 2021

Re: BEAM-8787 Contribution Guide and WIKI Improvements

Thanks for the update! On Tue, May 25, 2021 at 3:50 PM Josias Rico García wrote: > Hello, > > As a complement to the BEAM-8787 issue, I updated some of the pages at the > Developer > Guides > wiki. > You can see that most of

Re: Flake trends - better?

There is a cleanup in progress right now by the folks at Wizeline for the following query [1]. They're going to check and see which bugs are obsolete or still a problem and fix some of them as well. [1]: project="BEAM" AND status="OPEN" AND labels in (flake, currently-failing) Link:

Re: [ANNOUNCE] New committer: Yichi Zhang

Congrats! On Wed, Apr 21, 2021 at 4:37 PM Valentyn Tymofieiev wrote: > Well deserved and congrats, Yichi! > > On Wed, Apr 21, 2021 at 4:23 PM Pablo Estrada wrote: > >> Hi all, >> >> Please join me and the rest of the Beam PMC in welcoming a new committer: >> Yichi Zhang >> >> Yichi has been

Re: Builds Meeting this Thursday

d workloads in these VMs. As far as I >>> remember, we did not have this problem with our jenkins machines and anyone >>> being able to run code with their PRs. And Airflow has the suggestion of >>> use preemptible machines. We can do the same and these machines are always &g

Re: [ANNOUNCE] New committer: Tomo Suzuki

Congrats! On Fri, Apr 2, 2021 at 11:02 AM Pablo Estrada wrote: > Thank you Tomo! And congrats : ) > > On Fri, Apr 2, 2021 at 10:24 AM Robert Bradshaw > wrote: > >> Congratulations! >> >> On Fri, Apr 2, 2021 at 10:19 AM Chamikara Jayalath >> wrote: >> >>> Congrats Tomo! >>> >>> On Fri, Apr 2,

Re: Please help triage issues!

I'm guilty of creating issues and not moving them to 'open'. I'll do better to move them to open in the future. To recompense I will spend some additional time triaging =) Thanks for the review of the flow. On Thu, Mar 11, 2021 at 12:39 PM Kenneth Knowles wrote: > Hi all, > > You may or may

Re: Please help triage issues!

at don't need attention, but to be able to notice things that do >>> need attention. I did a few manually too, and the components, issue type, >>> and priority very often need fixing up. I especially want to get untriaged >>> P0s and P1s to zero. >>> >>> Kenn

Re: Adding flaky postcommit test suite

The Apache Airflow project has some interesting automation around flaky tests. They annotate such flaky tests as 'quarantined', those quarantined tests still run (maybe even with retries?) but won't fail a test suite. Quarantined tests are run in a separate scheduled job, when they start passing,

Re: Random outputs for ARRAY_CONCAT_AGG fn zetasql

I didn't find anything like that after a brief look. What you could do instead is something like: PAssert.thatSingleton(stream).satisfies( row -> assertThat("array_field containsInAnyOrder", row.getArray("array_field"), containsInAnyOrder(Arrays.asList(...))); using junit/hamcrest matchers. I

Re: Long term support versions of Beam Java

Does Apache Beam commit to the LTS, or is this just "another release" as far as Apache Beam is concerned? Or phrased differently, since it is Google (in this scenario) that wants to support a version with back-fixes to satisfy their customers, would the onus be on Google to propose/perform those

Re: Naming! Dataflow Worker/SDK "Harness" image flag

I'm in favor of dropping "harness" and going with "sdk_container_image". I don't feel like the word "harness" adds value or clarity. On Mon, Apr 19, 2021 at 11:34 AM Emily Ye wrote: > Hi all, > > *tl;dr*: keep harness in user-facing container image flag names? > > I have a few PRs in-progress

Re: [VOTE] Release 2.29.0, release candidate #1

+1 (non-binding) I ran the java local quickstarts and verified the nexmark tests. On Fri, Apr 16, 2021 at 1:50 PM Valentyn Tymofieiev wrote: > Hi Kenn, > > There is a version mismatch between Beam version and Dataflow Python > worker version (the latter currently has version 2.28.0.dev), you