Re: Issues and PR names and descriptions (or should we change the contribution guide)

BEAM-12173 is on me. I'm sorry about that. Re-reading committer guide [1], I see I was not following this > The reviewer should give the LGTM and then request that the author of the > pull request rebase, squash, split, etc, the commits, so that the history is > most useful Thank you for the f

Re: [PROPOSAL] Preparing for Beam 2.30.0 release

Thank you for the preparation! > a few responses that some high priority changes Would you be willing to share the items for visibility? On Wed, Apr 21, 2021 at 7:21 PM Kenneth Knowles wrote: > > Also the 2.29.0 was re-cut. > > Usually a delay in one release should not delay the next release,

Re: [ANNOUNCE] New committer: Yichi Zhang

Congratulations! On Wed, Apr 21, 2021 at 7:48 PM Tyson Hamilton wrote: > Congrats! > > On Wed, Apr 21, 2021 at 4:37 PM Valentyn Tymofieiev > wrote: > >> Well deserved and congrats, Yichi! >> >> On Wed, Apr 21, 2021 at 4:23 PM Pablo Estrada wrote: >> >>> Hi all, >>> >>> Please join me and the r

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

ith aarch64 support for > python!) that made me wonder about this, what is the current status of > upgrading the vendored dependency Tomo? > > > On Thu, Apr 8, 2021 at 4:16 PM Tomo Suzuki wrote: > >> We observed the cron job of Java Precommit for the master branch started >

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

adding a timeout to just that method should > save time (but will make it harder to capture stack traces, etc). I've > opened up https://github.com/apache/beam/pull/14781 for that. There may > be a nice way to add a timeout to the executor to capture the hung stack, > but I didn

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

"grpc-default-executor-XXX" threads reappear in the logs when the pause triggers a 5-second timeout set by JUnit. On Tue, May 11, 2021 at 1:12 PM Tomo Suzuki wrote: > Thank you for the advice. Yes, the latch not being counted-down is the > problem. (my memo: > https://g

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

so raised an question to grpc-java on how best to troubleshoot such situation https://github.com/grpc/grpc-java/issues/8174 On Wed, May 12, 2021 at 11:29 PM Tomo Suzuki wrote: > Update: still the root cause of is unknown. > > From my observation with debug logging and thread dump

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

+1 All classes in the JAR are shaded correctly under org/apache/beam/vendor On Wed, May 19, 2021 at 2:54 PM Ismaël Mejía wrote: > This release is only to publish the vendored dependency artifacts. We need > those to integrate it and be able to verify if it causes problems or not. > The PR for th

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

p and follow-up! On Thu, May 13, 2021 at 10:39 AM Tomo Suzuki wrote: > I'm giving up! Can anyone troubleshoot this gRPC concurrency problem > further? > My current view of the problem (link > <https://github.com/apache/beam/pull/14768#issuecomment-840576342>) is > that

Re: LGPL-2.1 in beam-vendor-grpc

y in >>> BEAM-11227 <https://issues.apache.org/jira/browse/BEAM-11227>, if I >>> remember correctly the newer version of gRPC does not require the jboss >>> dependency, so probably is the best upgrade path, can you confirm Tomo >>> Suzuki >>> <h

Re: [VOTE] Release 2.30.0, release candidate #1

+1 (non-binding) Thank you for the preparation. With the GCP dependencies of my interest, the GitHub checks worked. On Thu, Jun 3, 2021 at 4:55 AM Heejong Lee wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 2.30.0, > as follows: > [ ] +1, Approve t

Re: Aliasing Pub/Sub Lite IO in external repo

Hi Daniel, (You helped me apply some change to this strange setup a few months back. Thank you for working on rectifying the situation.) I like that idea overall. Question 1: How are you going to approach testing/CI? The pull requests in the java-pubsublite repo do not trigger Beam repo's CI. You

Re: Java post commit: HL7v2IOReadWriteIT failures

Here is a PR to fix: https://github.com/apache/beam/pull/15498 I confirmed "Run Java Postcommit" passes. On Fri, Sep 10, 2021 at 4:32 PM Tomo Suzuki wrote: > I'm troubleshooting Java postcommit build failure on HL7v2IOReadWriteIT. > > https://issues.apache.or

Completeness of Beam Java Dependency Check Report

Hi Beam developers, (I'm thinking to contribute to upgrades of Java dependencies of Beam; I just read https://beam.apache.org/contribute/dependencies/) As per the weekly report, Apache Beam Java SDK only has 8 outdated dependencies based on the criteria. However, it seems many others are not up-to

Re: Completeness of Beam Java Dependency Check Report

Chamikara and Yifan, Thank you for the responses! Looking forward to hearing the investigation result. In the meantime, I'll explore .test-infra/jenkins/dependency_check directory.

Re: Completeness of Beam Java Dependency Check Report

gt; against 26.0-jre. ZetaSQL against 26.0-android. > > (or maybe I messed up in my analysis) > > Kenn > > On Mon, Nov 11, 2019 at 12:07 PM Tomo Suzuki wrote: > >> >> Chamikara and Yifan, >> Thank you for the responses! Looking forward to hearing the investigation >> result. >> In the meantime, I'll explore .test-infra/jenkins/dependency_check >> directory. >> >> -- Regards, Tomo

Test Failure: GcpOptionsTest$CommonTests. testDefaultGcpTempLocationDoesNotExist

Hi Beam developers, I'm trying to build Apache Beam from the source. But GcpOptionsTest fails (error below). Did anybody solve this problem already? I'm using master (c2e58c55) suztomo@suxtomo24:~/beam4$ ./gradlew -p sdks/java check ... FAILURE: Build failed with an exception. * What went wron

Re: Completeness of Beam Java Dependency Check Report

Yifan, I created a ticket to track this finding: https://issues.apache.org/jira/browse/BEAM-8621 . On Mon, Nov 11, 2019 at 5:08 PM Tomo Suzuki wrote: > Kenn, > > Thank you for the analysis. Although Guava was randomly picked up, it's > great learning for me to learn how y

contributor permission for Beam Jira tickets: suztomo

Hi Beam Devs, This is Tomo from Google New York. I'd like to contribute to Beam Java dependencies upgrade. Can someone add me as a contributor for Beam's JIRA issue tracker? GitHub account: suztomo Apache JIRA username: suztomo -- Regards, Tomo

Re: contributor permission for Beam Jira tickets: suztomo

Thank you so much On Tue, Nov 12, 2019 at 4:28 PM Pablo Estrada wrote: > Hi Tomo! > I've added you as contributor. Welcome! > Best > -P. > > On Tue, Nov 12, 2019 at 11:51 AM Tomo Suzuki wrote: > >> Hi Beam Devs, >> >> This is Tomo from Google N

Re: Test Failure: GcpOptionsTest$CommonTests. testDefaultGcpTempLocationDoesNotExist

rfitting as > a result. Probably we should mock something so that the test doesn't > actually need to call gcloud. > > I have created a JIRA issue for this and assigned to myself: > https://issues.apache.org/jira/browse/BEAM-8628 > > Kyle > > On Tue, Nov 12, 2019

org.apache.beam.sdk.io.clickhouse.AtomicInsertTest.testIdempotentInsert fails

Hi Beam developers, The org.apache.beam.sdk.io.clickhouse.AtomicInsertTest fails in my development environment. Created https://issues.apache.org/jira/browse/BEAM-8650 The error message indicates that ClickHouse (which I'm not familiar with) is trying to connect (random) strange IP address for Zoo

Re: org.apache.beam.sdk.io.clickhouse.AtomicInsertTest.testIdempotentInsert fails

I cannot reproduce this issue. Somehow after rebooting my computer and run "iptables -L", the problem disappeared. Closed ticket BEAM-8650 with my observation. Gleb, thank you for your help.

Re: Completeness of Beam Java Dependency Check Report

s (ZetaSQL has it on API surface so it is OK here, but >> should be correctly declared) >> >> We used to have an analysis that prevented this class of error. >> >> Once the errors are fixed, the guava_version is simply a version that we >> have discovered that

Re: Beam Dependency Check Report (2019-11-19)

It’s empty because I’m verifying latest changes. Give me some more time. On Tue, Nov 19, 2019 at 12:41 Apache Jenkins Server < jenk...@builds.apache.org> wrote: > ERROR: File > 'src/build/dependencyUpdates/beam-dependency-check-report.html' does not > exist -- Regards, Tomo

Re: [PROPOSAL] Preparing for Beam 2.18 release

Hi Udi, (Question) I started learning how Beam dependencies are maintained through releases. https://beam.apache.org/contribute/dependencies/ says *Beam community has agreed on following policies regarding upgrading dependencies.* ... *A significantly outdated dependency (identified manually o

Re: [PROPOSAL] Preparing for Beam 2.18 release

Thank you for response. On Wed, Nov 20, 2019 at 16:49 Chamikara Jayalath wrote: > > > On Wed, Nov 20, 2019 at 1:04 PM Tomo Suzuki wrote: > >> Hi Udi, >> >> (Question) I started learning how Beam dependencies are maintained >> through releases. https://bea

Re: Python tests and setup

I resolved my Python3.6 problem by installing Python3.6 from source: https://issues.apache.org/jira/browse/BEAM-8787#comment-16979785 On Mon, Nov 18, 2019 at 1:49 PM Elliotte Rusty Harold wrote: > and probably related to that: > > https://github.com/pyinstaller/pyinstaller/issues/4064 > > On Mon

Re: Beam Dependency Check Report (2019-11-25)

Can anybody take action on this error? > The service account was not found. The instance must be restarted via the Compute Engine API to restore service account access. Regards, Tomo On Mon, Nov 25, 2019 at 07:04 Apache Jenkins Server < jenk...@builds.apache.org> wrote: > ('Failed to retrieve >

Re: Failed retrieving service account

Thank you for looking into this. On Mon, Nov 25, 2019 at 12:59 PM Yifan Zou wrote: > Greetings, > > We're seeing some tests encountering permission issues such as *'Failed > to retrieve > http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/844138762903-comp...@developer.

Re: Failed retrieving service account

bels.email_id%3d%22844138762903-comp...@developer.gserviceaccount.com%22%0A%22DeleteServiceAccount%22&pinnedLogId=170pqn8e3ofe0&pinnedLogTimestamp=2019-11-25T07:55:19.038646702Z> > . > I restored the default service account. All workers should be backing to > normal and jobs start passing now. >

Hadoop client version 2.8.5 from 2.7 (EOL)

Hi Beam developers, I created a PR to upgrade Hadoop client version. https://github.com/apache/beam/pull/10222 . However, I don't have Hadoop cluster to test this. Can anybody try to see whether this change is compatible with a real Hadoop 2.7 / 2.8 cluster or not? -- Regards, Tomo

BEAM-8865: Code Review Wanted for a Javadoc update

Hi Beam developers, Can somebody review/merge this javadoc update? The example code there was not compiling. https://github.com/apache/beam/pull/10256 "Updating Javadoc of FileIO example" The original author of the javadoc (jkff) wrote my proposal is good. -- Regards, Tomo

Re: BEAM-8865: Code Review Wanted for a Javadoc update

Thank you, Pablo and Chamikara. On Mon, Dec 16, 2019 at 12:49 PM Tomo Suzuki wrote: > > Hi Beam developers, > > Can somebody review/merge this javadoc update? The example code there > was not compiling. > https://github.com/apache/beam/pull/10256 "Updating Javadoc of

Re: Apache beam Python Error runners-spark-job-server-2.19.0-SNAPSHOT.jar not found

I don't use spark-job server but the error says you need to build the JAR file by cd C:\apache_beam; ./gradlew runners:spark:job-server:shadowJar Did you try that? On Wed, Dec 18, 2019 at 3:08 PM Dhiren Pachchigar wrote: > > Hi Team, > > I am trying to submit beam job in local spark with bel

[BEAM-9000] Java Test Assertions without relying on toString

Hi Beam developers, There are many Java tests relying on toString() methods for assertions [1]. This style is prone to unnecessary maintenance of the test code when upgrading dependencies. For example, BEAM-8695 encountered ~10 comparison failures due to change in toString implementation when I tr

Re: [BEAM-9000] Java Test Assertions without relying on toString

gt; On Thu, Dec 19, 2019 at 1:17 PM Tomo Suzuki wrote: >> >> Hi Beam developers, >> >> There are many Java tests relying on toString() methods for assertions >> [1]. This style is prone to unnecessary maintenance of the test code >> when upgrading dependencies. F

Jenkins jobs not running for my PR 10438

Hi Beam developers, Does anybody know why my PR does not trigger Jenkins jobs today? https://github.com/apache/beam/pull/10438 -- Regards, Tomo

Re: Jenkins jobs not running for my PR 10438

Jenkins started working. Thank you for whoever fixed it. On Fri, Dec 20, 2019 at 1:42 PM Boyuan Zhang wrote: > > Same here. Even the phrase trigger doesn't work. > > On Fri, Dec 20, 2019 at 10:16 AM Luke Cwik wrote: >> >> I'm also affected by this. >>

Re: Flaky Java warning/error inventory (cannot find symbol)

I’m interested in that problem. “Cannot find symbol” problem is usually caused by incompatible dependencies. No randomness. Would you share the 2 URLs for the Java Precommit Jenkins jobs which you think should fail with the same cause? Regards, Tomo On Sun, Jan 5, 2020 at 05:48 Alex Van Boxel w

Re: Flaky Java warning/error inventory (cannot find symbol)

reCommit_Java_Phrase/1595/ > https://builds.apache.org/job/beam_PreCommit_Java_Phrase/1594/ > https://builds.apache.org/job/beam_PreCommit_Java_Phrase/1588/ > > > > > _/ > _/ Alex Van Boxel > > > On Mon, Jan 6, 2020 at 12:56 AM Tomo Suzuki wrote: >>

Re: Jenkins jobs not running for my PR 10438

> On Fri, Jan 3, 2020 at 1:42 AM Kai Jiang >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Alan for checking this out! I closed PR 9903 and reopen it >>>>>>>>> in pull/10493 <https://github.com/a

Re: Jenkins jobs not running for my PR 10438

Hi Bean developers, I appreciate a committer can trigger precommit build for https://github.com/apache/beam/pull/10554. In addition to normal precommit checks, I want the followings: Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Da

Re: Jenkins jobs not running for my PR 10438

Thank you, Andrew! On Fri, Jan 10, 2020 at 15:59 Tomo Suzuki wrote: > Hi Bean developers, > > I appreciate a committer can trigger precommit build for > https://github.com/apache/beam/pull/10554. > > In addition to normal precommit checks, I want the followings: > Run Java

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, Four Jenkins jobs did not report back for this PR https://github.com/apache/beam/pull/10554 . Can somebody trigger them? On Fri, Jan 10, 2020 at 4:51 PM Andrew Pilloud wrote: > > Done. > > On Fri, Jan 10, 2020 at 12:59 PM Tomo Suzuki wrote: >> >> Hi

Re: Jenkins jobs not running for my PR 10438

Thanks Yifan (but Java Precommit is still missing). Can somebody run "Run Java PreCommit" on https://github.com/apache/beam/pull/10554? On Mon, Jan 13, 2020 at 2:59 AM Yifan Zou wrote: > > done. > > On Sun, Jan 12, 2020 at 6:27 PM Tomo Suzuki wrote: >> >> Hi B

Re: Jenkins jobs not running for my PR 10438

Thank you, Mark and Ismaël. On Mon, Jan 13, 2020 at 2:34 PM Mark Liu wrote: > > done > > On Mon, Jan 13, 2020 at 8:03 AM Tomo Suzuki wrote: >> >> Thanks Yifan (but Java Precommit is still missing). >> Can somebody run "Run Java PreCommit" on >

Re: Jenkins jobs not running for my PR 10438

Hi Beam developers, Can somebody run the following to https://github.com/apache/beam/pull/10554 ? Run Dataflow ValidatesRunner Run Java PreCommit On Mon, Jan 13, 2020 at 2:35 PM Tomo Suzuki wrote: > > Thank you, Mark and Ismaël. > > On Mon, Jan 13, 2020 at 2:34 PM Mark Liu wrote:

Re: Jenkins jobs not running for my PR 10438

Valentyn, thank you. On Tue, Jan 14, 2020 at 12:05 PM Valentyn Tymofieiev wrote: > > Done. If tests still don't trigger, you could try to make a push to the > branch to reset the test status. > > On Tue, Jan 14, 2020 at 8:38 AM Tomo Suzuki wrote: >> >> Hi Bea

Re: Jenkins jobs not running for my PR 10438

It hit Dataflow quota error again. Can somebody run Run Dataflow ValidatesRunner for https://github.com/apache/beam/pull/10554 ? On Tue, Jan 14, 2020 at 12:14 PM Tomo Suzuki wrote: > > Valentyn, thank you. > > On Tue, Jan 14, 2020 at 12:05 PM Valentyn Tymofieiev > wrote: > &g

Re: Quota limitation for Java tests

Hi Beam committers, I encountered a similar problem today for "Run Dataflow ValidatesRunner": Dataflow quota error for jobs-per-project quota. Project apache-beam-testing is running 303 jobs. https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/190/testReport/junit/o

Re: Jenkins jobs not running for my PR 10438

neth Knowles, any updates from INFRA on this? > > On Tue, Jan 14, 2020 at 6:43 PM Tomo Suzuki wrote: >> >> It hit Dataflow quota error again. Can somebody run >> Run Dataflow ValidatesRunner >> for https://github.com/apache/beam/pull/10554 ? >> >&g

Re: Beam's Avro 1.8.x dependency

I've been upgrading dependencies around gRPC. This Avro-problem is interesting to me. I'll study BEAM-8388 more tomorrow. On Wed, Jan 15, 2020 at 10:51 PM Luke Cwik wrote: > > +Tomo Suzuki +jincheng sun > There have been a few contributors upgrading the dependencies and valid

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, I appreciate if somebody can trigger precommit checks for https://github.com/apache/beam/pull/10614 with the following additional checks: Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Ru

Re: Jenkins jobs not running for my PR 10438

Performance Test Java Run Dataflow ValidatesRunner Run Spark ValidatesRunner Run SQL Postcommit On Thu, Jan 16, 2020 at 4:11 PM Andrew Pilloud wrote: > > done. > > On Thu, Jan 16, 2020 at 1:07 PM Tomo Suzuki wrote: >> >> Hi Beam committers, >> >> I appreciate if s

Re: Jenkins jobs not running for my PR 10438

020 at 1:35 PM Ismaël Mejía wrote: > >> done >> >> On Fri, Jan 17, 2020 at 5:51 AM Tomo Suzuki wrote: >> >>> Hi Beam Committers, >>> (Andrew, thanks! but I needed to fix tests) >>> >>> I appreciate if somebody can re-trigger precommi

Re: Beam's Avro 1.8.x dependency

Ismaël and I think the short-term solution for Aaron's issue is to define Beam's own class for Avro's TimestampConversion. Luckily it's the only blocker for Avro 1.9 I found. I created a ticket https://issues.apache.org/jira/browse/BEAM-9144 and PR https://github.com/apache/beam/pull/10628 . Regar

Re: Jenkins jobs not running for my PR 10438

Hi Beam committer, I appreciate if somebody can trigger the following checks for https://github.com/apache/beam/pull/10631 Run JavaPortabilityApi PreCommit Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run

Re: Jenkins jobs not running for my PR 10438

Thank you, Ahmet. On Fri, Jan 17, 2020 at 7:22 PM Tomo Suzuki wrote: > > Hi Beam committer, > > I appreciate if somebody can trigger the following checks for > https://github.com/apache/beam/pull/10631 > > Run JavaPortabilityApi PreCommit > Run Java PostCommit &

Re: help with this error, please

Hi Vasu, (Ignore my message if Luke's advice resolves the issue already) Would you add the entire error message, if any? A NoClassDefFoundError is usually caused by a ClassNotFoundError saying a class is not found. I don't see the missing class name in your stacktrace. I'll need (1) the entire err

Re: Jenkins jobs not running for my PR 10438

Hi Beam Committers, I appreciate if you can run precommit checks for https://github.com/apache/beam/pull/10674 plus the following 6 extra commands: Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run Spark Va

Re: Jenkins jobs not running for my PR 10438

Hi Rui, (Thank you for quick response) Would you run one command per one comment? I don't think Jenkins recognizes multiple at once. On Thu, Jan 23, 2020 at 2:25 PM Rui Wang wrote: > > Done > > On Thu, Jan 23, 2020 at 11:20 AM Tomo Suzuki wrote: >> >> Hi Beam Comm

Re: Jenkins jobs not running for my PR 10438

Hi Beam Comitters, Can somebody trigger 2 failed checks below for https://github.com/apache/beam/pull/10674 ? Run Java PreCommit Run JavaPortabilityApi PreCommit On Thu, Jan 23, 2020 at 14:32 Tomo Suzuki wrote: > Hi Rui, > (Thank you for quick response) > > Would you run one com

Re: PreCommit Java Portability is failing

Hannah, Thank you for the fix (#10676)! On Thu, Jan 23, 2020 at 2:55 PM Hannah Jiang wrote: > > I will submit a small patch this afternoon, the original PR to fix this issue > needs some time to merge. > > > On Thu, Jan 23, 2020 at 11:33 AM Andrew Pilloud wrote: >> >> This is still hard failin

Re: anyone working on updating com.google.cloud:google-cloud-spanner?

I'm going to take that in the coming weeks. Currently I'm focusing on updating the bigtable-client-core version. On Tue, Jan 28, 2020 at 4:32 PM Luke Cwik wrote: > > +Tomo Suzuki has been working on plenty of dependency updates related to GCP > libraries and could give gui

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, I appreciate if you can trigger the precommit checks for https://github.com/apache/beam/pull/10714 with following 6 additional commands (one command per comment): Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run D

Re: Jenkins jobs not running for my PR 10438

HI Beam committers: (Thanks, Luke!) Can somebody retrigger the following 2 failed checks for https://github.com/apache/beam/pull/10714 ? Run Java PreCommit Run Java_Examples_Dataflow PreCommit On Wed, Jan 29, 2020 at 4:48 PM Luke Cwik wrote: > > done > > On Wed, Jan 29, 2020 at 1

flaky WordCountIT.testE2EWordCount in Jenkins Java_Examples_Dataflow Job

Hi Beam developers, I filed a ticket BEAM-9224 "flaky WordCountIT.testE2EWordCount in Jenkins Java_Examples_Dataflow Job". The trend [1] shows the flakiness started around Jan 28 13:00 ET (Build #6538 [2]). Example job failure is in [3]. This Jenkins log does not provide why it failed: java.lang

Re: flaky WordCountIT.testE2EWordCount in Jenkins Java_Examples_Dataflow Job

ge-for-end-to-end-tests//testpipeline-jenkins-0128181056-28c844f3/output/results/staging/","streaming":true,"tempLocation":"gs://temp-storage-for-end-to-end-tests//testpipeline-jenkins-0128181056-28c844f3/output/results","tempRoot":"gs://temp-storag

Re: Jenkins jobs not running for my PR 10438

ub.com/apache/beam/pull/10712. >>> >>> Thanks! >>> >>> *Shoaib Zafar* >>> >>> Software Engineering Lead >>> Mobile: +92 333 274 6242 >>> Skype: live:shoaibzafar_1 >>> >>> <http://venturedive.com/> >>

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, I appreciate if you can trigger precommit checks for https://github.com/apache/beam/pull/10756 . Regards, Tomo

Re: Jenkins jobs not running for my PR 10438

Hi Beam Committers, Would you run the precommit checks for https://github.com/apache/beam/pull/10765 with following 6 additional commands (one command per comment) ?: Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow Validates

Re: Jenkins jobs not running for my PR 10438

Thank you, Ahmet! On Tue, Feb 4, 2020 at 12:36 PM Tomo Suzuki wrote: > > Hi Beam Committers, > > Would you run the precommit checks for > https://github.com/apache/beam/pull/10765 > with following 6 additional commands (one command per comment) ?: > > Run Ja

SQL PostCommit failure: ClassCastException: java.lang.Integer cannot be cast to java.lang.Long

HI Beam developers, Can somebody help this SQL PostCommit integration test failure? https://issues.apache.org/jira/browse/BEAM-9253 (since https://github.com/apache/beam/pull/10563) SQL PostCommit failure: ClassCastException: java.lang.Integer cannot be cast to java.lang.Long (Why not?) The dif

Re: SQL PostCommit failure: ClassCastException: java.lang.Integer cannot be cast to java.lang.Long

#x27;m not sure if we have any formal > process for that. > > Brian > > On Wed, Feb 5, 2020 at 5:31 AM Tomo Suzuki wrote: >> >> HI Beam developers, >> >> Can somebody help this SQL PostCommit integration test failure? >> https://issues.apache.or

Re: Jenkins jobs not running for my PR 10438

Hi Beam Committers, Would you run the precommit checks for https://github.com/apache/beam/pull/10769 with the following 6 additional commands (one command per comment) ? Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow Valida

Re: SQL PostCommit failure: ClassCastException: java.lang.Integer cannot be cast to java.lang.Long

(My understanding) The test ensures the CSV data stored in GCS should be readable through Datacatalog. It fails because an Integer value in the CSV was read as Long as per Datacatalog. > setting up from scratch is a good idea. I agree. Furthermore, it would be nice if it can test different type-

Re: SQL PostCommit failure: ClassCastException: java.lang.Integer cannot be cast to java.lang.Long

gt; INT64), and then passing it to a PAssert that checked against Rows with a >> different schema (id: INT32). So I think it's a legitimate error because we >> were asserting that a PCollection contained rows with one schema but it >> actually had a different sch

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, I appreciate if you can run precommit checks for https://github.com/apache/beam/pull/10769 with the following 6 commands: Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run Spark Validate

BEAM-8758: Code Review Wanted for PR 10765

Hi Udi, Luke, and Beam committers, Would you review/merge this google-cloud-spanner dependency upgrade? https://github.com/apache/beam/pull/10765 -- Regards, Tomo

Re: Upgrades gcsio to 2.0.0

> What prevents the usage of the newer version of Guava? Cassandra is referencing an old field of Guava's CharMatcher. The field "DIGIT" is no longer available after Guava 26. https://github.com/apache/beam/pull/10769#issuecomment-583698718 On Mon, Feb 10, 2020 at 5:39 PM Luke Cwik wrote: > > Wh

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, Would you run precommit checks on https://github.com/apache/beam/pull/10765 with the following 6 additional commands? Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run Spark ValidatesRunn

Re: Jenkins jobs not running for my PR 10438

Ahmet, thanks. But it seems Jenkins is not reporting the status correctly. Will check tomorrow. On Thu, Feb 13, 2020 at 2:45 PM Tomo Suzuki wrote: > > Hi Beam committers, > > Would you run precommit checks on https://github.com/apache/beam/pull/10765 > with the following 6 addi

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, Would you trigger the precommit checks for https://github.com/apache/beam/pull/11042 with the following 6 commands ? Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run Spark ValidatesRunn

Proposal: Beam to use GCP Libraries BOM

Hi Beam developers, Shall we use GCP Libraries BOM [1] to specify the Google-related library versions in Beam? I've been working on Beam's dependency upgrades in the past few months. It's time to consider a long-term solution to keep the libraries up-to-date with small maintenance effort. To achi

Re: Proposal: Beam to use GCP Libraries BOM

endencies. >>> > >>> > On Thu, Mar 5, 2020, 04:07 Chamikara Jayalath >>> wrote: >>> >> >>> >> +1 for this. >>> >> >>> >> This will make life easy for many of our users and will help us keep >>>

Re: Proposal: Beam to use GCP Libraries BOM

2020 at 6:59 AM Alex Van Boxel wrote: >>>> > >>>> > +1, I can remember the countless hours that we fought with Google >>>> dependencies. >>>> > >>>> > On Thu, Mar 5, 2020, 04:07 Chamikara Jayalath >>

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, (Thank you, Michał!) Would you trigger the precommit checks for https://github.com/apache/beam/pull/11063 with the following 6 commands ? Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run

Re: Jenkins jobs not running for my PR 10438

Thanks, Luke! On Fri, Mar 6, 2020 at 13:34 Tomo Suzuki wrote: > Hi Beam committers, > (Thank you, Michał!) > > Would you trigger the precommit checks for > https://github.com/apache/beam/pull/11063 > with the following 6 commands ? > Run Java PostCommit > Run Java Ha

Re: Jenkins jobs not running for my PR 10438

2020 at 10:35 AM Tomo Suzuki wrote: > >> Hi Beam committers, >> (Thank you, Michał!) >> >> Would you trigger the precommit checks for >> https://github.com/apache/beam/pull/11063 >> with the following 6 commands ? >> Run Java PostCommit >> Run

Re: Jenkins jobs not running for my PR 10438

Java PreCommit failed due to Flink's UnboundedSourceWrapperTest (BEAM-9164). Can somebody retrigger "Run Java PreCommit" for https://github.com/apache/beam/pull/11063 ? On Mon, Mar 9, 2020 at 9:58 AM Tomo Suzuki wrote: > Hi Beam committers, > > 2 of the previous chec

Re: Jenkins jobs not running for my PR 10438

Thank you, Luke. Now Java Precommit is green. On Mon, Mar 9, 2020 at 2:21 PM Tomo Suzuki wrote: > Java PreCommit failed due to Flink's UnboundedSourceWrapperTest > (BEAM-9164). > Can somebody retrigger "Run Java PreCommit" for > https://github.com/apache/beam/pull/

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, Would you trigger precomimt checks for https://github.com/apache/beam/pull/11095 with the following 6 commands ? Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run Spark ValidatesRunner Ru

Re: [Newbie Contributing Question] How to add a dependency?

Hi Jacob, You'll need to modify BeamModulePlugin.groovy, which defines a big map of Maven artifacts. The map is referenced by each module such as the google-cloud-platform/build.gradle. Example PR to touch dependencies: https://github.com/apache/beam/pull/11063/files On Tue, Mar 10, 2020 at 11:22

Re: Proposal: Beam to use GCP Libraries BOM

Thank you for favorable responses. I'll start implementation. On Thu, Mar 5, 2020 at 2:22 PM Tomo Suzuki wrote: > > Do Spark or Flink have BOMs? > > Not that I know of. I couldn't find "bom" in their artifacts [1, 2]. > > [1]: https://search.maven.org/se

Re: [Newbie Contributing Question] How to add a dependency?

ompile dependencies such as > junit aren't getting resolved when running > ``` > ./gradlew -p sdks/java/io/google-cloud-platform/ test > ``` > I didn't touch these. Is there some place I have to add new test > directory? > > On 2020/03/11 04:17:50, Tomo Suzuki wrot

Re: Hello Beam Community!

Welcome. On Fri, Mar 13, 2020 at 1:20 PM Udi Meiri wrote: > Welcome! > > > On Fri, Mar 13, 2020 at 9:47 AM Yichi Zhang wrote: > >> Welcome! >> >> On Fri, Mar 13, 2020 at 9:40 AM Ahmet Altay wrote: >> >>> Welcome Brittany! >>> >>> On Thu, Mar 12, 2020 at 6:32 PM Brittany Hermann >>> wrote: >>>

/zetasql/local_service/liblocal_service_jni.jnilib was not found inside JAR

Hi Beam developers, I started getting test failures when building Beam in my MacBook Pro. Module: sdks/java/extensions/sql/zetasql. The NoClassDefFoundError occurs because of jnilib file is missing. Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: File /zetasql/local_service/

Re: /zetasql/local_service/liblocal_service_jni.jnilib was not found inside JAR

ilib for OS X. A few have tried but no one has > succeeded (as far as I know), it is on the ZetaSQL team's todo list. You'll > need to run that module on Linux for now. > > See: https://github.com/google/zetasql/pull/3 > > Andrew > > On Mon, Mar 16, 2020 at 12:09 PM To

Re: Jenkins jobs not running for my PR 10438

Hi Beam committers, Would you trigger the precommit checks for https://github.com/apache/beam/pull/11156 with the following 6 commands ? Run Java PostCommit Run Java HadoopFormatIO Performance Test Run BigQueryIO Streaming Performance Test Java Run Dataflow ValidatesRunner Run Spark ValidatesRunn

  1   2   >