New contributor to BEAM SQL

Hello everyone! My name is Kirill Kozlov, I recently joined a Dataflow team at Google and will be working on SQL filter pushdown. Can I get permission to work issues in jira, my username is: kirillkozlov Looking forward to developing Beam together! Thank you, Kirill Kozlov

[DISCUSS] Beam SQL filter push-down

on this proposal. Thank you! Here is a more complete design doc: https://docs.google.com/document/d/1-ysD7U7qF3MAmSfkbXZO_5PLJBevAL9bktlLCerd_jE/edit?usp=sharing -- Kirill Kozlov

Re: (Question) SQL integration tests for MongoDb

s` parameter. > > > > On Thu, Nov 7, 2019 at 6:40 PM Kirill Kozlov > wrote: > >> Thank you for your response! >> >> I want to make sure that when tests run on Jenkins they get supplied with >> pipelines options containing hostName and Port of a running MongoDb service

Re: (Question) SQL integration tests for MongoDb

h `enableJavaPerformanceTesting()` will allow such > options to be passed. > > If you're trying to do something else, please let me know. > > Regards > Michal > > On Thu, Nov 7, 2019 at 1:44 AM Kirill Kozlov > wrote: > >> Hi everyone! >> >>

Re: [Discuss] Beam mascot

Angler fish? Found a few animated examples that may look interesting [1, 2, 3, 4]. [1] https://www.pinterest.com/pin/121175046208927340/ [2] https://www.pinterest.com/pin/353180795779533157/ [3] https://www.pinterest.com/pin/121175046208927334/ [4]

Re: [ANNOUNCE] New committer: Brian Hulette

Congratulations, Brian! On Fri, Nov 15, 2019 at 10:56 AM Udi Meiri wrote: > Congrats Brian! > > > On Fri, Nov 15, 2019 at 10:47 AM Ruoyun Huang wrote: > >> Congrats Brian! >> >> On Fri, Nov 15, 2019 at 10:41 AM Robin Qiu wrote: >> >>> Congrats, Brian! >>> >>> On Fri, Nov 15, 2019 at 10:02 AM

(Question) SQL integration tests for MongoDb

Hi everyone! I am trying to test MongoDb Sql Table, but not quite sure how to pass pipeline options with the hostName, port, and databaseName used by Jenkins. It looks like the integration test for MongoDbIO Connector obtain those values from the

Quota limitation for Java tests

Hello everyone! It looks like JavaPostCommit Jenkins tests [1] are failing due to CPU quota limitations. Could someone please look into this? [1]

Jenkins jobs are not being displayed on GitHub

Hello everyone! It looks like for the PRs created within the last 30 minutes status of Jenkins jobs is not being displayed. Seed job appears to be stuck [1]. #5293 is waiting for #5292 to finish, but #5293 shows that it is complete. [1] https://builds.apache.org/job/beam_SeedJob/ -- Kirill

Re: [Question] Cannot resolve symbol 'AutoValue_KafkaIO_WriteRecords'

I found this document [1] helpful when setting up an Intellij to work with Beam. Make sure that (File | Settings | Build, Execution, Deployment | Build Tools | Gradle | Runner) has a "Delegate IDE build/run actions to Gradle" checked and "Run test using" is set to "Gradle Test Runner". Under

Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

[ ] Beaver [ ] Hedgehog [X] Lemur [X] Owl [ ] Salmon [ ] Trout [ ] Robot dinosaur [ ] Firefly [ ] Cuttlefish [ ] Dumbo Octopus [X] Angler fish On Wed, Nov 20, 2019, 08:38 Cyrus Maden wrote: > Here's my vote, but I'm curious about the distinction between salmon and > trout mascots :) > > [ ]

Update on push-down for SQL IOs.

Hello everyone! I have been working on a push-down feature and would like to give a brief update on what is done and is still under works. *Things that are done*: General API for SQL IOs to provide information about what filters/projects they support [1]: - *Filter* can be unsupported, supported

Re: Update on push-down for SQL IOs.

ras that could potentially be >> tagged "starter"? They seem complex but maybe your framework will make it >> feasible for someone with slightly less experience to implement new >> versions of what you have already finished? >> >> Kenn >> >> On Tue, Nov

Failing Java PostCommit for Dataflow runner

Hello everyone! I have noticed that Jenkins tests for Dataflow runner [1] are failing with a runtime exception. It looks like the issue originated here [2], failed Dataflow job [3]. We should look into fixing it. Failing test: :runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest »

Re: Failing Java PostCommit for Dataflow runner

t;> On Mon, Jan 13, 2020 at 11:23 AM Boyuan Zhang wrote: >> >>> Thanks Kirill! I'm going to look into it. >>> >>> On Mon, Jan 13, 2020 at 11:18 AM Kirill Kozlov >>> wrote: >>> >>>> Hello everyone! >>>> >>>

Re: Jenkins job execution policy

Thanks for working on this! I have noticed that tests run for new PRs and force-pushed commits, but if a test fails due to a flake I am unable to re-run it (ex: "Run Java PreCommit"). PR that has this issue: https://github.com/apache/beam/pull/10369. Is this intended behaviour? - Kirill On Tue,

Re: Jenkins jobs not running for my PR 10438

Thank you Rui and Ankur! On Tue, Jan 21, 2020 at 4:14 PM Ankur Goenka wrote: > Done > > On Tue, Jan 21, 2020 at 4:03 PM Kirill Kozlov > wrote: > >> Forgot to paste a link to a PR: https://github.com/apache/beam/pull/10649 >> >> On Tue, Jan 21, 2020 at 3:47 PM R

Re: Jenkins jobs not running for my PR 10438

Forgot to paste a link to a PR: https://github.com/apache/beam/pull/10649 On Tue, Jan 21, 2020 at 3:47 PM Rui Wang wrote: > Done but I am not seeing tests are triggered by those two commands. > > On Tue, Jan 21, 2020 at 3:42 PM Kirill Kozlov > wrote: > >> Hello again

Re: Jenkins jobs not running for my PR 10438

Hello Beam community, Can a committer re-run tests for this PR please? https://github.com/apache/beam/pull/10440 Thank you! On Fri, Jan 17, 2020 at 4:44 PM Tomo Suzuki wrote: > Thank you, Ahmet. > > > On Fri, Jan 17, 2020 at 7:22 PM Tomo Suzuki wrote: > > > > Hi Beam committer, > > > > I

Re: Jenkins jobs not running for my PR 10438

Thank you, Ismaël! On Mon, Jan 20, 2020 at 2:14 PM Ismaël Mejía wrote: > done > > On Mon, Jan 20, 2020 at 9:27 PM Kirill Kozlov > wrote: > >> Hello Beam community, >> >> Can a committer re-run tests for this PR please? >> https://github.com/apache/beam/p

PreCommit Java Portability is failing

Hello beam developers! Jenkins job page: https://builds.apache.org/job/beam_PreCommit_JavaPortabilityApi_Cron/ The following tasks fails: ` *:runners:google-cloud-dataflow-java:buildAndPushDockerContainer*` with error: *'command 'docker'' finished with non-zero exit value 1* Is it possible that

Re: Jenkins jobs not running for my PR 10438

Hello again! Could someone trigger tests on this PR please? *Run SQL postcommit* *Run JavaBeamZetaSQL PreCommit* On Mon, Jan 20, 2020 at 2:38 PM Kirill Kozlov wrote: > Thank you, Ismaël! > > On Mon, Jan 20, 2020 at 2:14 PM Ismaël Mejía wrote: > >> done >> >> O

[Design Proposal] DataStore SQL Connector

Hello everyone! I have written up a proposal [1] for a DataStore SQL connector. I would love to hear comments and suggestions from the Beam dev community! A quick summary: DataStore [2] is a NoSQL database with a dynamic schema, where entities (documents) are stored in Kinds (databases). Each

Edit access to Wiki

Hello everyone! I was hoping to add a design doc for SQL push-down [1] to the Wiki page [2], but I need edit access. What is the process for obtaining edit access? My wiki username is: Kirill Kozlov [1] https://docs.google.com/document/d/1-ysD7U7qF3MAmSfkbXZO_5PLJBevAL9bktlLCerd_jE/edit [2

Re: Edit access to Wiki

Thank you! On Fri, Jan 3, 2020 at 10:39 AM Luke Cwik wrote: > I have added you. Happy editing. > > On Fri, Jan 3, 2020 at 10:31 AM Kirill Kozlov > wrote: > >> Hello everyone! >> >> I was hoping to add a design doc for SQL push-down [1] to the Wiki page >&g

Re: Intellij Issue in Imports

Hi Zohaib, What command are you using to build and run the project? Does building from command line work? Try checking for problems via "File -> Project Structure -> Problems". Intellij setup notes can be found here: