Re: Structured Logging in python

Hi, I believe this wasn't implemented for Python (only Java). You can try adding structured data (extra keyword) under the key "custom_data" and that might work. On 2024/04/11 17:49:43 Valentyn Tymofieiev wrote: > Thanks for reaching out. There was a proposal a while back: >

Re: Proposal: Structured Logging

The document has been updated. Please take another look and let me know if there are any objections Thanks! On Wed, Mar 8, 2023 at 1:59 PM Udi Meiri wrote: > Hi all, > I have written a proposal for Structured Logging in Beam: > https://s.apache.org/beam-structured-logging > >

Proposal: Structured Logging

Hi all, I have written a proposal for Structured Logging in Beam: https://s.apache.org/beam-structured-logging Please LMK what you think. Any comments welcome here or in the doc. - Udi

[ANNOUNCE] Beam 2.33.0 Release

The Apache Beam team is pleased to announce the release of version 2.33.0. Apache Beam is an open source unified programming model to define and execute data processing pipelines, including ETL, batch and stream (continuous) processing. See https://beam.apache.org You can download the release

[RESULT] [VOTE] Release 2.33.0, release candidate #2

I'm happy to announce that we have unanimously approved this release. There are 8 approving votes, 5 of which are binding: * Ahmet Altay * Alexey Romanenko * Robert Bradshaw * Kenneth Knowles * Chamikara Jayalath There are no disapproving votes. Thanks everyone!

[VOTE] Release 2.33.0, release candidate 2

Hi everyone, Please review and vote on the release candidate #2 for the version 2.33.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [RESULT] [VOTE] Release 2.33.0, release candidate #1

I spoke too soon. We will be doing an rc2 On Mon, Sep 27, 2021 at 1:29 PM Udi Meiri wrote: > I'm happy to announce that we have unanimously approved this release. > > There are 8 approving votes, 4 of which are binding: > * Ahmet Altay > * Alexey Romanenko > * Robert Brad

[RESULT] [VOTE] Release 2.33.0, release candidate #1

I'm happy to announce that we have unanimously approved this release. There are 8 approving votes, 4 of which are binding: * Ahmet Altay * Alexey Romanenko * Robert Bradshaw * Chamikara Jayalath There are no disapproving votes. Thanks everyone!

[VOTE] Release 2.33.0, release candidate 1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.33.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [PROPOSAL] Preparing for Beam 2.33.0 Release

t; See my comment below. > > On 26 Aug 2021, at 03:44, Ahmet Altay wrote: > > On Wed, Aug 25, 2021 at 1:50 PM Udi Meiri wrote: > >> I've found 2 performance regressions in IO. Could someone triage these? >> > > >> https://issues.apache.org/jira/browse/BEA

Re: One Pager - Test Command Line Discoverability in Beam

My first place to go would be here: https://cwiki.apache.org/confluence/display/BEAM/Java+Tips (although it doesn't document your use-case) You are right that finding the correct gradle task or jenkins job is not straightforward. On Tue, May 25, 2021 at 12:48 PM Alex Amato wrote: > Friendly

Re: BEAM-3713: Moving from nose to pytest

ner') -> @pytest.mark.it_validatesrunner On Thu, Mar 25, 2021 at 10:45 AM Udi Meiri wrote: > Hi Benjamin, > > AFAIK nose is only used for integration tests (unit tests were converted > to pytest a while back). > These ITs should all be running periodically (except maybe the r

Re: Codecov Bash Uploader Security Notice

>From the notice: "We strongly recommend affected users immediately re-roll all of their credentials, tokens, or keys located in the environment variables in their CI processes that used one of Codecov’s Bash Uploaders." On Thu, Apr 15, 2021 at 11:35 AM Udi Meiri wrote: >

Codecov Bash Uploader Security Notice

I got this email: https://about.codecov.io/security-update/ This is where we use codecov: https://github.com/apache/beam/blob/39923d8f843ecfd3d89443dccc359c14aea8f26f/sdks/python/tox.ini#L105 I'm not sure if this runs the "bash uploader", but we do set a CODECOV_TOKEN environment variable.

Re: BEAM-3713: Moving from nose to pytest

tests to run. So if I wanted to run post-commit tests on direct runner I would use this pytest flag: "-m 'it_postcommit and not no_direct'". On Wed, Mar 24, 2021 at 5:41 PM Ahmet Altay wrote: > All PRs look either merged or closed. > > +Udi Meiri might have more information ab

Re: [ANNOUNCE] New PMC Member: Chamikara Jayalath

Congrats Cham! On Thu, Jan 21, 2021 at 4:25 PM Griselda Cuevas wrote: > Congratulations Cham!!! Well deserved :) > > On Thu, 21 Jan 2021 at 15:23, Connell O'Callaghan > wrote: > >> Well done Cham!!! Thank you for all your contributions to date!!! >> >> >> On Thu, Jan 21, 2021 at 3:18 PM Rui

Re: Jenkins trigger phrase "run seed job" not working?

Yeah, I've had that issue recently: https://github.com/apache/beam/pull/13213 (seed job did not trigger) I've personally never seen where this is configured, but it sounds like it. What happens if you edit a file in the PR? It should make you a co-author. On Mon, Nov 9, 2020 at 3:36 AM Kamil

Re: [PROPOSAL] Preparing for Beam 2.26.0 release

+1 sg! On Tue, Oct 27, 2020 at 10:02 AM Robert Burke wrote: > Hello everyone! > > The next Beam release (2.26.0) is scheduled to be cut on November 4th > according to the release calendar [1]. > > I'd like to volunteer myself to handle this release. I plan on cutting the > branch on November

Re: Shutting down Perfkit Explorer

hat image, but the > datastore related stuff remains functional. > > On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri wrote: > >> Is it possible to create a simple "hello world" application instead? >> >> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri wrote: >> >>>

Re: JIRA components

Sep 17, 2020 at 7:17 PM Kenneth Knowles wrote: > >> What does it mean? >> >> Kenn >> >> On Thu, Sep 17, 2020 at 6:18 PM Udi Meiri wrote: >> >>> Hi, I was going to create a Python types component in our JIRA. >>> (sdk-py-types) >>> Any objections? >>> >> smime.p7s Description: S/MIME Cryptographic Signature

JIRA components

Hi, I was going to create a Python types component in our JIRA. (sdk-py-types) Any objections? smime.p7s Description: S/MIME Cryptographic Signature

Re: JIRA - can't set resolution?

browse/INFRA-20563 >> >> On Tue, Aug 25, 2020 at 10:34 AM Udi Meiri wrote: >> >>> Example: https://issues.apache.org/jira/browse/BEAM-10751 >>> When I click "resolve issue" the status changes to "resolved" but >>> res

JIRA - can't set resolution?

Example: https://issues.apache.org/jira/browse/BEAM-10751 When I click "resolve issue" the status changes to "resolved" but resolution is still "unresolved" and I can't change it. I believe there was a change a while back to JIRA? smime.p7s Description: S/MIME Cryptographic Signature

Re: Memory Issue When Running Beam On Flink

HI David, I'm not familiar with Flink, but assuming there aren't any memory management issues in the runner or SDK, try reducing proceesing_time_duration (30 minutes currently) to 60 seconds and see how long it takes for memory usage to reach the limit. Could you also say how long it currently

Re: Broken links in code velocity dashboard

There's this old bug about it: https://issues.apache.org/jira/browse/BEAM-8389 On Fri, Aug 7, 2020 at 4:45 AM Damian Gadomski wrote: > Unfortunately, I'm not aware of any recent changes. > > On Thu, Aug 6, 2020 at 10:00 PM Ahmet Altay wrote: > >> Damian, or anyone else, do you know if there

Re: Git commit history: "fixup" commits

https://github.com/marketplace/actions/gs-commit-message-checker On Tue, Aug 4, 2020 at 10:25 AM Robert Bradshaw wrote: > +1, thanks for the reminder. > > This should be really easy to automate, using > https://developer.github.com/webhooks/event-payloads/#pull_request to > give a warning when

Re: No space left on device - beam-jenkins 1 and 7

Inventory_apache-beam-jenkins-14/501/console > > > > On Fri, Jul 24, 2020 at 1:07 PM Tyson Hamilton > wrote: > > > > > Yes, these are on the same volume in the /var/lib/docker directory. I'm > > > unsure if they clean up leftover images. > > > &

Re: [VOTE] Make Apache Beam 2.24.0 the final release supporting Python 2.

+1 On Fri, Jul 24, 2020 at 10:47 AM Maximilian Michels wrote: > +1 > > On 24.07.20 18:54, Pablo Estrada wrote: > > +1 - thank you Valentyn! > > -P. > > > > On Thu, Jul 23, 2020 at 1:29 PM Chamikara Jayalath > > wrote: > > > > +1 > > > > On Thu, Jul 23, 2020

Re: Jenkins trigger phrase "run seed job" not working?

ad this >> from LDAP as a part of the seed job and configure the jobs with the apache >> committers present on the ghprb whitelist. >> >> >> Hope that I didn't miss anything ;) It isn't that easy to investigate >> that kind of issues with my poor pri

Re: No space left on device - beam-jenkins 1 and 7

test jobs need to try harder at cleaning up their leftover images. (assuming they're already doing it?) On Fri, Jul 24, 2020 at 12:31 PM Udi Meiri wrote: > The additional slots (@3 directories) take up even more space now than > before. > > I'm testing out https://github.com/apache/bea

Re: No space left on device - beam-jenkins 1 and 7

>>> directory. When I run a du on that, it takes really long. I'll let it keep >>> running for a while to see if it ever returns a result but so far this >>> seems suspect. >>> >>> >>> >>> >>> >>> On Fri, Jul 24,

Re: No space left on device - beam-jenkins 1 and 7

In the "jenkins" user home directory. On Fri, Jul 24, 2020, 11:19 Tyson Hamilton wrote: > Everything I've been looking at is in the /tmp dir. Where are the > workspaces, or what are the named? > > > > > On Fri, Jul 24, 2020 at 11:03 AM Udi Meiri wrote: &g

Re: No space left on device - beam-jenkins 1 and 7

19 12:14./beam-pipeline-tempUTXqlM >>>>>> 242M2020-07-19 12:11./beam-pipeline-tempx3Yno3 >>>>>> 242M2020-07-19 12:05./beam-pipeline-tempyCrMYq >>>>>> 236M2020-07-19 12:14./beam-pipeline-tempUTXqlM/tmpstXoL0 >>>&g

Re: Jenkins trigger phrase "run seed job" not working?

the ghprb whitelist. > > > Hope that I didn't miss anything ;) It isn't that easy to investigate that > kind of issues with my poor privileges ;) > > > Regards, > > Damian > > > On Thu, Jul 23, 2020 at 6:52 PM Udi Meiri wrote: > >> Thanks Damian! I saw

Re: Jenkins trigger phrase "run seed job" not working?

s change: https://github.com/apache/beam/pull/12319. >> >> /cc +Tyson Hamilton >> >> On Wed, Jul 22, 2020 at 1:17 PM Udi Meiri wrote: >> >>> HI, >>> I'm trying to test a groovy change but I can't seem to trigger the seed >>> job. It worked yesterday so I'm not sure what changed. >>> >>> https://github.com/apache/beam/pull/12326 >>> >>> smime.p7s Description: S/MIME Cryptographic Signature

Jenkins trigger phrase "run seed job" not working?

HI, I'm trying to test a groovy change but I can't seem to trigger the seed job. It worked yesterday so I'm not sure what changed. https://github.com/apache/beam/pull/12326 smime.p7s Description: S/MIME Cryptographic Signature

Re: No space left on device - beam-jenkins 1 and 7

ing the tests without write permission to /tmp, etc) > > Kenn > > On Mon, Jul 20, 2020 at 11:39 AM Ahmet Altay wrote: > >> Related to workspace directory growth, +Udi Meiri filed >> a relevant issue previously ( >> https://issues.apache.org/jira/browse/BEAM-9865) f

Re: [PROPOSAL] Make PBegin and PDone public in the Python SDK

re's any use for it in the Python SDK (a >> transform can simply not return any value (which is equivalent to >> returning None) if it has no outputs. >> >> On Mon, Jul 13, 2020 at 8:17 PM Udi Meiri wrote: >> > >> > Details: >> > One item of int

[PROPOSAL] Make PBegin and PDone public in the Python SDK

Details: One item of interest that came up during the implementation of BEAM-10258 [1] is how to treat PTransforms that act like sources or sinks. These transforms have either no input or output PCollections, respectively. Internally, we use PBegin and PDone to denote this. (ex: [2]) IIUC, PBegin

Re: Monitoring performance for releases

On Thu, Jul 9, 2020 at 12:48 PM Maximilian Michels wrote: > Not yet, I just learned about the migration to a new frontend, including > a new backend (InfluxDB instead of BigQuery). > > > - Are the metrics available on metrics.beam.apache.org? > > Is http://metrics.beam.apache.org online? I was

Re: [Proposal] Supporting Python Annotation Typehints on PTransform

you have any suggestions for > improvements. Thanks! > > Best, > Saavan > > --- > > Supporting Python Annotations on PTransforms > > https://issues.apache.org/jira/browse/BEAM-10258 > > Author: Saavan Nanavati (saa...@google.com ) > > Reviewer: Udi

Re: [Proposal] Adding Python Coverage Reports To CI/CD

r >> tests need revision. >> > >> > You can view/comment on the proposal here, or the full text of the >> proposal at the end of this email. Let me know what you think, or if there >> are any suggestions for improvements. Thanks! >> > >&g

Re: Commands to detect style issues quickly before sending PR

Another tip for Python: Also "pip install pre-commit" will run yapf and pylint on changed files during "git commit". On Fri, Jun 26, 2020 at 4:03 PM Valentyn Tymofieiev wrote: > See also: > https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-Formatting > > > Task

Re: [ANNOUNCE] New PMC Member: Alexey Romanenko

Congratulations! On Tue, Jun 16, 2020 at 9:33 AM rahul patwari wrote: > Congrats Alexey! > > On Tue, Jun 16, 2020 at 9:27 PM Ismaël Mejía wrote: > >> Please join me and the rest of Beam PMC in welcoming Alexey Romanenko as >> our >> newest PMC member. >> >> Alexey has significantly contributed

Re: Automation for Jira

Interesting: you could consider the JIRA as active as long as the linked PRs are open. On Mon, Jun 15, 2020 at 2:28 PM Luke Cwik wrote: > One thing I noticed is that links being added to issues automatically > (e.g. a PR is opened that tags something) doesn't reset the activity > counter so

Re: Python2.7 Beam End-of-Life Date

+1 On Mon, Jun 15, 2020 at 4:27 PM Ahmet Altay wrote: > As a concrete proposal, could we commit to removing python 2 support by > 2.24? In other words, mark the next release 2.23 as the last python 2 > compatible Beam version. > > On Mon, Jun 15, 2020 at 2:09 PM Valentyn Tymofieiev > wrote: >

Re: DISCUSS: FnAPI proto stabiliization

I'm not very familiar with this effort. Were there ITs / POCs created for these changes? (to surface any obvious bugs) Are these changes usable in DirectRunner? On Fri, Jun 12, 2020 at 8:50 AM Luke Cwik wrote: > A few months back there was a discussion[1] about performing work to > stabilize

Re: Beam Jenkins Migration

This is great! Looking forward to it. Would any metrics need to be migrated over to the new Jenkins? http://metrics.beam.apache.org/ On Fri, Jun 12, 2020 at 9:57 AM Tyson Hamilton wrote: > Very exciting! Thanks for the advanced notice Damian. > > On Fri, Jun 12, 2020 at 7:58 AM Damian

Re: python precommit error - google-auth depenedency?

gt;> >>>> Could we add it as a test dependency? Or if that is not possible, add >>>> it but remove it before next release? >>>> >>>> It seems like there is a release PR on google-auth ( >>>> https://github.com/googleapis/google-auth-library-pyth

Re: python precommit error - google-auth depenedency?

Yes you're right, Py2 envs are still using 4.0. On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay wrote: > > > On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri wrote: > >> 4.1 drops Python 2 support, so I'm not sure if we're ready for that yet. >> > > Wouldn't tha

Re: python precommit error - google-auth depenedency?

4.1 drops Python 2 support, so I'm not sure if we're ready for that yet. On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay wrote: > Looks like there is an attempt to fix this: > https://github.com/googleapis/google-auth-library-python/pull/524 > > On Wed, Jun 10, 2020 at 2:07 PM Udi

Re: python precommit error - google-auth depenedency?

On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay wrote: > > > On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles wrote: > >> You may be interested in following https://github.com/pypa/pip/issues/988 if >> you are not already. >> >> Kenn >> >> O

Re: python precommit error - google-auth depenedency?

Seems like manually installing rsa==4.0 satisfies deps, but pip doesn't do transitive deps well. Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in setup.py? On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri wrote: > Thanks, that helped in an unexpected way. :) > I shou

Re: python precommit error - google-auth depenedency?

> You could try installing and calling pipdeptree [1] from a Jenkins job, > and see if it helps. > > [1] https://pypi.org/project/pipdeptree/ > On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri wrote: > >> Hi, >> I'm trying to understand these "pip check" failure

python precommit error - google-auth depenedency?

Hi, I'm trying to understand these "pip check" failures: ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console However, when I do pip install

Kafka IO performance tests leaving behind disks on GCP

Hi, I opened a bug on what seems to be leftover GKE disk images: https://issues.apache.org/jira/browse/BEAM-10145 Can anyone familiar with these take a look? Thanks! smime.p7s Description: S/MIME Cryptographic Signature

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

That's great! Can someone please make custom emojis for our Slack from the expression sheet images? :) ( https://github.com/apache/beam/blob/master/website/www/site/static/images/mascot/model_sheet.png ) A smiling firefly and one with heart eyes would be awesome. On Wed, Jun 3, 2020 at 11:04 AM

Re: latest release on Github

ating Github releases [1], short of that however we can just use the >> UI. >> >> [1] >> https://lists.apache.org/thread.html/re367d262333078501c47856e9b8a0fc3fd7db60c2d2ebb181275481a%40%3Cdev.beam.apache.org%3E >> >> >> On Mon, Jun 1, 2020 at 3:07 PM Udi Meir

latest release on Github

Hi, Trying out the new Github layout (feature preview) I've noticed that the latest release is featured more prominently and is at v2.16.0 for some reason. https://github.com/apache/beam/releases/tag/v2.16.0 Any idea how to fix this, and what instructions should be added to the release guide?

Re: [ANNOUNCE] Beam 2.21.0 Released

Woohoo! On Thu, May 28, 2020 at 4:16 AM Kyle Weaver wrote: > The Apache Beam team is pleased to announce the release of version 2.21.0. > > Apache Beam is an open source unified programming model to define and > execute data processing pipelines, including ETL, batch and stream > (continuous)

Re: [ANNOUNCE] New committer: Robin Qiu

Congratulations Robin! On Tue, May 19, 2020, 10:15 Valentyn Tymofieiev wrote: > Congratulations, Robin! > > On Tue, May 19, 2020 at 9:10 AM Yichi Zhang wrote: > >> Congrats Robin! >> >> On Tue, May 19, 2020 at 8:56 AM Kamil Wasilewski < >> kamil.wasilew...@polidea.com> wrote: >> >>> Congrats!

Re: Beam 2.21 release update

distribution.run_command(command) > File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command > cmd_obj.run() > File "setup.py", line 235, in run > gen_protos.generate_proto_files() > File > "/src/streamingpla

Re: Beam 2.21 release update

is cannot happen automatically when >> running python3.6 setup.py sdist bdist_wheel ? >> > > I _believe_ this should happen automatically here: > https://github.com/apache/beam/blob/master/sdks/python/gen_protos.py#L365. > Maybe there is a problem there? > > >> &g

Re: Beam 2.21 release update

Probably not the issue, but double checking: are you running "pip install -r sdks/python/build-requirements.txt" first? On Wed, May 6, 2020 at 7:22 PM Thomas Weise wrote: > I'm working on rebasing our fork to 2.21.0 and run into a problem > installing grpcio-tools that leads to

Re: Python 3.7 docker container fails to build

the log above: > > > > --info tag was passed to docker image build commands with PythonDocker > > Precommit to capture more logs. Without the tag, errors from DockerFile > > step are not printed out to the console. > > > > On Thu, Apr 30, 2020 at 11:19 AM Udi Meiri

Re: Python 3.7 docker container fails to build

I checked node 8 and it had over 40GB space available. Does your job require more than that? Long term, I'm thinking we could clean up workspaces for successful jobs. This should free up additional space (I guess at least 100GB). https://plugins.jenkins.io/ws-cleanup/ - we already use this plugin

Jenkins test triggers

Hi, Has anyone noticed any changes today (since ~6h ago) to how tests are triggered on PRs? Are they triggering always, some of the time, not at all? Are phrase comment triggers working? Thanks. (background: https://issues.apache.org/jira/browse/INFRA-19836) smime.p7s Description: S/MIME

Re: How to submit PRs for dependant changes?

(a) or (c) should work. (c) is preferred if you want faster reviews. For multiple JIRAs, I've seen both [BEAM-123,BEAM-456] and [BEAM-123][BEAM-456] formats. One of them works but I'm not sure which. :D You can always manually add a PR to a JIRA. On Sun, Apr 26, 2020 at 2:49 PM Reuven Lax

Re: Jenkins jobs not running for my PR 10438

Alexey, what you're doing should be working (commits should trigger tests, as should "retest this please" and other phrases). https://issues.apache.org/jira/browse/INFRA-19836 tracks this issue On Tue, Apr 28, 2020 at 10:04 AM Alexey Romanenko wrote: > Does anyone know the “golden rule” how to

Re: Jira PR links not being generated?

s file last week, and now we do this week. > https://github.com/apache/beam/pull/11541 > > On Mon, Apr 27, 2020 at 6:24 PM Udi Meiri wrote: > >> We had such a file for a short while but it was removed: >> https://github.com/apache/beam/pull/10645 >> I don't belie

Re: JIRA Committer Permissions

Should this step be added to our new committer guide? On Fri, Apr 24, 2020 at 6:21 PM Luke Cwik wrote: > I noticed that several committers only had contributor level permissions > and I went and updated your account permissions for the Beam project to be > committer level. Feel free to let me

Re: Jira PR links not being generated?

We had such a file for a short while but it was removed: https://github.com/apache/beam/pull/10645 I don't believe it contained any PR link settings though +Pablo Estrada On Mon, Apr 27, 2020 at 1:56 PM Kyle Weaver wrote: > I went ahead and filed https://issues.apache.org/jira/browse/BEAM-9833

Re: [ANNOUNCE] Beam 2.20.0 Released

You'll need to add --tags On Fri, Apr 24, 2020 at 11:53 AM Jan Lukavský wrote: > Hm, that is strange: > > ~/git/apache/beam$ git status > On branch master > Your branch is up to date with 'origin/master'. > ~/git/apache/beam$ git pull > Already up to date. > ~/git/apache/beam$ git tag | grep

Re: Website publish jobs fail recently

elease. >> >> On Tue, Apr 14, 2020 at 9:01 PM Kenneth Knowles wrote: >> >>> Indeed, publish jobs have write access to things, which normal builds do >>> not. >>> >>> I suggest reaching out to bui...@apache.org >>> >>>

Re: Website publish jobs fail recently

enneth Knowles wrote: > >> Indeed, publish jobs have write access to things, which normal builds do >> not. >> >> I suggest reaching out to bui...@apache.org >> >> Kenn >> >> On Tue, Apr 14, 2020 at 5:58 PM Udi Meiri wrote: >> >>>

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

If this process is used in releases we would benefit from running it regularly to ensure it isn't broken and thus delay releases (and add work for the release manager). Does it make sense to put it in postcommit? On Wed, Apr 15, 2020 at 2:30 PM Kyle Weaver wrote: > Looks like the same error as

Re: Website publish jobs fail recently

Hey, I was looking at this today but could not figure it out. The machines we run the publish jobs probably vary from our regular apache-beam-testing Jenkins ones. I tried researching all the reasons why this might be happening but came up empty. On Tue, Apr 14, 2020 at 10:19 AM Kyle Weaver

Re: Implementing type hints on multi-output PTransforms

PROPSAL] ..." to this mailing list describing what you want to change should be enough. A document could also work; I'm not aware of one that touches on this. Your PR would need to have an associated JIRA (feel free to take over any of the ones I've mentioned). > Best, > Joshua > > On

Re: [DISCUSS] Let's establish a guideline for using Python type annotations in Beam codebase

I agree with Robert to only put Any where it really can be any type. I'm not sure how much typing we should add. At minimum: external APIs and wherever mypy complains. Ideally I would like to have annotations everywhere, because this reduces uncertainty when modifying existing code. You are

Re: Implementing type hints on multi-output PTransforms

Hi Joshua, I've been working on type hints recently. Your issue is similar to this: https://issues.apache.org/jira/browse/BEAM-8782 (exactly the same if tags are passed to with_outputs() in the example). There's also this related bug about type inference:

Re: [BEAM-9322] Python SDK discussion on correct output tag names

On Thu, Mar 26, 2020 at 10:13 AM Luke Cwik wrote: > The issue seems to be that a PCollection can have a "tag" associated with > it and PTransform expansion can return an arbitrary nested dictionary/tuple > yet we need to figure out what the user wanted as the local name for the > PCollection

Re: Are docker image tags shared within a jenkins worker?

The Python HDFS IT uses the jenkins BUILD_TAG to create unique names: PROJECT_NAME=$(echo hdfs_IT-${BUILD_TAG:-non-jenkins}) The BUILD_TAG is unique and easily traced back to the Jenkins job that made it. It might need some sanitizing though if it contains any invalid characters. On Tue, Mar 24,

Re: Hello Beam Community!

Welcome! On Fri, Mar 13, 2020 at 9:47 AM Yichi Zhang wrote: > Welcome! > > On Fri, Mar 13, 2020 at 9:40 AM Ahmet Altay wrote: > >> Welcome Brittany! >> >> On Thu, Mar 12, 2020 at 6:32 PM Brittany Hermann >> wrote: >> >>> Hello Beam Community! >>> >>> My name is Brittany Hermann and I

Re: Install Jenkins AnsiColor plugin

> seem to be some ansi plugin already installed but I couldn’t get it to > work. > > -chad > > > On Sun, Mar 8, 2020 at 10:52 AM Ismaël Mejía wrote: > >> Did this ever happen? If not what is blocking it? >> >> >> >> On Tue, Oct 22, 2019 at

Re: Python Static Typing: Next Steps

Off-topic: Python lint via pre-commit should be much faster. (I wrote my own modified-file-only lint in the past) On Mon, Mar 2, 2020 at 2:08 PM Kyle Weaver wrote: > > Python lint takes 4-5mins to complete. I think if the mypy analysis is > really on the order of 10s, the additional time won't

Re: Python Static Typing: Next Steps

Let's go forward with this and see. I volunteer to help as well. I believe that mypy via pre-commit hook will be faster than 10s since it only applies to modified files. On Mon, Mar 2, 2020 at 10:53 AM Robert Bradshaw wrote: > +1 > > We should enable this on jenkins, plus trivial instructions

Re: [ANNOUNCE] New Committer: Kamil Wasilewski

Welcome Kamil! On Fri, Feb 28, 2020 at 12:53 PM Mark Liu wrote: > Congrats, Kamil! > > On Fri, Feb 28, 2020 at 12:23 PM Ismaël Mejía wrote: > >> Congratulations Kamil! >> >> On Fri, Feb 28, 2020 at 7:09 PM Yichi Zhang wrote: >> >>> Congrats, Kamil! >>> >>> On Fri, Feb 28, 2020 at 9:53 AM

Re: [DISCUSS] How many Python 3.x minor versions should Beam Python SDK aim to support concurrently?

I agree with having low-frequency tests for low-priority versions. Low-priority versions could be determined according to least usage. On Wed, Feb 26, 2020 at 4:06 PM Robert Bradshaw wrote: > On Wed, Feb 26, 2020 at 3:29 PM Kenneth Knowles wrote: > > > > Are these divergent enough that they

Re: [ANNOUNCE] New committer: Chad Dombrova

Congrats and welcome, Chad! On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada wrote: > Hi everyone, > > Please join me and the rest of the Beam PMC in welcoming a new committer: > Chad Dombrova > > Chad has contributed to the project in multiple ways, including > improvements to the testing

Re: jira search in chrome omnibox

create the > page Udi and I will add the Firefox section if you agree. > > > > > On Fri, Aug 31, 2018 at 2:31 AM Udi Meiri wrote: > >> Correction: this is the correct URL: >> https://issues.apache.org/jira/secure/QuickSearch.jspa?searchString=%s >> >> It u

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

0 at 1:17 PM jincheng sun >> wrote: >> >>> I have verified that this issue could be reproduced in my local >>> environment (MacOS) and the solution suggested by Udi could work! >>> >>> Best, >>> Jincheng >>> >>> Udi Meiri 于2020年2月11日周

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

I don't have those issues (running on Linux), but a possible workaround could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh. On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar wrote: > Hello Beamers. > > Just curious does anyone having trouble running >

Re: Labels on PR

Cool! On Mon, Feb 10, 2020 at 9:27 AM Robert Burke wrote: > +1 to autolabeling > > On Mon, Feb 10, 2020, 9:21 AM Luke Cwik wrote: > >> Nice >> >> On Mon, Feb 10, 2020 at 2:52 AM Alex Van Boxel wrote: >> >>> Ha, cool. I'll have a look at the autolabeler. The infra stuff is not >>> something

Re: [DISCUSS] Autoformat python code with Black

as to just notice every PR that was affected by the >> change. >> >>>>>> And clearly document how to validate and autoformat the code. >> >>>>>> >> >>>>>> So the earlier the better, go go autoformat! >> >>

Re: [RELEASE VOTE RESULT] Release 2.19.0, release candidate #1

Thank you Boyuan! On Mon, Feb 3, 2020 at 3:40 PM Ahmet Altay wrote: > On Mon, Feb 3, 2020 at 1:22 PM Thomas Weise wrote: > >> Impressive, probably the fastest/smoothest Beam release so far. >> > > I agree! Thank you, Boyuan! > > >> >> On Mon, Feb 3, 2020 at 10:45 AM Boyuan Zhang wrote: >> >>>

Re: [DISCUSSION] Improve release notes by adding a change list file

+1 to add this to the checklist On Mon, Feb 3, 2020 at 4:57 PM Robert Bradshaw wrote: > On Mon, Feb 3, 2020 at 4:49 PM Ahmet Altay wrote: > > > > On Mon, Feb 3, 2020 at 2:09 PM Robert Bradshaw > wrote: > >> > >> I would suggest we start with the simpler single file. If merge > >> conflicts

Re: Jenkins jobs not running for my PR 10438

done On Fri, Jan 31, 2020 at 9:07 AM Tomo Suzuki wrote: > HI Beam committers, > > Would you re-trigger the 2 failed checks in > https://github.com/apache/beam/pull/10714 ? > Run Java PreCommit > Run Java_Examples_Dataflow PreCommit > > > On Fri, Jan 31, 2020 at 7:51 AM Rehman Murad Ali < >

Re: [ANNOUNCE] New committer: Hannah Jiang

Welcome and congrats Hannah! On Tue, Jan 28, 2020 at 4:52 PM Robin Qiu wrote: > Congratulations, Hannah! > > On Tue, Jan 28, 2020 at 4:50 PM Alan Myrvold wrote: > >> Congrats, Hannah >> >> On Tue, Jan 28, 2020 at 4:46 PM Connell O'Callaghan >> wrote: >> >>> Thank you for sharing Luke!!!

anyone working on updating com.google.cloud:google-cloud-spanner?

It's currently at 1.6.0, which a year old. https://github.com/googleapis/java-spanner/releases?after=1.9.0 I would appreciate any help with this. Tracking issues: https://issues.apache.org/jira/browse/BEAM-8758 https://issues.apache.org/jira/browse/BEAM-8682 smime.p7s Description: S/MIME

[ANNOUNCE] Beam 2.18.0 Released

. -- Udi Meiri, on behalf of The Apache Beam team

  1   2   3   4   >