Need maintainer permission of PyPI apache-beam package

2021-05-18 Thread Heejong Lee
Hi, I'm currently working on Beam 2.30.0 release and need help adding myself to the maintainer group of PyPI apache-beam package. My PyPI username is 'ihji'. Does anybody have permission for adding a new member to the apache-beam maintainer group? Thanks!

Re: Compatibility Check Badges need attention

2021-05-18 Thread Valentyn Tymofieiev
Looks like they were added in https://github.com/apache/beam/pull/8791. +Lily Li do you know by chance if this tooling is still maintained? On Tue, May 18, 2021 at 2:27 PM Brian Hulette wrote: > Hi all, > I just noticed that the two "compatibility check" badges on > github.com/apache/beam are

Flaky test issue report

2021-05-18 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests. These are P1 issues because they have a major negative impact on the community and make it hard to determine the quality of the software. BEAM-12322: FnApiRunnerTestWithGrpcAndMultiWorkers flaky (py precommit)

P1 issues report

2021-05-18 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests. See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the meaning and expectations around P1 issues. BEAM-12352: GcsIOIntegrationTest.test_copy_rewrite_token and

Re: Question about SplittableDoFn

2021-05-18 Thread Miguel Anzo Palomo
Boyuan Zhang, It's about this issue , the code can be found here On Tue, May 18, 2021 at 5:29 PM Boyuan Zhang wrote: > Would you like to share your draft code? Iterating on the code might be >

Re: Question about SplittableDoFn

2021-05-18 Thread Boyuan Zhang
Would you like to share your draft code? Iterating on the code might be easier to figure out the issue. On Tue, May 18, 2021 at 3:28 PM Robert Burke wrote: > IIRC the Initial Restrictions method gives you an element and you return > the restrictions relative to that element. > > It's entirely

Re: Question about SplittableDoFn

2021-05-18 Thread Robert Burke
IIRC the Initial Restrictions method gives you an element and you return the restrictions relative to that element. It's entirely appropriate to stat files or query databases in order to determine the initial restrictions and partitions of the data. On Tue, May 18, 2021, 3:21 PM Miguel Anzo

Question about SplittableDoFn

2021-05-18 Thread Miguel Anzo Palomo
Hi, I’m looking at how to implement a reader as a SplittableDoFn and I'm having some problems with the initial restriction, specifically, how do you set the initial restriction if you don’t know the size of the data? The DoFn that I'm working on takes a PCollection of Spanner *ReadOperations *and

Compatibility Check Badges need attention

2021-05-18 Thread Brian Hulette
Hi all, I just noticed that the two "compatibility check" badges on github.com/apache/beam are in a bad state [1,2]. Does anyone have context on these? What do we need to do to fix them? It looks like it may be a configuration issue. Some of the complaints are related to python 2. It's also

Re: BEAM-11838: An x-lang wrapper for DebeziumIO

2021-05-18 Thread Benjamin Gonzalez Delgado
Hi Pablo, It runs perfectly with your changes, I will make some fixes on the verifiers and changes for minutesToRun, I will upload a PR in a couple of hours. Thanks a lot, you really helped me! > -- *This email and its contents (including any attachments) are being sent to you on the condition

Re: pb2 file generation

2021-05-18 Thread Miguel Hernández Sandoval
Thanks for your help Tyson/Alex, this worked just as expected. Thanks a lot! On Mon, May 17, 2021 at 6:50 PM Alex Amato wrote: > Hmm, I don't recall if I needed to include the generated files in my PRs > or not, when i modified these in the past > You should be able to run this from the python

Re: New Contributor - Jira Access

2021-05-18 Thread Alexey Romanenko
Hi Heidi, I added you as a contributor, feel free to assign this Jira for yourself. Welcome! — Alexey > On 18 May 2021, at 11:04, Heidi Hurst wrote: > > Hi all - > > I'm Heidi, new to the Beam ecosystem. I'd like to contribute to a small > improvement in the Python SDK (Jira ticket [1])

New Contributor - Jira Access

2021-05-18 Thread Heidi Hurst
Hi all - I'm Heidi, new to the Beam ecosystem. I'd like to contribute to a small improvement in the Python SDK (Jira ticket [1]) to allow WithKeys to take function args & kwargs. Could I be added as a contributor to the Beam issue tracker please? Username is heidi_pachama. Thanks (and thanks