Re: Beam 2.21 release update

I'm working on rebasing our fork to 2.21.0 and run into a problem installing grpcio-tools that leads to *ModuleNotFoundError: No module named 'grpc_tools' *(see details below) I cannot reproduce this locally. Any =suggestions on what to look for? Thanks, Thomas [?25hBuilding wheels for

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

On Wed, May 6, 2020 at 2:33 PM Aizhamal Nurmamat kyzy wrote: > >> > 1) Currently, the main blocker for merging is Staging Test Failures. >> >> That and finishing the review. (Is someone tracking/coordinating this?) >> > > I am coordinating the work on the failed tests, but I would need other >

Re: [Proposal] Apache Beam Fn API - GCP IO Debuggability Metrics

Thanks everyone so far for taking a look so far :). I am hoping to have this finalize the two reviews by the end of next week, May 15th. I'll continue to follow up on feedback and make changes, and I will add some more mentions to the documents to draw attention

Re: Google Summer of Code 2020 [Accepted Proposal]

Congrats! On Tue, May 5, 2020 at 5:07 PM Rui Wang wrote: > Welcome! > > > -Rui > > On Tue, May 5, 2020 at 1:58 PM Chamikara Jayalath > wrote: > >> Congrats! >> >> On Tue, May 5, 2020 at 1:34 PM Robert Bradshaw >> wrote: >> >>> Congratulations and welcome! >>> >>> On Tue, May 5, 2020 at 12:46

Re: Python Static Typing: Next Steps

Just an update on this: we just merged https://github.com/apache/beam/pull/11620 which enforces typechecking for all files that currently pass. On Tue, Mar 3, 2020 at 1:12 PM Chad Dombrova wrote: >> >> This probably does not apply yet, does optional mean that opting-in for all >> or none of the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

> > > > 1) Currently, the main blocker for merging is Staging Test Failures. > > That and finishing the review. (Is someone tracking/coordinating this?) > I am coordinating the work on the failed tests, but I would need other committer's help to perform the review. @Ahmet, could you help us

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

On Wed, May 6, 2020 at 1:01 PM Aizhamal Nurmamat kyzy wrote: > > Hi all, > > Nam and Brian have been working together on blog post names and the decision > was to keep them as they are for now, because Hugo doesn’t fully support the > feature that was mentioned earlier [1]. Also I believe this

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

I believe the troublesome tasks are these three: https://github.com/apache/beam/blob/2592c8396ae1a68dadc351135c2a79397fb70942/website/build.gradle#L95-L108 (to avoid confusion, note that these are being added as part of the PR, although they appear in a commit on apache/beam in the link). Nam,

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

I added some detail about the failures in the PR [1]. It seems for some reason this PR is causing those directories (node_modules and themes) to be created with root ownership. All subsequent runs of those jobs on the same worker (for other PRs as well) try to clean up the old workspace, and they

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

Hi all, Nam and Brian have been working together on blog post names and the decision was to keep them as they are for now, because Hugo doesn’t fully support the feature that was mentioned earlier [1]. Also I believe this can be done after merging the PR. 1) Currently, the main blocker for

[GitHub] [beam-site] ibzib opened a new pull request #603: [DO NOT MERGE] Update beam-site for release 2.21.0\n\nContent generated based on commit

ibzib opened a new pull request #603: URL: https://github.com/apache/beam-site/pull/603 This PR should not be merged until the 2.21 release is finalized. I'm just leaving this PR here in case I can use it later. This is an

Re: Builtin IOs - Link to Java/Pydoc instead of code?

You can link to current, I believe: https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/FileSystems.html On Wed, May 6, 2020 at 12:13 PM Robert Bradshaw wrote: > On Wed, May 6, 2020 at 11:15 AM Chamikara Jayalath > wrote: > > > > Can't we use something like the "{{

Re: Builtin IOs - Link to Java/Pydoc instead of code?

On Wed, May 6, 2020 at 11:15 AM Chamikara Jayalath wrote: > > Can't we use something like the "{{ site.release_latest }}" tag used in > Website docs ? > https://github.com/apache/beam/blob/master/website/src/get-started/quickstart-java.md +1. It would be good to have a solution that works

Re: Builtin IOs - Link to Java/Pydoc instead of code?

Can't we use something like the "{{ site.release_latest }}" tag used in Website docs ? https://github.com/apache/beam/blob/master/website/src/get-started/quickstart-java.md On Wed, May 6, 2020 at 11:06 AM Robert Bradshaw wrote: > E.g. if I google "apache beam javadoc", the first link is for

Re: [PROPOSAL] Preparing for Beam 2.22.0 release

Thank you, Brian! -Rui On Wed, May 6, 2020 at 9:59 AM Ahmet Altay wrote: > Thank you, Brian. > > On Mon, May 4, 2020 at 1:59 PM Luke Cwik wrote: > >> Thanks Brian. >> >> On Mon, May 4, 2020 at 10:57 AM Kyle Weaver wrote: >> >>> Thanks Brian! >>> >>> I'd also like to remind everyone to

Re: Builtin IOs - Link to Java/Pydoc instead of code?

E.g. if I google "apache beam javadoc", the first link is for version 2.1.0 (no, that's not 2.21.0), the second for a (raw-looking) directory index of javadocs by version, the third for Beam 2.5.0, and the fourth for a class in 2.2.0. On Wed, May 6, 2020 at 11:02 AM Robert Bradshaw wrote: > >

Re: Builtin IOs - Link to Java/Pydoc instead of code?

One problem is that there's no way to link to the "latest" version of the docs, or toggle between them. What links to docs we have are pointing to specific (often very old or out of date) docs. If we can solve this that'd make me more in favor of this proposal. On Wed, May 6, 2020 at 9:23 AM

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

On Mon, May 4, 2020 at 7:07 PM Ahmet Altay wrote: > >> On Mon, May 4, 2020 at 6:30 PM Robert Bradshaw wrote: >>> >>> I took the massive commit and split it up into: >>> >>> (1) Infrastructure changes (basically everything outside of >>> (website/www/site/content) >>> (2) Sed script changes, and

Re: [Proposal] Apache Beam Fn API - GCP IO Debuggability Metrics

Thanks, also took a look and left some comments. On Tue, May 5, 2020 at 6:24 PM Alex Amato wrote: > Hello, > > I created another design document. This time for GCP IO Debuggability > Metrics. Which defines some new metrics to collect in the GCP IO libraries. > This is for monitoring request

Re: Support for AWS SDK v2 and enhanced fanout in KinesisIO

I’d like to get back to this question, fairly raised by Jonothan a while ago, since actually it affects not only KinesisIO but also all other AWS IOs in general, that uses AWS SDK of two different versions. My personal opinion - I’d strongly like to avoid a copy-pasted code for two different

Re: [Proposal] Apache Beam Fn API - Histogram Style Metrics (Correct link this time)

Thanks Alex, I had some minor comments. On Mon, May 4, 2020 at 4:04 PM Alex Amato wrote: > Thanks Ismaël :). Done > > On Mon, May 4, 2020 at 3:59 PM Ismaël Mejía wrote: > >> Moving the short link to this thread >> https://s.apache.org/beam-histogram-metrics >> >> Alex can you add this link

Re: [PROPOSAL] Preparing for Beam 2.22.0 release

Thank you, Brian. On Mon, May 4, 2020 at 1:59 PM Luke Cwik wrote: > Thanks Brian. > > On Mon, May 4, 2020 at 10:57 AM Kyle Weaver wrote: > >> Thanks Brian! >> >> I'd also like to remind everyone to update CHANGES.md with any important >> recent changes that might be missing. >>

Re: [DISCUSS] Dealing with @Ignored tests

Good point. The raw numbers are available in the test run output. See https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PreCommit_Java_Cron/2718/testReport/ for the "skipped" column. And you get the same on console or Gradle Scan:

Re: Builtin IOs - Link to Java/Pydoc instead of code?

+1 On Wed, May 6, 2020 at 9:11 AM Alexey Romanenko wrote: > +1 It would make more sense to point to the latest release Javadoc of IO > > On 6 May 2020, at 06:56, Chamikara Jayalath wrote: > > +1 for linking to the more readable version. > > - Cham > > On Tue, May 5, 2020, 9:26 AM Brian Hulette

Re: Builtin IOs - Link to Java/Pydoc instead of code?

+1 It would make more sense to point to the latest release Javadoc of IO > On 6 May 2020, at 06:56, Chamikara Jayalath wrote: > > +1 for linking to the more readable version. > > - Cham > > On Tue, May 5, 2020, 9:26 AM Brian Hulette > wrote: > +1 for doc links

Re: [DISCUSS] finishBundle once per window

This is a great idea. I thought that (long ago) we decided not to execute finishBundle per window for these reasons: (1) perf fears (2) naming bikeshed (3) backwards compatibility (even though we hadn't stabilized, it is pervasive). That was before the annotation-driven DoFn I believe, so we