Re: Full stream-stream join semantics

Have you seen the Join library that is part of schemas? I'm curious whether this fits your needs, or there's something lacking there. On Fri, Nov 22, 2019 at 12:31 AM Jan Lukavský wrote: > Hi, > > based on roadmap [1], we would like to define and implement a full set > of (unified)

Re: Python tests and setup

I resolved my Python3.6 problem by installing Python3.6 from source: https://issues.apache.org/jira/browse/BEAM-8787#comment-16979785 On Mon, Nov 18, 2019 at 1:49 PM Elliotte Rusty Harold wrote: > and probably related to that: > > https://github.com/pyinstaller/pyinstaller/issues/4064 > > On

Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

On Thu, Nov 21, 2019 at 7:05 PM David Cavazos wrote: > > > I created this Google Form > > if everyone is okay with it to make it easier to both vote and view the > results :) >

Re: Full stream-stream join semantics

On Fri, Nov 22, 2019 at 10:19 AM Jan Lukavský wrote: > Hi Reuven, > > I didn't investigate that particular one, but looking into that now, it > looks that is (same as the "classic" join library) builds around CoGBK. Is > that correct? If yes, then it essentially means that it: > - works only

Re: Python tests and setup

I've been looking at using pyenv in the past few days. I've also had a broken 3.6.8 install on my machine for a while now, but using pyenv I managed to install 3.6.9 with little difficulty: pyenv install 3.6.9 PYENV_VERSION=3.6.9 ./gradlew :sdks:python:test-suites:tox:py36:setupVirtualenv (You

Re: Full stream-stream join semantics

Hi Reuven, I didn't investigate that particular one, but looking into that now, it looks that is (same as the "classic" join library) builds around CoGBK. Is that correct? If yes, then it essentially means that it:  - works only for cases where both sides have the same windowfn (that is

Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

My vote: [X] Beaver [ ] Hedgehog [ ] Lemur [X] Owl [ ] Salmon [ ] Trout [ ] Robot dinosaur [X] Firefly [ ] Cuttlefish [X] Dumbo Octopus [ ] Angler fish

Re: GCP libraries up-to-date versions in Java

I couldn't think of a good flow that didn't lead me to clearing org.apache.beam artifacts in .m2 before running the analysis. There might be a way to override the maven local path in Gradle so that it publishes to a temporary directory but it wasn't obvious how to do this from Gradles maven

Re: Full stream-stream join semantics

On 11/22/19 7:54 PM, Reuven Lax wrote: On Fri, Nov 22, 2019 at 10:19 AM Jan Lukavský > wrote: Hi Reuven, I didn't investigate that particular one, but looking into that now, it looks that is (same as the "classic" join library) builds around CoGBK.

Re: Pipeline AttributeError on Python3

We have not seen the issue with Python 3.6 on 2.16+ after applying this patch.  Thanks! On Thu, Nov 21, 2019 at 4:41 PM Thomas Weise wrote: > We are currently verifying the patch. Will report back tomorrow. > > On Thu, Nov 21, 2019 at 8:40 AM Valentyn Tymofieiev > wrote: > >> That would be

Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

[ ] Beaver [ ] Hedgehog [ ] Lemur [ ] Owl [X] Salmon [X] Trout [ ] Robot dinosaur [X] Firefly [ ] Cuttlefish [ ] Dumbo Octopus [ ] Angler fish

Re: [Portability] Turn off artifact staging?

I'm running into the issue Kyle points out when I try to run a pipeline that does not use artifact staging: 2019-11-23 01:09:18,442 WARN org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService - GetManifest for

Re: [UPDATE] Preparing for Beam 2.17.0 release

https://github.com/apache/beam/pull/10197 will cut some time required to run python unit tests on release branch. There were other recent improvements to Python tests on master, but I think they also happened after a release was cut. On Fri, Nov 22, 2019 at 3:46 PM Mikhail Gryzykhin wrote: >

Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

[ ] Beaver [ ] Hedgehog [ ] Lemur [X] Owl [ ] Salmon [ ] Trout [ ] Robot dinosaur [ ] Firefly [ ] Cuttlefish [ ] Dumbo Octopus [ ] Angler fish On Fri, Nov 22, 2019 at 11:33 PM Andrew Pilloud wrote: > > [ ] Beaver > [ ] Hedgehog > [ ] Lemur > [ ] Owl > [X] Salmon > [X] Trout > [ ] Robot

Re: Full stream-stream join semantics

+Mikhail Gryzykhin +Rui Wang +Reza Rokni who have all done some investigations here. On Fri, Nov 22, 2019 at 11:48 AM Jan Lukavský wrote: > > On 11/22/19 7:54 PM, Reuven Lax wrote: > > > > On Fri, Nov 22, 2019 at 10:19 AM Jan Lukavský wrote: > >> Hi Reuven, >> >> I didn't investigate that

Re: [UPDATE] Preparing for Beam 2.17.0 release

UPD: on current branch there's timeout on gradle build job, I'm mitigating it by increasing job time. Seems that this job runs most of python tests. We might look into adjusting the target. Second failure is https://issues.apache.org/jira/browse/BEAM-8812 . I would really appreciate if someone

Re: [UPDATE] Preparing for Beam 2.17.0 release

Hi Mikhail,I created PR for [BEAM-8812]. It is linked in the JIRA.JanDne 23. 11. 2019 0:45 napsal uživatel Mikhail Gryzykhin :UPD:on current branch there's timeout on gradle build job, I'm mitigating it by increasing job time. Seems that this job runs most of python tests. We might look into

Re: Full stream-stream join semantics

I didn't want to go too much into detail, but to describe the idea roughly (ignoring the problem of different window fns on both sides to keep it as simple as possible): rhs -  \     flatten (on global window) stateful par do (sorted by event time)  output lhs

Re: [ANNOUNCE] New committer: Daniel Oliveira

Congrats Daniel! On 11/21/19 10:11 AM, Gleb Kanterov wrote: Congratulations! On Thu, Nov 21, 2019 at 6:24 AM Thomas Weise > wrote: Congratulations! On Wed, Nov 20, 2019, 7:56 PM Chamikara Jayalath mailto:chamik...@google.com>> wrote: Congrats!!

Full stream-stream join semantics

Hi, based on roadmap [1], we would like to define and implement a full set of (unified) stream-stream joins. That would include:  - joins (left, right, full outer) on global window with "immediate trigger"  - joins with different windowing functions on left and right side The approach

Re: [ANNOUNCE] New committer: Daniel Oliveira

Congratulations, Daniel! > On 22 Nov 2019, at 09:18, Jan Lukavský wrote: > > Congrats Daniel! > On 11/21/19 10:11 AM, Gleb Kanterov wrote: >> Congratulations! >> >> On Thu, Nov 21, 2019 at 6:24 AM Thomas Weise > > wrote: >> Congratulations! >> >> >> On Wed, Nov 20,