Congratulations!
On Fri, Sep 11, 2020, 5:54 PM Andrew Psaltis
wrote:
> Congrats!
>
> On Sat, Sep 12, 2020 at 7:43 AM Reza Rokni wrote:
>
>> Thanx everyone! Looking forward to being able to contribute more :-)
>>
>> On Sat, Sep 12, 2020 at 4:33 AM Valentyn Tymofieiev
>> wrote:
>>
>>>
+1 (binding)
Rebased fork and run internal performance tests.
While doing so, I run into the unit test issue below with the fn_runner
(Python direct runner), which did not occur with 2.21 [1]. That processing
time timers are not supported wasn't an issue previously, because the
timer, though
2.23 wasn't marked released in JIRA (which I just did). Please add the step
to the release guide if not already present.
https://issues.apache.org/jira/projects/BEAM/versions/12347145
On Thu, Jul 30, 2020 at 3:05 PM Kyle Weaver wrote:
> Hi Eleanore, there have been no changes to Beam's
It appears that there is coverage missing in the Grafana dashboards (it
could also be that I just don't find it).
For example:
https://apache-beam-testing.appspot.com/explore?dashboard=5751884853805056
The GBK and ParDo tests have a selection for {batch, streaming} and SDK. No
coverage for
Congrats!
On Tue, Jun 30, 2020 at 5:05 PM Aizhamal Nurmamat kyzy
wrote:
> Thanks everyone! It's been really nice to work with all of you :)
>
> On Tue, Jun 30, 2020, 1:45 PM Ismaël Mejía wrote:
>
>> Congratulations Aizhamal!
>> And thanks for all the dedication to the project.
>>
>> On Tue,
Congratulations!
On Tue, Jun 16, 2020 at 1:27 PM Valentyn Tymofieiev
wrote:
> Congratulations!
>
> On Tue, Jun 16, 2020 at 11:41 AM Ahmet Altay wrote:
>
>> Congratulations!
>>
>> On Tue, Jun 16, 2020 at 10:05 AM Pablo Estrada
>> wrote:
>>
>>> Yooohooo! Thanks for all your contributions and
+1
On Tue, Jun 9, 2020 at 9:41 AM Robert Bradshaw wrote:
> Makes sense to me.
>
> On Tue, Jun 9, 2020 at 8:45 AM Maximilian Michels wrote:
>
>> Thanks of the heads-up, Tyson! It's a sensible decision to remove
>> unsupported runners.
>>
>> -Max
>>
>> On 09.06.20 16:51, Tyson Hamilton wrote:
If all Python dependencies are pre-installed on the yarn container hosts,
then you can use the process environment to spawn processes, like so:
https://github.com/lyft/flinkk8soperator/blob/bb8834d69e8621d636ef2085fdc167a9d2c2bfa3/examples/beam-python/src/beam_example/pipeline.py#L16-L17
Thomas
>
> I think the "set_version.sh" script could be called in the release
> scripts to remove the -SNAPSHOT suffix on the release branch.
>
>
I would expect the release branch to have the next -SNAPSHOT version (not
the case currently):
Hi Jacek,
Most of the developer documentation can be found on the cwiki, in this case
https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips
To build just a runner, for example, Flink:
./gradlew :runners:flink:1.10:job-server:runShadow
Thomas
On Sat, May 23, 2020 at 10:29 AM Jacek
Please note https://github.com/apache/beam/pull/11777 - another bug we
found while trying to upgrade to 2.21
Other than that things look good.
On Thu, May 21, 2020 at 4:14 PM Ahmet Altay wrote:
> +1 (again, validated with new whl files.)
>
> What about
On Thu, May 21, 2020 at 4:08 PM Robert Bradshaw wrote:
> On Thu, May 21, 2020 at 3:55 PM Luke Cwik wrote:
>
>> The 2.22 release is also being worked on. Rather allow that release pick
>> up anything that isn't critical.
>>
>
> +1. The question is whether this will result in broken installs
ypa.io/en/stable/reference/pip/#pep-517-and-518-support
> It would help when using tools like pip ("pip wheel"), but I'm not sure
> what the alternative for "python setup.py sdist" is.
>
>
> On Thu, May 7, 2020 at 10:40 PM Thomas Weise wrote:
>
>> No additi
:
> It's hard to say without more details what's going on. Ahmet you're right
> that it installs build-requirements.txt and retries calling
> generate_proto_files().
>
> Thomas, were there additional stacktraces? (after a "During handling of
> the above exception, another excep
bly not the issue, but double checking: are you running "pip install
> -r sdks/python/build-requirements.txt" first?
>
> On Wed, May 6, 2020 at 7:22 PM Thomas Weise wrote:
>
>> I'm working on rebasing our fork to 2.21.0 and run into a problem
>> installing grpcio-tools that lea
I'm working on rebasing our fork to 2.21.0 and run into a problem
installing grpcio-tools that leads to *ModuleNotFoundError: No module named
'grpc_tools' *(see details below)
I cannot reproduce this locally.
Any =suggestions on what to look for?
Thanks,
Thomas
[?25hBuilding wheels for
fine with me as long as the old urls will work
>>>>> too.
>>>>>
>>>>> But do we need to change the filenames for the blog posts to
>>>>> accomplish that? It's nice that the blog post markdown files start with a
>>>>> date so the
+1 for removing the default runner. It has always been the Beam user
expectation that a runner needs to be selected.
"PortableRunner" isn't a runner (despite its name) - it's a proxy to a
runner that the user specifies via job_endpoint.
Thanks for cleaning this up!
On Thu, Apr 30, 2020 at 10:11
For changed URLs, will previous URLs be mapped to avoid broken external
links?
On Thu, Apr 30, 2020 at 9:34 AM Aizhamal Nurmamat kyzy
wrote:
> Hi,
>
> To give a little more context regarding the URLs, the date should still
> appear on the blog post, but not on the URL.
> For example, we'd
merged to 2.21.0.
>>>>
>>>> Thank you all for providing feedback and ideas.
>>>>
>>>>
>>>> On Thu, Apr 16, 2020 at 10:24 AM Kyle Weaver
>>>> wrote:
>>>>
>>>>> > I tried with multi processing and it imp
Here is the release tag: https://github.com/apache/beam/releases/tag/v2.20.0
On Fri, Apr 24, 2020 at 9:28 AM Kyle Weaver wrote:
> > Is is possible we are missing git tag for this release? I cannot find it.
>
> You mean https://github.com/apache/beam/tree/release-2.20.0?
>
> On Fri, Apr 24,
gt; Please let me know what you think and if there are other things can be
>>> improved.
>>>
>>> Hannah
>>>
>>>
>>>
>>> On Wed, Apr 15, 2020 at 2:30 PM Kyle Weaver wrote:
>>>
>>>> Looks like the same error as this J
The new feature to assemble licenses is very useful but appears to add
several minutes (7-8?) build time to jobs that need to build a container.
Does it also seem to cause occasional build failures?
https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Phrase/131/
Would it be possible
Thanks!
On Wed, Mar 4, 2020 at 1:56 PM Kyle Weaver wrote:
> (Link to previous thread on this issue:
> https://lists.apache.org/thread.html/r1e2640f6d06d0a09ed63cd7134423a390a6cd1332569869cba404df6%40%3Cdev.beam.apache.org%3E
> )
>
> On Wed, Mar 4, 2020 at 1:44 PM Thomas Weise
I run into this problem today and found that removing
https://oss.sonatype.org/content/repositories/staging/ from
buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy
also resolves the issue.
Is it possible that a flaky repository can poison the gradle cache? Do we
need this
Hi,
I run into the error shown below with local build.
The dependencies actually exist in the release repo, and the build is
successful when removing
https://oss.sonatype.org/content/repositories/staging
from
buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy
That raises the
Congratulations!
On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka wrote:
> Congratulations Chad!
>
> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne wrote:
>>
>>> Nice one Chad. Your typing efforts are very welcomed.
>>>
>>>
Congratulations!
On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía wrote:
> Congrats Jincheng!
>
> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng wrote:
>>
>>> Congratulations Jincheng, well deserved!
>>>
>>> Best,
>>>
Congratulations!
On Tue, Feb 18, 2020 at 8:33 AM Ismaël Mejía wrote:
> Congrats Alex! Well done!
>
> On Tue, Feb 18, 2020 at 5:10 PM Gleb Kanterov wrote:
>
>> Congratulations!
>>
>> On Tue, Feb 18, 2020 at 5:02 PM Brian Hulette
>> wrote:
>>
>>> Congratulations Alex! Well deserved!
>>>
>>> On
+1 for dropping Flink 1.7 support
I agree that we should avoid the SNAPSHOT dependency.
Generally I think that the version support should remain based on user
feedback and independent of the Flink release policy. If there are Beam
users that require an older Flink version and it is difficult to
Impressive, probably the fastest/smoothest Beam release so far.
On Mon, Feb 3, 2020 at 10:45 AM Boyuan Zhang wrote:
> I'm happy to announce that we have unanimously approved this release.
>
> There are 5 approving votes, 4 of which are binging:
> * Ahmet Altay
> * Ismaël Mejía
> * Jean-Baptiste
I don't have anything conclusive yet; it could also be related to our
infra. I would not block the release.
Thomas
On Wed, Jan 22, 2020 at 1:01 PM Udi Meiri wrote:
> Thomas, please let us know if you learn more about possible root causes to
> the regression you're seeing.
> Also, if you
When trying to upgrade our fork from 2.16 to 2.18, we see a significant
performance degradation. This applies to Flink portable streaming with the
Python SDK. I don't know what the cause is yet.
If anyone else has done validation with a similar setup and this RC, it
would be good to know the
+1 for the namespace proposal.
It is similar to github repos. Top-level is the org, then single level for
repo (beam-abc, beam-xzy, ..)
On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw wrote:
> Various tags of the same image should at least logically be the same
> thing, so I agree that we
ing, if we divided the key range across the workers.
>
State caching and load balancing are mutually exclusive and I added a check
to enforce that. For the use case that I'm looking at, the cost of state
access compared to that of running the expensive / high latency operations
is tiny to none.
Type();
>
>
> On Mon, Nov 25, 2019 at 10:05 AM Robert Bradshaw
> wrote:
>
>> boot.go could be updated to recognize NO_ARTIFACTS_STAGED_TOKEN as
>> well. (Should this constant be put in a common location?)
>>
>> On Sat, Nov 23, 2019 at 9:16 AM Thomas Weis
JIRA: https://issues.apache.org/jira/browse/BEAM-8816
On Thu, Nov 21, 2019 at 10:44 AM Thomas Weise wrote:
> Hi Luke,
>
> Thanks for the background and it is exciting to see the progress on the
> SDF side. It will help with this use case and many other challenges. I
> imagine
JIRA: https://issues.apache.org/jira/browse/BEAM-8815
On Fri, Nov 22, 2019 at 5:31 PM Thomas Weise wrote:
> I'm running into the issue Kyle points out when I try to run a pipeline
> that does not use artifact staging:
>
> 2019-11-23 01:09
I'm running into the issue Kyle points out when I try to run a pipeline
that does not use artifact staging:
2019-11-23 01:09:18,442 WARN
org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService
- GetManifest for
We have not seen the issue with Python 3.6 on 2.16+ after applying this
patch.
Thanks!
On Thu, Nov 21, 2019 at 4:41 PM Thomas Weise wrote:
> We are currently verifying the patch. Will report back tomorrow.
>
> On Thu, Nov 21, 2019 at 8:40 AM Valentyn Tymofieiev
> wrote:
>
ython.org/issue34572, it was very helpful.
>
> On Thu, Nov 21, 2019 at 8:25 AM Thomas Weise wrote:
>
>> Valentyn, thanks a lot for following up on this.
>>
>> If the change can be cherry picked in isolation, we should be able to
>> verify this soon (with 2.16).
>>
add a basic form of
> dynamic work rebalancing for all runners that decide to use it
>
> 1: https://github.com/apache/beam/pull/10045
> 2: https://github.com/apache/beam/pull/10065
> 3: https://github.com/apache/beam/pull/10074
>
> On Wed, Nov 20, 2019 at 10:49 AM Thomas Weise wr
Valentyn, thanks a lot for following up on this.
If the change can be cherry picked in isolation, we should be able to
verify this soon (with 2.16).
On Thu, Nov 21, 2019 at 8:12 AM Valentyn Tymofieiev
wrote:
> To close the loop here: To my knowledge this issue affects all Python 3
> users of
Congratulations!
On Wed, Nov 20, 2019, 7:56 PM Chamikara Jayalath
wrote:
> Congrats!!
>
> On Wed, Nov 20, 2019 at 5:21 PM Daniel Oliveira
> wrote:
>
>> Thank you everyone! I won't let you down. o7
>>
>> On Wed, Nov 20, 2019 at 2:12 PM Ruoyun Huang wrote:
>>
>>> Congrats Daniel!
>>>
>>> On
We found a problem with uneven utilization of SDK workers causing excessive
latency with Streaming/Python/Flink. Remember that with Python, we need to
execute multiple worker processes on a machine instead of relying on
threads in a single worker, which requires the runner to make a decision to
It would be great to have an index for those materials.
Maybe as cwiki page, which is easy to edit and watch. Similar to:
https://cwiki.apache.org/confluence/display/BEAM/Design+Documents
Thomas
On Fri, Nov 15, 2019 at 10:52 AM Kenneth Knowles wrote:
> We have a section for this:
>
Any update regarding the release?
The list still shows 10 open issues:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20fixVersion%20%3D%202.17.0%20and%20resolution%20is%20EMPTY
Is the RC blocked on those?
On Mon, Oct 28, 2019 at 12:46 PM Ahmet Altay wrote:
>
>
>
Congratulations!
On Fri, Nov 15, 2019 at 6:34 AM Connell O'Callaghan
wrote:
> Well done Brian!!!
>
> Kenn thank you for sharing
>
> On Fri, Nov 15, 2019 at 6:31 AM Cyrus Maden wrote:
>
>> Congrats Brian!
>>
>> On Fri, Nov 15, 2019 at 5:25 AM Ismaël Mejía wrote:
>>
>>> Congratulations
Done, you should be all set.
On Thu, Nov 14, 2019 at 9:57 AM Elliotte Rusty Harold
wrote:
> Hello,
>
> May I please have access to edit the Wiki? username is elharo
>
> Thanks.
>
> --
> Elliotte Rusty Harold
> elh...@ibiblio.org
>
Awesome, thanks Chad!
On Wed, Nov 13, 2019 at 10:26 PM Chad Dombrova wrote:
> Hi Thomas,
>
>
>> Will this include the ability for users to configure logging via pipeline
>> options?
>>
>
> We're working on a proposal to allow pluggable logging handlers that can
> be configured via pipeline
FWIW there are currently at least 2 instances of capability matrix [1] [2].
[1] has been in need of a refresh for a while.
[2] is more useful but only covers portable runners and is hard to find.
Thomas
[1] https://beam.apache.org/documentation/runners/capability-matrix/
[2]
tamp as a tag is an option as well, as long as runners know
>>>> which timestamp they should use.
>>>>
>>>> Hannah
>>>>
>>>> On Wed, Oct 2, 2019 at 10:13 AM Alan Myrvold
>>>> wrote:
>>>>
>>>>>
The expansion service can be provided by the job server, as done in the
Flink runner. It needs to be available at pipeline construction time, but
there is no need to run a separate service.
Thomas
On Mon, Nov 4, 2019 at 12:03 PM Robert Bradshaw wrote:
> On Mon, Nov 4, 2019 at 11:54 AM
This thread was very helpful to find more detail in
https://jira.apache.org/jira/browse/BEAM-7870
It would be great to have cross-language current state mentioned as top
level entry on https://beam.apache.org/roadmap/
On Mon, Sep 16, 2019 at 6:07 PM Chamikara Jayalath
wrote:
> Thanks for the
More here:
https://lists.apache.org/thread.html/07131e314e229ec60100eaa2c0cf6dfc206bf2b0f78c3cee9ebb0bda@%3Cdev.beam.apache.org%3E
On Fri, Nov 1, 2019 at 10:56 AM Chamikara Jayalath
wrote:
> I think it makes sense to override published docker images with locally
> built versions when testing
rating best practices for
> building an image that contains both Flink and Beam SDK dependencies? That
> would be useful.
>
> -chad
>
>
> On Fri, Nov 1, 2019 at 10:18 AM Thomas Weise wrote:
>
>> For folks looking to run Beam on Flink on k8s, see update in [1]
tions for creation of task managers in kubernetes.
>>
>> [1]
>> https://docs.google.com/document/d/1h68XOG-EyOFfcomd2N7usHK1X429pJSMiwZwAXCwx1k/edit#heading=h.72szmms7ufza
>> [2] https://issues.apache.org/jira/browse/FLINK-12761
>>
>> -chad
>>
LOOPBACK in this case seems fair.
>
> On 31.10.19 16:04, Thomas Weise wrote:
> >
> >
> > On Thu, Oct 31, 2019 at 3:55 AM Maximilian Michels > <mailto:m...@apache.org>> wrote:
> >
> > > Thanks for clarifying. So when I run "./flink my_
On Thu, Oct 31, 2019 at 3:55 AM Maximilian Michels wrote:
> > Thanks for clarifying. So when I run "./flink my_pipeline.jar" or
> > upload the jar via the REST API (and its main method invoked on the
> > master) then [auto] reads the config and does the right thing, but if
> > I do java
The current semantics of flink_master are tied to the Flink Java API. The
Flink client / Java API isn't a "REST API". It now uses the REST API
somewhere deep in RemoteEnvironment when the flink_master value is
host:port, but it does a lot of other things as well, such are parsing
config files and
is simply isomorphic to defining new primitive transforms
>> which some (all?) runners are just expected to understand.
>>
>> On Tue, Aug 20, 2019 at 10:11 AM Thomas Weise wrote:
>> >
>> >
>> >
>> > On Tue, Aug 20, 2019 at 8:56 AM Lukasz Cwik
Hi Chad,
Thanks for bringing up this discussion. I think for most of it the Flink
list would be the better suited place, but given that there are more and
more Beam users interested to deploy on Beam on Flink on k8s, I will leave
the placement to you ;-)
As for the differences between
Thanks Mikhail!
JIRA issues pointed to a resolved ticket.
This should list the open items:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20fixVersion%20%3D%202.17.0%20and%20resolution%20is%20EMPTY
On Thu, Oct 24, 2019 at 11:16 AM Mikhail Gryzykhin
wrote:
> Hello
Thanks for spotting this.
Access granted (for user "jincheng").
On Thu, Oct 10, 2019 at 12:01 AM jincheng sun
wrote:
> Hi all,
>
> The docker config has been removed with the latest python3 docker related
> commit [1], So the command:
> > ./gradlew :sdks:python:container:docker
> in `Python
Depends on JIRA volume, also. There are projects where the dev@ is heavily
populated with create JIRA notifications (and other traffic goes under a
bit).
I'm generally in favor of making the creation of JIRAs more visible. But if
there isn't broad enough interest these notifications can still be
e. Also
>> portrayal of coding as shooting lasers.
>>
>> Kenn
>>
>> On Wed, Oct 9, 2019 at 8:56 PM Thomas Weise wrote:
>>
>>> Cute, could be used as screensaver!
>>>
>>>
>>> On Wed, Oct 9, 2019 at 5:16 PM Pablo Estrada wrote:
Cute, could be used as screensaver!
On Wed, Oct 9, 2019 at 5:16 PM Pablo Estrada wrote:
> I saw this on the internets. It's pretty cute, there's all of us at some
> point - I think:
> https://www.youtube.com/watch?v=lRToJBDi7C0
>
> Best
> -P.
>
It probably makes sense to separate official project channels from external
ones like Beam Summit and meetups. Beam Summit is about Beam, but it is
"third party" and not under the project umbrella. Operation of the youtube
channel might also need clarification.
On Wed, Oct 9, 2019 at 4:35 PM
Welcome, Diksha!
On Fri, Oct 4, 2019 at 8:47 AM diksha gupta
wrote:
> Hi, I am Diksha Gupta, outreachy intern.
> I will work with your host on beamSQL.
>
Welcome, Manuela!
For getting familiar with the Beam development environment in general, I
would recommend to take a look at:
https://beam.apache.org/get-started/quickstart-py/
https://cwiki.apache.org/confluence/display/BEAM/Nexmark
For contributing and collaborating in general, please take a
I think there is a different reason why the release manager should probably
merge/approve all PRs that go into the release branch while the release is
in progress:
If/when the need arises for another RC, then only those changes should be
included that are deemed blockers or explicitly agreed.
On Tue, Sep 24, 2019 at 7:08 PM Thomas Weise wrote:
> Hi Hannah,
>
> I believe this is unexpected from the developer perspective. When building
> something locally, we do expect that to be used. We may need to change to
> not pull when the image is available locally, at least when it
Congratulations, Alan!
On Mon, Sep 30, 2019 at 4:47 AM Ismaël Mejía wrote:
> Congrats Alan!
>
> On Mon, Sep 30, 2019, 11:20 AM Tanay Tummalapalli
> wrote:
>
>> Congratulations, Alan!
>>
>>
>> On Mon, Sep 30, 2019 at 1:03 PM Gleb Kanterov wrote:
>>
>>> Congratulations!
>>>
>>> On Sat, Sep 28,
+1 for adding the coder
Please also add a test here:
https://github.com/apache/beam/blob/master/model/fn-execution/src/main/resources/org/apache/beam/model/fnexecution/v1/standard_coders.yaml
On Fri, Sep 27, 2019 at 5:17 PM Chad Dombrova wrote:
> Are there any dissenting votes to making a
here who may be able to advise.
>
>
> On Wed, Sep 25, 2019 at 6:58 AM Thomas Weise wrote:
>
>> After running through the entire bisect based on the 2.16 release branch
>> I found that the regression was caused by our own Cython setup. So green
>> light for the 2.16.0
After running through the entire bisect based on the 2.16 release branch I
found that the regression was caused by our own Cython setup. So green
light for the 2.16.0 release.
Thomas
On Tue, Sep 17, 2019 at 1:21 PM Thomas Weise wrote:
> Hi Valentyn,
>
> Thanks for the reminder. T
Hi Hannah,
I believe this is unexpected from the developer perspective. When building
something locally, we do expect that to be used. We may need to change to
not pull when the image is available locally, at least when it is a
snapshot/master branch. Release images should be immutable anyways.
+1 for making --experiments=beam_fn_api default.
Can the Dataflow runner driver just remove the setting if it is not
compatible?
On Tue, Sep 17, 2019 at 11:33 AM Maximilian Michels wrote:
> +dev
>
> The beam_fn_api flag and the way it is automatically set is error-prone.
> Is there anything
JIRAs once you have
>>>> additional information.
>>>>
>>>> Valentyn, I think the performance regression could be investigated now,
>>>> by running whatever benchmarks that is available against 2.14, 2.15 and
>>>>
| github.com/ibzib | kcwea...@google.com
>
>
> On Thu, Sep 12, 2019 at 5:48 PM Thomas Weise wrote:
>
>> This should become much better with 2.16 when we have the Docker images
>> prebuilt.
>>
>> Docker is probably still the best option for Python on a JVM bas
This should become much better with 2.16 when we have the Docker images
prebuilt.
Docker is probably still the best option for Python on a JVM based runner
in a local environment that does not have a development setup.
On Thu, Sep 12, 2019 at 1:09 PM Kyle Weaver wrote:
> +dev I think we
David, thanks for working on the Flink 1.9 support, this is very exciting!
Previous discussion on list/PRs (predating even the JIRA referenced by
Max), already pointed to the removal of support for Flink 1.5/1.6 support.
Although we generally need to consider what the Beam users need (not just
On Fri, Sep 6, 2019 at 6:23 PM Ahmet Altay wrote:
>
>
> On Fri, Sep 6, 2019 at 6:17 PM Thomas Weise wrote:
>
>>
>>
>> On Fri, Sep 6, 2019 at 2:24 PM Valentyn Tymofieiev
>> wrote:
>>
>>> +Mark Liu has added some benchmarks running across
>
> On Fri, Sep 6, 2019 at 8:38 AM Ahmet Altay wrote:
>
>> +Valentyn Tymofieiev do we have benchmarks in
>> different python versions? Was there a recent change that is specific to
>> python 3.x ?
>>
>> On Fri, Sep 6, 2019 at 8:36 AM Thomas Weise wrote:
>>
>
The issue is only visible with Python 3.6, not 2.7.
If there is a framework in place to add a streaming test, that would be
great. We would use what we have internally as starting point.
On Thu, Sep 5, 2019 at 5:00 PM Ahmet Altay wrote:
>
>
> On Thu, Sep 5, 2019 at 4:15 PM Thomas Wei
4074373
>
> They don't seem to show a very drastic jump either, but they aren't very
> old.
>
> There is also work ongoing to add alerting for this sort of regressions by
> Kasia and Kamil (added). The work is not there yet (it's in progress).
> Best
> -P.
>
> On Thu, Sep
flink
> benchmarks for chicago example?
> +Alan Myrvold +Yifan Zou It
> would be good to have alerts on benchmarks. Do we have such an ability
> today?
>
> [1] https://apache-beam-testing.appspot.com/dashboard-admin
>
> On Thu, Sep 5, 2019 at 3:15 PM Thomas Weise wrote
Hi,
Are there any performance tests run for the Python SDK as part of release
verification (or otherwise as well)?
I see what appears to be a regression in master (compared to 2.14) with our
in-house application (~ 25% jump in cpu utilization and corresponds drop in
throughput).
I wanted to see
Thomas Weise wrote:
> Awesome, thank you!
>
>
> On Wed, Sep 4, 2019 at 3:22 PM Hannah Jiang
> wrote:
>
>> Hi Thomas
>>
>> I created snapshot images from head as of around 2PM today.
>> You can pull images from gcr.io/apache-beam-testing/beam/sdks/snapshot.
&
Awesome, thank you!
On Wed, Sep 4, 2019 at 3:22 PM Hannah Jiang wrote:
> Hi Thomas
>
> I created snapshot images from head as of around 2PM today.
> You can pull images from gcr.io/apache-beam-testing/beam/sdks/snapshot.
>
> Thanks,
> Hannah
>
> On Wed, Sep 4, 2
t; head.
>
> Please let me know if you have any questions.
> Hannah
>
> On Wed, Sep 4, 2019 at 10:57 AM Thomas Weise wrote:
>
>> I actually found something in [1], but it is 2.15 unfortunately.
>>
>> [1]
>> https://console.cloud.google.com/gcr/images/apa
I actually found something in [1], but it is 2.15 unfortunately.
[1]
https://console.cloud.google.com/gcr/images/apache-beam-testing/GLOBAL/beam/sdks/release/python2.7?gcrImageListsize=30
On Wed, Sep 4, 2019 at 10:35 AM Thomas Weise wrote:
> Thanks for working on this. Do you happen to h
Thanks for working on this. Do you happen to have publicly accessible
snapshots published for your testing currently (even when the final
location isn't sorted out)?
I would like to use a 2.16 based Python SDK image for working on my
downstream project, but could not find anything in
Yifan, thanks for managing this release. It went smoothly!
On Fri, Aug 23, 2019 at 2:32 PM Kenneth Knowles wrote:
> Nice work!
>
> On Fri, Aug 23, 2019 at 11:26 AM Charles Chen wrote:
>
>> Thank you Yifan!
>>
>> On Fri, Aug 23, 2019 at 11:12 AM Hannah Jiang
>> wrote:
>>
>>> Thank you Yifan!
Congrats!
On Mon, Aug 26, 2019 at 3:22 PM Heejong Lee wrote:
> Congratulations! :)
>
> On Mon, Aug 26, 2019 at 2:44 PM Rui Wang wrote:
>
>> Congratulations!
>>
>>
>> -Rui
>>
>> On Mon, Aug 26, 2019 at 2:36 PM Hannah Jiang
>> wrote:
>>
>>> Congratulations Valentyn, well deserved!
>>>
>>> On
Ffcomd2N7usHK1X429pJSMiwZwAXCwx1k/edit#heading=h.72szmms7ufza
> [2] https://issues.apache.org/jira/browse/FLINK-12761
>
> -chad
>
>
> On Tue, Aug 13, 2019 at 9:14 AM Thomas Weise wrote:
>
>> There have been a few comments in the doc and I'm going to start working
>&g
g the key ranges. Even in case of fine-grained
> recovery of workers and their key ranges, we would simply generate new
> cache tokens for a particular worker.
>
>
> Thanks,
> Max
>
> On 21.08.19 09:33, Reuven Lax wrote:
> > Dataflow does something like this, however
Commenting here vs. on the PR since related to the overall approach.
Wouldn't it be simpler to have the runner just track a unique ID for each
worker and use that to communicate if the cache is valid or not?
* When the bundle is started, the runner tells the worker if the cache has
become
On Tue, Aug 20, 2019 at 8:56 AM Lukasz Cwik wrote:
>
>
> On Mon, Aug 19, 2019 at 5:52 PM Ahmet Altay wrote:
>
>>
>>
>> On Sun, Aug 18, 2019 at 12:34 PM Thomas Weise wrote:
>>
>>> There is a PR open for this: https://github.com/apache/beam/pull
cific
>>>> deployment mechanism (e.g. Flink k8s operator) and in general simplify the
>>>> deployment story.
>>>>
>>>> On Fri, Aug 9, 2019 at 12:46 AM Robert Bradshaw
>>>> wrote:
>>>>
>>>>> The expansion se
1 - 100 of 424 matches
Mail list logo
