On Thu, Sep 5, 2019 at 4:15 PM Thomas Weise <t...@apache.org> wrote: > The workload is quite different. What I have is streaming with state and > timers. > > > > On Thu, Sep 5, 2019 at 3:47 PM Pablo Estrada <pabl...@google.com> wrote: > >> We only recently started running Chicago Taxi Example. +Michał Walenia >> <michal.wale...@polidea.com> I don't see it in the dashboards. Do you >> know if it's possible to see any trends in the data? >> >> We have a few tests running now: >> - Combine tests: >> https://apache-beam-testing.appspot.com/explore?dashboard=5763764733345792&widget=201943890&container=1334074373 >> - GBK tests: >> https://apache-beam-testing.appspot.com/explore?dashboard=5763764733345792&widget=201943890&container=1334074373 >> >> They don't seem to show a very drastic jump either, but they aren't very >> old. >> >> There is also work ongoing to add alerting for this sort of regressions >> by Kasia and Kamil (added). The work is not there yet (it's in progress). >> Best >> -P. >> >> On Thu, Sep 5, 2019 at 3:35 PM Thomas Weise <t...@apache.org> wrote: >> >>> It probably won't be practical to do a bisect due to the high cost of >>> each iteration with our fork/deploy setup. >>> >>> Perhaps it is time to setup something with the synthetic source that >>> works just with Beam as dependency. >>> >> I agree with this.
Pablo, Kasia, Kamil, does the new benchmarks give us a easy to use framework for using synthetic source in benchmarks? > >>> On Thu, Sep 5, 2019 at 3:23 PM Ahmet Altay <al...@google.com> wrote: >>> >>>> There are a few in this dashboard [1], but not very useful in this case >>>> because they do not go back more than a month and not very comprehensive. I >>>> do not see a jump there. Thomas, would it be possible to bisect to find >>>> what commit caused the regression? >>>> >>>> +Pablo Estrada <pabl...@google.com> do we have any python on flink >>>> benchmarks for chicago example? >>>> +Alan Myrvold <amyrv...@google.com> +Yifan Zou <yifan...@google.com> It >>>> would be good to have alerts on benchmarks. Do we have such an ability >>>> today? >>>> >>>> [1] https://apache-beam-testing.appspot.com/dashboard-admin >>>> >>>> On Thu, Sep 5, 2019 at 3:15 PM Thomas Weise <t...@apache.org> wrote: >>>> >>>>> Hi, >>>>> >>>>> Are there any performance tests run for the Python SDK as part of >>>>> release verification (or otherwise as well)? >>>>> >>>>> I see what appears to be a regression in master (compared to 2.14) >>>>> with our in-house application (~ 25% jump in cpu utilization and >>>>> corresponds drop in throughput). >>>>> >>>>> I wanted to see if there is anything available to verify that within >>>>> Beam. >>>>> >>>>> Thanks, >>>>> Thomas >>>>> >>>>>
