Cool. I filed https://issues.apache.org/jira/browse/BEAM-4089 to track the
idea of reporting Nexmark runs to the dashboard.

Kenn

On Mon, Apr 16, 2018 at 9:29 AM Pablo Estrada <pabl...@google.com> wrote:

> This is very cool!
> Are these dashboards for tests running on Dataflow only? Are there plans
> for other runners? : )
> -P.
>
> On Mon, Apr 16, 2018 at 9:23 AM Chamikara Jayalath <chamik...@google.com>
> wrote:
>
>> Thanks Dariusz. This sounds great. Added some comments.
>> Also, +Jeff Gardner <gardn...@google.com>  who has experience on
>> performance regression analysis of integration tests.
>>
>> Thanks,
>> Cham
>>
>>
>> On Mon, Apr 16, 2018 at 2:58 AM Łukasz Gajowy <lukasz.gaj...@gmail.com>
>> wrote:
>>
>>> @Etienne +1 to doing that! :) if we have both results (Nexmark and
>>> IOITs) in BQ we could use the same (similar?) tools to detect anomalies
>>> captured by Nexmark (if there's need for doing that).
>>>
>>> 2018-04-16 11:17 GMT+02:00 Etienne Chauchot <echauc...@apache.org>:
>>>
>>>> Very nice to see the dashboards !
>>>>
>>>> Regarding Kenn's comment:  Nexmark supports outputing the results to
>>>> Bigquery so it could be easily integrated into the dashboards. We're, with
>>>> Kenn, scheduling Nexmark runs. We could configure the output to bigquery
>>>> dashboard tables ?
>>>> WDYT?
>>>>
>>>> Etienne
>>>> Le samedi 14 avril 2018 à 23:20 +0000, Kenneth Knowles a écrit :
>>>>
>>>> This is very cool. So is it easy for someone to integrate the proposal
>>>> to regularly run Nexmark benchmarks and get those on the dashboard? (or a
>>>> separate one to keep IOs in their own page)
>>>>
>>>> Kenn
>>>>
>>>> On Fri, Apr 13, 2018 at 9:02 AM Dariusz Aniszewski <
>>>> dariusz.aniszew...@polidea.com> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *Hello Beam devs!As you might already noticed, together with Łukasz
>>>> Gajowy, Kamil Szewczyk and Katarzyna Kucharczyk (all directly cc’d here)
>>>> we’re working on adding some performance tests to the project. We were
>>>> following directions from the Testing I/O Transforms in Apache Beam
>>>> <https://beam.apache.org/documentation/io/testing/> site (which we plan to
>>>> update in near future).We started from testing various FileBasedIOs as part
>>>> of BEAM-3060 <https://issues.apache.org/jira/browse/BEAM-3060>. So far we
>>>> have tests for: - TextIO (with and without compression)- AvroIO- XmlIO-
>>>> TFRecordIOthat may run on following filesystems: - local- GCS- HDFS (except
>>>> for TFRecordIO, see BEAM-3945
>>>> <https://issues.apache.org/jira/browse/BEAM-3945>)Besides FileBasedIOs we
>>>> also covered: - HadoopInputFormatIO- MongoDBIO- JdbcIO (in this case test
>>>> was there, but was disabled; we fixed it and enabled)- HCatalogIO
>>>> (currently in PR <https://github.com/apache/beam/pull/5097>)While currently
>>>> all the tests are maven-based, we responded to ongoing Gradle migration and
>>>> created PR <https://github.com/apache/beam/pull/5003> that allows running
>>>> them via Gradle.All of those tests are executed on daily basis on Apache
>>>> Jenkins <https://builds.apache.org/> and their results are published to
>>>> individual BigQuery tables. There is also a dashboard on which tests
>>>> results may be viewed and
>>>> compared:https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688
>>>> <https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688>As
>>>> we have some amount of tests already, we’re currently working on a tool
>>>> that will analyze the results and search for anomalies, so devs are
>>>> notified if degraded performance is observed. You can find proposal
>>>> document
>>>> here:https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA
>>>> <https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA>We
>>>> welcome you to share your thoughts on performance tests in general as well
>>>> as proposed solution for anomaly detection.Best,Dariusz Aniszewski*
>>>>
>>>>
>>> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>

Reply via email to