Very nice to see the dashboards !
Regarding Kenn's comment: Nexmark supports outputing the results to Bigquery
so it could be easily integrated into the
dashboards. We're, with Kenn, scheduling Nexmark runs. We could configure the
output to bigquery dashboard tables ?
WDYT?
Etienne
Le samedi 14 avril 2018 à 23:20 +0000, Kenneth Knowles a écrit :
> This is very cool. So is it easy for someone to integrate the proposal to
> regularly run Nexmark benchmarks and get
> those on the dashboard? (or a separate one to keep IOs in their own page)
>
> Kenn
>
> On Fri, Apr 13, 2018 at 9:02 AM Dariusz Aniszewski
> <dariusz.aniszew...@polidea.com> wrote:
> > Hello Beam devs!
> >
> > As you might already noticed, together with Łukasz Gajowy, Kamil Szewczyk
> > and Katarzyna Kucharczyk (all directly
> > cc’d here) we’re working on adding some performance tests to the project.
> > We were following directions from the
> > Testing I/O Transforms in Apache Beam site (which we plan to update in near
> > future).
> >
> > We started from testing various FileBasedIOs as part of BEAM-3060. So far
> > we have tests for:
> > TextIO (with and without compression)
> > AvroIO
> > XmlIO
> > TFRecordIO
> > that may run on following filesystems:
> > local
> > GCS
> > HDFS (except for TFRecordIO, see BEAM-3945)
> >
> > Besides FileBasedIOs we also covered:
> > HadoopInputFormatIO
> > MongoDBIO
> > JdbcIO (in this case test was there, but was disabled; we fixed it and
> > enabled)
> > HCatalogIO (currently in PR)
> >
> > While currently all the tests are maven-based, we responded to ongoing
> > Gradle migration and created PR that allows
> > running them via Gradle.
> >
> > All of those tests are executed on daily basis on Apache Jenkins and their
> > results are published to individual
> > BigQuery tables. There is also a dashboard on which tests results may be
> > viewed and compared:
> > https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688
> >
> >
> > As we have some amount of tests already, we’re currently working on a tool
> > that will analyze the results and search
> > for anomalies, so devs are notified if degraded performance is observed.
> > You can find proposal document here:
> > https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA
> >
> > We welcome you to share your thoughts on performance tests in general as
> > well as proposed solution for anomaly
> > detection.
> >
> > Best,
> > Dariusz Aniszewski
> >
> >
> >
