*Hello Beam devs!As you might already noticed, together with Łukasz Gajowy,
Kamil Szewczyk and Katarzyna Kucharczyk (all directly cc’d here) we’re
working on adding some performance tests to the project. We were following
directions from the Testing I/O Transforms in Apache Beam
<https://beam.apache.org/documentation/io/testing/> site (which we plan to
update in near future).We started from testing various FileBasedIOs as part
of BEAM-3060 <https://issues.apache.org/jira/browse/BEAM-3060>. So far we
have tests for: - TextIO (with and without compression)- AvroIO- XmlIO-
TFRecordIOthat may run on following filesystems: - local- GCS- HDFS (except
for TFRecordIO, see BEAM-3945
<https://issues.apache.org/jira/browse/BEAM-3945>)Besides FileBasedIOs we
also covered: - HadoopInputFormatIO- MongoDBIO- JdbcIO (in this case test
was there, but was disabled; we fixed it and enabled)- HCatalogIO
(currently in PR <https://github.com/apache/beam/pull/5097>)While currently
all the tests are maven-based, we responded to ongoing Gradle migration and
created PR <https://github.com/apache/beam/pull/5003> that allows running
them via Gradle.All of those tests are executed on daily basis on Apache
Jenkins <https://builds.apache.org/> and their results are published to
individual BigQuery tables. There is also a dashboard on which tests
results may be viewed and
compared:https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688
<https://apache-beam-testing.appspot.com/explore?dashboard=5755685136498688>As
we have some amount of tests already, we’re currently working on a tool
that will analyze the results and search for anomalies, so devs are
notified if degraded performance is observed. You can find proposal
document
here:https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA
<https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA>We
welcome you to share your thoughts on performance tests in general as well
as proposed solution for anomaly detection.Best,Dariusz Aniszewski*
