See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/503/display/redirect?page=changes>
Changes: [Robert Bradshaw] Move logging to its own module. [Robert Bradshaw] Cleanup worker logging. [Robert Bradshaw] Add basic counter setting and getting to the typescript SDK. [Robert Bradshaw] Support metrics over the portability API. [Robert Bradshaw] Add distribution metric type. [Robert Bradshaw] old prettier change [noreply] TFX image classification example (#23456) [noreply] Immediately truncate full restriction on drain of periodic impulse [noreply] [Task]: PR Bot will push commits only if they are non-empty (#23937) [noreply] Bump cloud.google.com/go/datastore from 1.8.0 to 1.9.0 in /sdks (#23916) [Robert Bradshaw] Remove obsolete TODO. [Robert Bradshaw] Only report counters that were actually used. [noreply] Add custom inference fn suport to the sklearn model handlers (#23642) [noreply] removed trailing whitespace (#23987) [noreply] Beam starter projects blog post (#23964) [noreply] Enable more portable-runner requiring tests. (#23970) [noreply] Website add and update logos (#23899) ------------------------------------------ [...truncated 178.31 KB...] Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.151Z: Fusing consumer Step: 116/ParMultiDo(CounterOperation) into Step: 115/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.184Z: Fusing consumer Step: 117/ParMultiDo(CounterOperation) into Step: 116/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.221Z: Fusing consumer Step: 118/ParMultiDo(CounterOperation) into Step: 117/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.247Z: Fusing consumer Step: 119/ParMultiDo(CounterOperation) into Step: 118/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.282Z: Fusing consumer Step: 120/ParMultiDo(CounterOperation) into Step: 119/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.315Z: Fusing consumer Step: 121/ParMultiDo(CounterOperation) into Step: 120/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.336Z: Fusing consumer Step: 122/ParMultiDo(CounterOperation) into Step: 121/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.368Z: Fusing consumer Step: 123/ParMultiDo(CounterOperation) into Step: 122/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.406Z: Fusing consumer Step: 124/ParMultiDo(CounterOperation) into Step: 123/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.441Z: Fusing consumer Step: 125/ParMultiDo(CounterOperation) into Step: 124/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.466Z: Fusing consumer Step: 126/ParMultiDo(CounterOperation) into Step: 125/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.496Z: Fusing consumer Step: 127/ParMultiDo(CounterOperation) into Step: 126/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.524Z: Fusing consumer Step: 128/ParMultiDo(CounterOperation) into Step: 127/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.560Z: Fusing consumer Step: 129/ParMultiDo(CounterOperation) into Step: 128/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.583Z: Fusing consumer Step: 130/ParMultiDo(CounterOperation) into Step: 129/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.610Z: Fusing consumer Step: 131/ParMultiDo(CounterOperation) into Step: 130/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.649Z: Fusing consumer Step: 132/ParMultiDo(CounterOperation) into Step: 131/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.679Z: Fusing consumer Step: 133/ParMultiDo(CounterOperation) into Step: 132/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.707Z: Fusing consumer Step: 134/ParMultiDo(CounterOperation) into Step: 133/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.730Z: Fusing consumer Step: 135/ParMultiDo(CounterOperation) into Step: 134/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.766Z: Fusing consumer Step: 136/ParMultiDo(CounterOperation) into Step: 135/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.804Z: Fusing consumer Step: 137/ParMultiDo(CounterOperation) into Step: 136/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.831Z: Fusing consumer Step: 138/ParMultiDo(CounterOperation) into Step: 137/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.868Z: Fusing consumer Step: 139/ParMultiDo(CounterOperation) into Step: 138/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.893Z: Fusing consumer Step: 140/ParMultiDo(CounterOperation) into Step: 139/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.929Z: Fusing consumer Step: 141/ParMultiDo(CounterOperation) into Step: 140/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.949Z: Fusing consumer Step: 142/ParMultiDo(CounterOperation) into Step: 141/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:53.983Z: Fusing consumer Step: 143/ParMultiDo(CounterOperation) into Step: 142/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.017Z: Fusing consumer Step: 144/ParMultiDo(CounterOperation) into Step: 143/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.042Z: Fusing consumer Step: 145/ParMultiDo(CounterOperation) into Step: 144/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.075Z: Fusing consumer Step: 146/ParMultiDo(CounterOperation) into Step: 145/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.110Z: Fusing consumer Step: 147/ParMultiDo(CounterOperation) into Step: 146/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.138Z: Fusing consumer Step: 148/ParMultiDo(CounterOperation) into Step: 147/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.167Z: Fusing consumer Step: 149/ParMultiDo(CounterOperation) into Step: 148/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.200Z: Fusing consumer Step: 150/ParMultiDo(CounterOperation) into Step: 149/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.234Z: Fusing consumer Step: 151/ParMultiDo(CounterOperation) into Step: 150/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.269Z: Fusing consumer Step: 152/ParMultiDo(CounterOperation) into Step: 151/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.304Z: Fusing consumer Step: 153/ParMultiDo(CounterOperation) into Step: 152/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.338Z: Fusing consumer Step: 154/ParMultiDo(CounterOperation) into Step: 153/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.373Z: Fusing consumer Step: 155/ParMultiDo(CounterOperation) into Step: 154/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.397Z: Fusing consumer Step: 156/ParMultiDo(CounterOperation) into Step: 155/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.426Z: Fusing consumer Step: 157/ParMultiDo(CounterOperation) into Step: 156/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.459Z: Fusing consumer Step: 158/ParMultiDo(CounterOperation) into Step: 157/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.493Z: Fusing consumer Step: 159/ParMultiDo(CounterOperation) into Step: 158/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.524Z: Fusing consumer Step: 160/ParMultiDo(CounterOperation) into Step: 159/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.548Z: Fusing consumer Step: 161/ParMultiDo(CounterOperation) into Step: 160/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.585Z: Fusing consumer Step: 162/ParMultiDo(CounterOperation) into Step: 161/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.620Z: Fusing consumer Step: 163/ParMultiDo(CounterOperation) into Step: 162/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.655Z: Fusing consumer Step: 164/ParMultiDo(CounterOperation) into Step: 163/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.690Z: Fusing consumer Step: 165/ParMultiDo(CounterOperation) into Step: 164/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.725Z: Fusing consumer Step: 166/ParMultiDo(CounterOperation) into Step: 165/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.755Z: Fusing consumer Step: 167/ParMultiDo(CounterOperation) into Step: 166/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.785Z: Fusing consumer Step: 168/ParMultiDo(CounterOperation) into Step: 167/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.812Z: Fusing consumer Step: 169/ParMultiDo(CounterOperation) into Step: 168/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.846Z: Fusing consumer Step: 170/ParMultiDo(CounterOperation) into Step: 169/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.881Z: Fusing consumer Step: 171/ParMultiDo(CounterOperation) into Step: 170/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.905Z: Fusing consumer Step: 172/ParMultiDo(CounterOperation) into Step: 171/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.940Z: Fusing consumer Step: 173/ParMultiDo(CounterOperation) into Step: 172/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:54.973Z: Fusing consumer Step: 174/ParMultiDo(CounterOperation) into Step: 173/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.005Z: Fusing consumer Step: 175/ParMultiDo(CounterOperation) into Step: 174/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.040Z: Fusing consumer Step: 176/ParMultiDo(CounterOperation) into Step: 175/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.063Z: Fusing consumer Step: 177/ParMultiDo(CounterOperation) into Step: 176/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.089Z: Fusing consumer Step: 178/ParMultiDo(CounterOperation) into Step: 177/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.114Z: Fusing consumer Step: 179/ParMultiDo(CounterOperation) into Step: 178/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.140Z: Fusing consumer Step: 180/ParMultiDo(CounterOperation) into Step: 179/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.175Z: Fusing consumer Step: 181/ParMultiDo(CounterOperation) into Step: 180/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.213Z: Fusing consumer Step: 182/ParMultiDo(CounterOperation) into Step: 181/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.244Z: Fusing consumer Step: 183/ParMultiDo(CounterOperation) into Step: 182/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.268Z: Fusing consumer Step: 184/ParMultiDo(CounterOperation) into Step: 183/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.289Z: Fusing consumer Step: 185/ParMultiDo(CounterOperation) into Step: 184/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.319Z: Fusing consumer Step: 186/ParMultiDo(CounterOperation) into Step: 185/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.353Z: Fusing consumer Step: 187/ParMultiDo(CounterOperation) into Step: 186/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.383Z: Fusing consumer Step: 188/ParMultiDo(CounterOperation) into Step: 187/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.418Z: Fusing consumer Step: 189/ParMultiDo(CounterOperation) into Step: 188/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.464Z: Fusing consumer Step: 190/ParMultiDo(CounterOperation) into Step: 189/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.500Z: Fusing consumer Step: 191/ParMultiDo(CounterOperation) into Step: 190/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.534Z: Fusing consumer Step: 192/ParMultiDo(CounterOperation) into Step: 191/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.560Z: Fusing consumer Step: 193/ParMultiDo(CounterOperation) into Step: 192/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.596Z: Fusing consumer Step: 194/ParMultiDo(CounterOperation) into Step: 193/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.631Z: Fusing consumer Step: 195/ParMultiDo(CounterOperation) into Step: 194/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.666Z: Fusing consumer Step: 196/ParMultiDo(CounterOperation) into Step: 195/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.692Z: Fusing consumer Step: 197/ParMultiDo(CounterOperation) into Step: 196/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.713Z: Fusing consumer Step: 198/ParMultiDo(CounterOperation) into Step: 197/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.750Z: Fusing consumer Step: 199/ParMultiDo(CounterOperation) into Step: 198/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.776Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 199/ParMultiDo(CounterOperation) Nov 05, 2022 12:46:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:55.891Z: Running job using Streaming Engine Nov 05, 2022 12:46:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:46:57.180Z: Starting 5 ****s in us-central1-b... Nov 05, 2022 12:47:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:47:03.284Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Nov 05, 2022 12:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:47:42.273Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Nov 05, 2022 12:48:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:48:53.313Z: Workers have started successfully. Nov 05, 2022 12:52:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:52:12.975Z: Cleaning up. Nov 05, 2022 12:52:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:52:13.076Z: Stopping **** pool... Nov 05, 2022 12:52:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:52:13.182Z: Stopping **** pool... Nov 05, 2022 12:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:54:33.391Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Nov 05, 2022 12:54:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-11-05T12:54:33.437Z: Worker pool stopped. Nov 05, 2022 12:54:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2022-11-05_05_46_32-12902007784033828977 finished with status DONE. Load test results for test (ID): 78dda737-f467-4a11-a887-cf3b06dc5c5d and timestamp: 2022-11-05T12:46:26.746000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 125.426 dataflow_v2_java11_total_bytes_count 1.999998E9 > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20221105124554 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2162f77112c325c990bc9f451841a3570f35ab9247be8aa9c920c469c3859d06 WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20221105124554] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2162f77112c325c990bc9f451841a3570f35ab9247be8aa9c920c469c3859d06] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20221105124554] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2162f77112c325c990bc9f451841a3570f35ab9247be8aa9c920c469c3859d06])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:276278bdf67d74d89ea1d8d1252097c42ad5ccf5b5ac3486422624ac0ee75155 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:276278bdf67d74d89ea1d8d1252097c42ad5ccf5b5ac3486422624ac0ee75155 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:276278bdf67d74d89ea1d8d1252097c42ad5ccf5b5ac3486422624ac0ee75155]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:422e1cba6ec0a1ca20ac4f9aeb71418fc40216d4b70a94d40a68c3e1003d4114 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:422e1cba6ec0a1ca20ac4f9aeb71418fc40216d4b70a94d40a68c3e1003d4114 ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'content-encoding': 'gzip', 'date': 'Sat, 05 Nov 2022 12:55:03 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': 404} Failed to compute blob liveness for manifest: 'sha256:422e1cba6ec0a1ca20ac4f9aeb71418fc40216d4b70a94d40a68c3e1003d4114': None > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 304 * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'. > Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 9m 10s 109 actionable tasks: 9 executed, 100 up-to-date Publishing build scan... https://gradle.com/s/zzsxqvdx54pwm Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
