See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/13/display/redirect?page=changes>
Changes: [Luke Cwik] [BEAM-12547] Enable epoll again. [daniel.o.programmer] [BEAM-12383] Go Kafka integration test and framework changes. [noreply] Update [noreply] [BEAM-12529] Allow converting arbitrary pandas dtypes back to Python [noreply] [BEAM-12533] Add simple `__repr__` for `DeferredDataFrame` and ------------------------------------------ [...truncated 185.42 KB...] Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.219Z: Fusing consumer Step: 112/ParMultiDo(CounterOperation) into Step: 111/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.254Z: Fusing consumer Step: 113/ParMultiDo(CounterOperation) into Step: 112/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.289Z: Fusing consumer Step: 114/ParMultiDo(CounterOperation) into Step: 113/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.325Z: Fusing consumer Step: 115/ParMultiDo(CounterOperation) into Step: 114/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.356Z: Fusing consumer Step: 116/ParMultiDo(CounterOperation) into Step: 115/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.383Z: Fusing consumer Step: 117/ParMultiDo(CounterOperation) into Step: 116/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.405Z: Fusing consumer Step: 118/ParMultiDo(CounterOperation) into Step: 117/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.426Z: Fusing consumer Step: 119/ParMultiDo(CounterOperation) into Step: 118/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.448Z: Fusing consumer Step: 120/ParMultiDo(CounterOperation) into Step: 119/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.481Z: Fusing consumer Step: 121/ParMultiDo(CounterOperation) into Step: 120/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.505Z: Fusing consumer Step: 122/ParMultiDo(CounterOperation) into Step: 121/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.528Z: Fusing consumer Step: 123/ParMultiDo(CounterOperation) into Step: 122/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.563Z: Fusing consumer Step: 124/ParMultiDo(CounterOperation) into Step: 123/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.596Z: Fusing consumer Step: 125/ParMultiDo(CounterOperation) into Step: 124/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.621Z: Fusing consumer Step: 126/ParMultiDo(CounterOperation) into Step: 125/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.643Z: Fusing consumer Step: 127/ParMultiDo(CounterOperation) into Step: 126/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.668Z: Fusing consumer Step: 128/ParMultiDo(CounterOperation) into Step: 127/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.686Z: Fusing consumer Step: 129/ParMultiDo(CounterOperation) into Step: 128/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.718Z: Fusing consumer Step: 130/ParMultiDo(CounterOperation) into Step: 129/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.748Z: Fusing consumer Step: 131/ParMultiDo(CounterOperation) into Step: 130/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.775Z: Fusing consumer Step: 132/ParMultiDo(CounterOperation) into Step: 131/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.802Z: Fusing consumer Step: 133/ParMultiDo(CounterOperation) into Step: 132/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.830Z: Fusing consumer Step: 134/ParMultiDo(CounterOperation) into Step: 133/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.856Z: Fusing consumer Step: 135/ParMultiDo(CounterOperation) into Step: 134/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.921Z: Fusing consumer Step: 136/ParMultiDo(CounterOperation) into Step: 135/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.955Z: Fusing consumer Step: 137/ParMultiDo(CounterOperation) into Step: 136/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:37.979Z: Fusing consumer Step: 138/ParMultiDo(CounterOperation) into Step: 137/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.013Z: Fusing consumer Step: 139/ParMultiDo(CounterOperation) into Step: 138/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.033Z: Fusing consumer Step: 140/ParMultiDo(CounterOperation) into Step: 139/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.051Z: Fusing consumer Step: 141/ParMultiDo(CounterOperation) into Step: 140/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.096Z: Fusing consumer Step: 142/ParMultiDo(CounterOperation) into Step: 141/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.122Z: Fusing consumer Step: 143/ParMultiDo(CounterOperation) into Step: 142/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.149Z: Fusing consumer Step: 144/ParMultiDo(CounterOperation) into Step: 143/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.181Z: Fusing consumer Step: 145/ParMultiDo(CounterOperation) into Step: 144/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.215Z: Fusing consumer Step: 146/ParMultiDo(CounterOperation) into Step: 145/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.244Z: Fusing consumer Step: 147/ParMultiDo(CounterOperation) into Step: 146/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.268Z: Fusing consumer Step: 148/ParMultiDo(CounterOperation) into Step: 147/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.298Z: Fusing consumer Step: 149/ParMultiDo(CounterOperation) into Step: 148/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.334Z: Fusing consumer Step: 150/ParMultiDo(CounterOperation) into Step: 149/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.367Z: Fusing consumer Step: 151/ParMultiDo(CounterOperation) into Step: 150/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.393Z: Fusing consumer Step: 152/ParMultiDo(CounterOperation) into Step: 151/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.426Z: Fusing consumer Step: 153/ParMultiDo(CounterOperation) into Step: 152/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.458Z: Fusing consumer Step: 154/ParMultiDo(CounterOperation) into Step: 153/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.494Z: Fusing consumer Step: 155/ParMultiDo(CounterOperation) into Step: 154/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.514Z: Fusing consumer Step: 156/ParMultiDo(CounterOperation) into Step: 155/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.547Z: Fusing consumer Step: 157/ParMultiDo(CounterOperation) into Step: 156/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.581Z: Fusing consumer Step: 158/ParMultiDo(CounterOperation) into Step: 157/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.615Z: Fusing consumer Step: 159/ParMultiDo(CounterOperation) into Step: 158/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.648Z: Fusing consumer Step: 160/ParMultiDo(CounterOperation) into Step: 159/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.672Z: Fusing consumer Step: 161/ParMultiDo(CounterOperation) into Step: 160/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.704Z: Fusing consumer Step: 162/ParMultiDo(CounterOperation) into Step: 161/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.730Z: Fusing consumer Step: 163/ParMultiDo(CounterOperation) into Step: 162/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.755Z: Fusing consumer Step: 164/ParMultiDo(CounterOperation) into Step: 163/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.781Z: Fusing consumer Step: 165/ParMultiDo(CounterOperation) into Step: 164/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.813Z: Fusing consumer Step: 166/ParMultiDo(CounterOperation) into Step: 165/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.830Z: Fusing consumer Step: 167/ParMultiDo(CounterOperation) into Step: 166/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.861Z: Fusing consumer Step: 168/ParMultiDo(CounterOperation) into Step: 167/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.886Z: Fusing consumer Step: 169/ParMultiDo(CounterOperation) into Step: 168/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.928Z: Fusing consumer Step: 170/ParMultiDo(CounterOperation) into Step: 169/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.945Z: Fusing consumer Step: 171/ParMultiDo(CounterOperation) into Step: 170/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.973Z: Fusing consumer Step: 172/ParMultiDo(CounterOperation) into Step: 171/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:38.998Z: Fusing consumer Step: 173/ParMultiDo(CounterOperation) into Step: 172/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.030Z: Fusing consumer Step: 174/ParMultiDo(CounterOperation) into Step: 173/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.054Z: Fusing consumer Step: 175/ParMultiDo(CounterOperation) into Step: 174/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.092Z: Fusing consumer Step: 176/ParMultiDo(CounterOperation) into Step: 175/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.113Z: Fusing consumer Step: 177/ParMultiDo(CounterOperation) into Step: 176/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.149Z: Fusing consumer Step: 178/ParMultiDo(CounterOperation) into Step: 177/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.181Z: Fusing consumer Step: 179/ParMultiDo(CounterOperation) into Step: 178/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.214Z: Fusing consumer Step: 180/ParMultiDo(CounterOperation) into Step: 179/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.243Z: Fusing consumer Step: 181/ParMultiDo(CounterOperation) into Step: 180/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.288Z: Fusing consumer Step: 182/ParMultiDo(CounterOperation) into Step: 181/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.315Z: Fusing consumer Step: 183/ParMultiDo(CounterOperation) into Step: 182/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.355Z: Fusing consumer Step: 184/ParMultiDo(CounterOperation) into Step: 183/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.379Z: Fusing consumer Step: 185/ParMultiDo(CounterOperation) into Step: 184/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.407Z: Fusing consumer Step: 186/ParMultiDo(CounterOperation) into Step: 185/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.441Z: Fusing consumer Step: 187/ParMultiDo(CounterOperation) into Step: 186/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.473Z: Fusing consumer Step: 188/ParMultiDo(CounterOperation) into Step: 187/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.506Z: Fusing consumer Step: 189/ParMultiDo(CounterOperation) into Step: 188/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.539Z: Fusing consumer Step: 190/ParMultiDo(CounterOperation) into Step: 189/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.564Z: Fusing consumer Step: 191/ParMultiDo(CounterOperation) into Step: 190/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.593Z: Fusing consumer Step: 192/ParMultiDo(CounterOperation) into Step: 191/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.623Z: Fusing consumer Step: 193/ParMultiDo(CounterOperation) into Step: 192/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.651Z: Fusing consumer Step: 194/ParMultiDo(CounterOperation) into Step: 193/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.680Z: Fusing consumer Step: 195/ParMultiDo(CounterOperation) into Step: 194/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.714Z: Fusing consumer Step: 196/ParMultiDo(CounterOperation) into Step: 195/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.741Z: Fusing consumer Step: 197/ParMultiDo(CounterOperation) into Step: 196/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.778Z: Fusing consumer Step: 198/ParMultiDo(CounterOperation) into Step: 197/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.803Z: Fusing consumer Step: 199/ParMultiDo(CounterOperation) into Step: 198/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:39.829Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 199/ParMultiDo(CounterOperation) Jun 30, 2021 12:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:40.160Z: Starting 5 ****s in us-central1-f... Jun 30, 2021 12:58:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:58:42.614Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Jun 30, 2021 12:59:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T12:59:30.998Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Jun 30, 2021 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T13:00:04.842Z: Workers have started successfully. Jun 30, 2021 1:00:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T13:00:04.873Z: Workers have started successfully. Jun 30, 2021 4:01:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T16:01:34.577Z: Cancel request is committed for workflow job: 2021-06-30_05_58_22-9555275467178275453. Jun 30, 2021 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T16:01:34.645Z: Cleaning up. Jun 30, 2021 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T16:01:34.834Z: Stopping **** pool... Jun 30, 2021 4:01:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T16:01:34.890Z: Stopping **** pool... Jun 30, 2021 4:02:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T16:02:25.066Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Jun 30, 2021 4:02:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-06-30T16:02:25.108Z: Worker pool stopped. Jun 30, 2021 4:02:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-06-30_05_58_22-9555275467178275453 finished with status CANCELLED. Load test results for test (ID): abe6e5c3-43ba-493d-95d4-3a898a73c12b and timestamp: 2021-06-30T12:58:16.890000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 10852.216 dataflow_v2_java11_total_bytes_count 2.037675604E11 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53) at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210630125736 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:838079ebc1e86ad91edd4b5fba80254fdf02aeb87a34112dcd8918dc7fb61e77 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:838079ebc1e86ad91edd4b5fba80254fdf02aeb87a34112dcd8918dc7fb61e77 Associated tags: - 20210630125736 Tags: - us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210630125736 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210630125736]. Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:838079ebc1e86ad91edd4b5fba80254fdf02aeb87a34112dcd8918dc7fb61e77]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 5m 4s 99 actionable tasks: 14 executed, 85 up-to-date Publishing build scan... https://gradle.com/s/ecmq3x3nja7fe Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
