Only on HDFS @reuvenlax This is the change I was testing from the discussion we recently had on [dev@](https://lists.apache.org/thread.html/b904779aefec1b6d01d28a492f626075de009a71587e1b9df3aa0f2b@%3Cdev.beam.apache.org%3E)
Repeating here - on a 10 node YARN CDH 5.12.2 cluster, rewriting a 1.5TB AvroIO file (code [here](https://github.com/gbif/beam-perf/tree/master/avro-to-avro)) I observed: ``` - Spark API: 35 minutes - Beam AvroIO (2.6.0): 1.7hrs - Beam AvroIO with this rename() patch: 42 minutes ``` [ Full content available at: https://github.com/apache/beam/pull/6289 ] This message was relayed via gitbox.apache.org for [email protected]
