srowen commented on issue #24720: [SPARK-27852][Spark Core] updateBytesWritten() operaton is missed URL: https://github.com/apache/spark/pull/24720#issuecomment-496524544 @BestOreo the caller calls `write` potentially may times to write a record, then calls `recordWritten`, where metrics are update every 16384 records. There's a `write` overload that writes a whole record at once and then calls `recordWritten`. We specifically don't want to update metrics several times for every record. That would be very slow.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
