ravwojdyla commented on code in PR #1240:
URL: https://github.com/apache/parquet-mr/pull/1240#discussion_r1430866723
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/MemoryManager.java:
##########
@@ -74,7 +75,7 @@ private void checkRatio(float ratio) {
* @param writer the new created writer
* @param allocation the requested buffer size
*/
- synchronized void addWriter(InternalParquetRecordWriter<?> writer, Long
allocation) {
+ void addWriter(InternalParquetRecordWriter<?> writer, Long allocation) {
Review Comment:
@ConeyLiu yes it does. One of our production Spark job took close to 2 hours
with partitioned parquet output and about 45 minutes without. Also please see
the stats in JIRA issues:
* https://issues.apache.org/jira/browse/PARQUET-2412
* https://issues.apache.org/jira/browse/SPARK-44003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
