rahil-c commented on code in PR #815:
URL: https://github.com/apache/incubator-xtable/pull/815#discussion_r3414232596


##########
xtable-core/src/main/java/org/apache/xtable/hudi/HudiFileStatsExtractor.java:
##########
@@ -104,7 +104,11 @@ public Stream<InternalDataFile> addStatsToFiles(
 
   private Stream<InternalDataFile> computeColumnStatsFromParquetFooters(
       Stream<InternalDataFile> files, Map<String, InternalField> nameFieldMap) 
{
-    return files.map(
+    // Use the common ForkJoinPool for parquet footer reads; parallelism can 
be tuned via
+    // -Djava.util.concurrent.ForkJoinPool.common.parallelism.
+    return files
+        .parallel()

Review Comment:
   +1 to vinish's comment here around avoiding `ForkJoinPool` and to use a 
dedicated `ExecutorService`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to