Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21601
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21601#discussion_r201055350
--- Diff:
core/src/test/scala/org/apache/spark/input/WholeTextFileInputFormatSuite.scala
---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache
Github user dhruve commented on a diff in the pull request:
https://github.com/apache/spark/pull/21601#discussion_r199597945
--- Diff:
core/src/main/scala/org/apache/spark/input/WholeTextFileInputFormat.scala ---
@@ -53,6 +53,19 @@ private[spark] class WholeTextFileInputFormat
Github user dhruve commented on a diff in the pull request:
https://github.com/apache/spark/pull/21601#discussion_r199602993
--- Diff:
core/src/main/scala/org/apache/spark/input/WholeTextFileInputFormat.scala ---
@@ -53,6 +53,19 @@ private[spark] class WholeTextFileInputFormat
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21601#discussion_r198667795
--- Diff:
core/src/main/scala/org/apache/spark/input/WholeTextFileInputFormat.scala ---
@@ -53,6 +53,19 @@ private[spark] class WholeTextFileInputFormat
GitHub user dhruve opened a pull request:
https://github.com/apache/spark/pull/21601
[SPARK-24610] fix reading small files via wholeTextFiles
## What changes were proposed in this pull request?
The `WholeTextFileInputFormat` determines the `maxSplitSize` for the file/s
being
