Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144745063
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -270,6 +270,12 @@ package object config {
.longConf
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144745022
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -270,6 +270,12 @@ package object config {
.longConf
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19464
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144687101
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,83 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user liutang123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144683771
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,87 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144617310
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,87 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144617011
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,87 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144604878
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,87 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144472362
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,86 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144472244
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,86 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144452771
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -270,6 +270,12 @@ package object config {
.longConf
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144365104
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,83 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144357376
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,83 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144357089
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,83 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144353093
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,54 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144330429
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -270,6 +270,15 @@ package object config {
.longConf
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144324123
--- Diff: docs/configuration.md ---
@@ -1192,6 +1192,14 @@ Apart from these, the following properties are also
available, and may be useful
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144319503
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -270,6 +270,15 @@ package object config {
.longConf
.cr
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144319836
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,65 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144320027
--- Diff: docs/configuration.md ---
@@ -1192,6 +1192,14 @@ Apart from these, the following properties are also
available, and may be useful
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144319668
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -196,7 +196,11 @@ class HadoopRDD[K, V](
// add the credentials here as thi
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144311831
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,65 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user liutang123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144235787
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,16 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user liutang123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144235417
--- Diff: core/src/main/scala/org/apache/spark/rdd/NewHadoopRDD.scala ---
@@ -122,7 +122,10 @@ class NewHadoopRDD[K, V](
case _ =>
}
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144197159
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -196,7 +196,10 @@ class HadoopRDD[K, V](
// add the credentials here a
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144181321
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -196,7 +196,10 @@ class HadoopRDD[K, V](
// add the credentials here as
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144030461
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -196,7 +196,10 @@ class HadoopRDD[K, V](
// add the credentials here a
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144029852
--- Diff: docs/configuration.md ---
@@ -1211,6 +1211,14 @@ Apart from these, the following properties are also
available, and may be useful
dat
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144031728
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,16 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144030646
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -196,7 +196,10 @@ class HadoopRDD[K, V](
// add the credentials here a
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r144031167
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -510,4 +510,16 @@ class FileSuite extends SparkFunSuite with
LocalSparkContext {
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19464#discussion_r143984294
--- Diff: core/src/main/scala/org/apache/spark/rdd/NewHadoopRDD.scala ---
@@ -122,7 +122,10 @@ class NewHadoopRDD[K, V](
case _ =>
}
GitHub user liutang123 opened a pull request:
https://github.com/apache/spark/pull/19464
Spark 22233
## What changes were proposed in this pull request?
add spark.hadoop.filterOutEmptySplit confituration to allow user to filter
out empty split in HadoopRDD.
You can merge this p
33 matches
Mail list logo