[GitHub] spark pull request #21419: Branch 2.2

2018-05-23 Thread gentlewangyu
GitHub user gentlewangyu opened a pull request: https://github.com/apache/spark/pull/21419 Branch 2.2 ## What changes were proposed in this pull request? compiling spark with scala-2.10 should use the -P parameter instead of -D ## How was this patch tested?

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3540/ Tes

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21416 **[Test build #91089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91089/testReport)** for PR 21416 at commit [`de3296c`](https://github.com/apache/spark/commit/de

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 +@cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spar

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-05-23 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r190479026 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -162,7 +162,9 @@ case class FilterExec(condition: Expr

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-05-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21402 while we are here, could we also add or at least propose some metrics around this, such as number of open block failure, or even number of block threads? we have suffer a lot from the l

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21415 **[Test build #91087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91087/testReport)** for PR 21415 at commit [`0aef16b`](https://github.com/apache/spark/commit/0a

[GitHub] spark issue #21391: [SPARK-24343][SQL] Avoid shuffle for the bucketed table ...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21391 **[Test build #91088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91088/testReport)** for PR 21391 at commit [`d9a440a`](https://github.com/apache/spark/commit/d9

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21415 jenkins, retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21399: [SPARK-22269][BUILD] Run Java linter via SBT for ...

2018-05-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21399 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21399: [SPARK-22269][BUILD] Run Java linter via SBT for Jenkins

2018-05-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21399 Merged to master. Thanks all, @dongjoon-hyun, @advancedxy, @vanzin and @felixcheung. --- - To unsubscribe, e-mail: r

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21399 ^ this one was ran with Maven. I am merging this in. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For a

[GitHub] spark pull request #21418: Branch 2.2

2018-05-23 Thread gentlewangyu
Github user gentlewangyu closed the pull request at: https://github.com/apache/spark/pull/21418 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark pull request #21392: [SPARK-24063][SS] Control maximum epoch backlog f...

2018-05-23 Thread efimpoberezkin
Github user efimpoberezkin commented on a diff in the pull request: https://github.com/apache/spark/pull/21392#discussion_r190473131 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -45,6 +45,11 @@ private[sql] cas

[GitHub] spark pull request #21417: Branch 2.0

2018-05-23 Thread gentlewangyu
Github user gentlewangyu closed the pull request at: https://github.com/apache/spark/pull/21417 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3539/ Tes

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21295 **[Test build #91086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91086/testReport)** for PR 21295 at commit [`497bdd8`](https://github.com/apache/spark/commit/49

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190472138 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -219,7 +219,11 @@ object ReorderAssociativeOperator e

[GitHub] spark issue #21418: Branch 2.2

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21418 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21418: Branch 2.2

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21418 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21415 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21295 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21415 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91079/ Test PASSed. ---

[GitHub] spark issue #21310: [SPARK-24256][SQL] SPARK-24256: ExpressionEncoder should...

2018-05-23 Thread fangshil
Github user fangshil commented on the issue: https://github.com/apache/spark/pull/21310 @viirya thanks for the feedback. We internally customized the AvroEncoder based on the open source PR, since it never gets merged into spark-avro. we propose this feature since it should apply to

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91080/ Test FAILed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3538/ Tes

[GitHub] spark pull request #21418: Branch 2.2

2018-05-23 Thread gentlewangyu
GitHub user gentlewangyu opened a pull request: https://github.com/apache/spark/pull/21418 Branch 2.2 ## What changes were proposed in this pull request? compiling spark with scala-2.10 should use the -p parameter instead of -d ## How was this patch tested? You ca

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21415: [SPARK-24244][SPARK-24368][SQL] Passing only required co...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21415 **[Test build #91079 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91079/testReport)** for PR 21415 at commit [`0aef16b`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21295 **[Test build #91080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91080/testReport)** for PR 21295 at commit [`497bdd8`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91084/ Test FAILed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #91084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91084/testReport)** for PR 21399 at commit [`294e189`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190470525 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala --- @@ -397,6 +399,68 @@ class ColumnExpressionSuite extends QueryTest wit

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3537/ Tes

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21417: Branch 2.0

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21417 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #91085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91085/testReport)** for PR 21399 at commit [`6943ff8`](https://github.com/apache/spark/commit/69

[GitHub] spark issue #21417: Branch 2.0

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21417 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21417: Branch 2.0

2018-05-23 Thread gentlewangyu
GitHub user gentlewangyu opened a pull request: https://github.com/apache/spark/pull/21417 Branch 2.0 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this pa

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #91084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91084/testReport)** for PR 21399 at commit [`294e189`](https://github.com/apache/spark/commit/29

[GitHub] spark pull request #21408: [SPARK-24364][SS] Prevent InMemoryFileIndex from ...

2018-05-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21408 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21408: [SPARK-24364][SS] Prevent InMemoryFileIndex from failing...

2018-05-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21408 Merged to master and branch-2.3. Thanks @cloud-fan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3536/ Tes

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21390 **[Test build #91083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91083/testReport)** for PR 21390 at commit [`4a4ab59`](https://github.com/apache/spark/commit/4a

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21390 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91074/ Test PASSed. ---

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #91074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91074/testReport)** for PR 21366 at commit [`4f58393`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91078/ Test FAILed. ---

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21390 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21390 **[Test build #91078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91078/testReport)** for PR 21390 at commit [`4a4ab59`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21408: [SPARK-24364][SS] Prevent InMemoryFileIndex from failing...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21408 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21408: [SPARK-24364][SS] Prevent InMemoryFileIndex from failing...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21408 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91077/ Test PASSed. ---

[GitHub] spark issue #21408: [SPARK-24364][SS] Prevent InMemoryFileIndex from failing...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21408 **[Test build #91077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91077/testReport)** for PR 21408 at commit [`a5614f8`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91076/ Test FAILed. ---

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21399 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21399: [SPARK-22269][BUILD][test-maven] Run Java linter via SBT...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21399 **[Test build #91076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91076/testReport)** for PR 21399 at commit [`7bb0eb3`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21411: [SPARK-24367][SQL]Parquet: use JOB_SUMMARY_LEVEL instead...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21411 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21405: [SPARK-24361][SQL] Polish code block manipulation API

2018-05-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21405 @cloud-fan Thanks. I give a use case of splitting code into method in the PR description. I think it can show the basic idea. --- ---

[GitHub] spark issue #21411: [SPARK-24367][SQL]Parquet: use JOB_SUMMARY_LEVEL instead...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21411 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91073/ Test PASSed. ---

[GitHub] spark issue #21411: [SPARK-24367][SQL]Parquet: use JOB_SUMMARY_LEVEL instead...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21411 **[Test build #91073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91073/testReport)** for PR 21411 at commit [`3a6a87b`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r190461717 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r190461628 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r190461517 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed to the

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-05-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r190461402 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed to the

[GitHub] spark issue #20708: [SPARK-21209][MLLLIB] Implement Incremental PCA algorith...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20708 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21366 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91075/ Test FAILed. ---

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

2018-05-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21319 The problem is that plan visitor can only visit plan but not changing it, and pushing down operators to data source needs to remove filters from plan... --- -

[GitHub] spark pull request #21406: [Minor][Core] Cleanup unused vals in `DAGSchedule...

2018-05-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21406 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21366: [SPARK-24248][K8S][WIP] Use the Kubernetes API to popula...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21366 **[Test build #91075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91075/testReport)** for PR 21366 at commit [`2a2374c`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21406: [Minor][Core] Cleanup unused vals in `DAGScheduler.handl...

2018-05-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21406 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21372 Thank you, @cloud-fan , @gatorsmile , @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-23 Thread cxzl25
Github user cxzl25 commented on the issue: https://github.com/apache/spark/pull/21311 @cloud-fan Thank you very much for your help. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark pull request #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21372 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21372 LGTM Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21372 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17583 **[Test build #91082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91082/testReport)** for PR 17583 at commit [`47aa749`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17583 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91082/ Test FAILed. ---

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17583 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #17583: [SPARK-20271]Add FuncTransformer to simplify custom tran...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17583 **[Test build #91082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91082/testReport)** for PR 17583 at commit [`47aa749`](https://github.com/apache/spark/commit/47

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21311 thanks, merging to master/2.3/2.2/2.1/2.0! There is no conflict so I backported all the way to 2.0. I'll watch the jenkins build in the few days. ---

[GitHub] spark pull request #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate th...

2018-05-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21311 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3535/ Tes

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21069 **[Test build #91081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91081/testReport)** for PR 21069 at commit [`9281ae2`](https://github.com/apache/spark/commit/92

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

2018-05-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21069 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21383: [SPARK-23754][Python] Re-raising StopIteration in client...

2018-05-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21383 cc @icexelloss too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3534/ Tes

[GitHub] spark pull request #21383: [SPARK-23754][Python] Re-raising StopIteration in...

2018-05-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21383#discussion_r190453705 --- Diff: python/pyspark/rdd.py --- @@ -791,9 +792,11 @@ def foreach(self, f): >>> def f(x): print(x) >>> sc.parallelize([1, 2,

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21295 Congratulation, @rdblue ! :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21295 **[Test build #91080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91080/testReport)** for PR 21295 at commit [`497bdd8`](https://github.com/apache/spark/commit/49

[GitHub] spark issue #21372: [SPARK-24322][BUILD] Upgrade Apache ORC to 1.4.4

2018-05-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21372 Finally! Could you review this again, @HyukjinKwon , @gatorsmile , @cloud-fan ? --- - To unsubscribe, e-mail: reviews-unsu

[GitHub] spark issue #21295: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase w...

2018-05-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21295 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

  1   2   3   4   5   6   7   8   >