[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-13 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r144688074 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -277,13 +292,25 @@ class CodegenContext

[GitHub] spark issue #19497: [SPARK-21549][CORE] Respect OutputFormats with no/invali...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19497 **[Test build #82753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82753/testReport)** for PR 19497 at commit [`a319df3`](https://github.com/apache/spark/commit/a3

[GitHub] spark pull request #19491: [SPARK-22273][SQL] Fix key/value schema field nam...

2017-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19491 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19491: [SPARK-22273][SQL] Fix key/value schema field names in H...

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19491 Thanks! Merged to master/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark issue #19497: [SPARK-21549][CORE] Respect OutputFormats with no/invali...

2017-10-13 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19497 +CC @HyukjinKwon, @steveloughran Sorry for messing up PR #19487 The only change in this PR is to use `::invalid::` instead of `test:` in the test to address @steveloughran's comment.

[GitHub] spark pull request #19497: [SPARK-21549][CORE] Respect OutputFormats with no...

2017-10-13 Thread mridulm
GitHub user mridulm opened a pull request: https://github.com/apache/spark/pull/19497 [SPARK-21549][CORE] Respect OutputFormats with no/invalid output directory provided ## What changes were proposed in this pull request? PR #19294 added support for null's - but s

[GitHub] spark issue #19496: [SPARK-22271][SQL]mean overflows and returns null for so...

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19496 OK to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark pull request #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-13 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19451#discussion_r144687542 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1242,6 +1244,51 @@ object ReplaceIntersectWithSemiJoin e

[GitHub] spark pull request #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19451#discussion_r144687479 --- Diff: sql/core/src/test/resources/sql-tests/inputs/except.sql --- @@ -0,0 +1,43 @@ +-- Tests different scenarios of except operation +create t

[GitHub] spark pull request #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19451#discussion_r144687477 --- Diff: sql/core/src/test/resources/sql-tests/inputs/except.sql --- @@ -0,0 +1,43 @@ +-- Tests different scenarios of except operation +create t

[GitHub] spark pull request #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19451#discussion_r144687484 --- Diff: sql/core/src/test/resources/sql-tests/inputs/except.sql --- @@ -0,0 +1,43 @@ +-- Tests different scenarios of except operation +create t

[GitHub] spark issue #19487: [SPARK-21549][CORE] Respect OutputFormats with no/invali...

2017-10-13 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19487 crap, my branch got messed up. I will resubmit the PR - sorry for the mess up. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #19487: [SPARK-21549][CORE] Respect OutputFormats with no...

2017-10-13 Thread mridulm
Github user mridulm closed the pull request at: https://github.com/apache/spark/pull/19487 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19451#discussion_r144687436 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1242,6 +1244,51 @@ object ReplaceIntersectWithSemi

[GitHub] spark pull request #18979: [SPARK-21762][SQL] FileFormatWriter/BasicWriteTas...

2017-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18979 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19451#discussion_r144687416 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1242,6 +1244,51 @@ object ReplaceIntersectWithSemi

[GitHub] spark issue #18979: [SPARK-21762][SQL] FileFormatWriter/BasicWriteTaskStatsT...

2017-10-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18979 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19487: [SPARK-21549][CORE] Respect OutputFormats with no/invali...

2017-10-13 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19487 @HyukjinKwon That was exactly the initial solution I tested locally when we saw the problem with Phoenix. The reason to expand it was two fold: a) This change preserves existing behavior from

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19464 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82752/ Test PASSed. ---

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19464 **[Test build #82752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82752/testReport)** for PR 19464 at commit [`a6818b6`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #19487: [SPARK-21549][CORE] Respect OutputFormats with no/invali...

2017-10-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19487 @mridulm, what do you think about dealing with empty string for now and other cases later if we can't male sure for other cases for now? I guess the actual issue found is about empty string anyw

[GitHub] spark issue #19496: [SPARK-22271][SQL]mean overflows and returns null for so...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19496 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19496: [SPARK-22271][SQL]mean overflows and returns null...

2017-10-13 Thread huaxingao
GitHub user huaxingao opened a pull request: https://github.com/apache/spark/pull/19496 [SPARK-22271][SQL]mean overflows and returns null for some decimal variables ## What changes were proposed in this pull request? In Average.scala, it has ``` override

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r144685307 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2103,4 +2103,35 @@ class DataFrameSuite extends QueryTest with SharedSQL

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19464 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r144684909 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2103,4 +2103,35 @@ class DataFrameSuite extends QueryTest with SharedSQL

[GitHub] spark issue #19495: [SPARK-22278][SS] Expose current event time watermark an...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82751/ Test PASSed. ---

[GitHub] spark issue #19495: [SPARK-22278][SS] Expose current event time watermark an...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19495 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19495: [SPARK-22278][SS] Expose current event time watermark an...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19495 **[Test build #82751 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82751/testReport)** for PR 19495 at commit [`67114ab`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r144684397 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -277,13 +292,25 @@ class CodegenContext {

[GitHub] spark pull request #19488: [SPARK-22266][SQL] The same aggregate function wa...

2017-10-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19488#discussion_r144684245 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -205,14 +205,17 @@ object PhysicalAggregation {

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82749/ Test PASSed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82749/testReport)** for PR 19467 at commit [`971f579`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19464 **[Test build #82752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82752/testReport)** for PR 19464 at commit [`a6818b6`](https://github.com/apache/spark/commit/a6

[GitHub] spark pull request #19464: [SPARK-22233] [core] Allow user to filter out emp...

2017-10-13 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19464#discussion_r144683771 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -510,4 +510,87 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r144683715 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2103,4 +2103,35 @@ class DataFrameSuite extends QueryTest with SharedSQL

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19452 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82748/ Test PASSed. ---

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19452 **[Test build #82748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82748/testReport)** for PR 19452 at commit [`0a753ed`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19491: [SPARK-22273][SQL] Fix key/value schema field names in H...

2017-10-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19491 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-13 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19272 @vanzin Thanks for the review. I'll address the comments ASAP. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark pull request #19374: [SPARK-22145][MESOS] fix supervise with checkpoin...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19374#discussion_r144682661 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -896,8 +913,8 @@ private[spark

[GitHub] spark pull request #19374: [SPARK-22145][MESOS] fix supervise with checkpoin...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19374#discussion_r144682434 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -373,10 +374,16 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #19374: [SPARK-22145][MESOS] fix supervise with checkpoin...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19374#discussion_r144682554 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -374,6 +375,15 @@ private[spar

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r144682189 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -28,6 +28,8 @@ import com.google

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r144682211 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -175,14 +175,22 @@ trait MesosSc

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r144682163 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -380,7 +389,8 @@

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r144682379 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -469,6 +474,12 @@ trait MesosSch

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144682312 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -17,10 +17,14 @@

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144681733 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -170,9 +175,119 @@ private

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144680213 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala --- @@ -159,7 +160,8 @@ pr

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144680478 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -170,9 +175,119 @@ private

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144680608 --- Diff: docs/running-on-mesos.md --- @@ -522,6 +522,43 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144680489 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -170,9 +175,119 @@ private

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144680353 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -122,7 +126,8 @@ private[m

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144681758 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -170,9 +175,119 @@ private

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-13 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r144680448 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -122,7 +126,8 @@ private[m

[GitHub] spark issue #19495: [SPARK-22278][SS] Expose current event time watermark an...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19495 **[Test build #82751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82751/testReport)** for PR 19495 at commit [`67114ab`](https://github.com/apache/spark/commit/67

[GitHub] spark pull request #19495: [SPARK-22278][SS] Expose current event time water...

2017-10-13 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/19495 [SPARK-22278][SS] Expose current event time watermark and current processing time in GroupState ## What changes were proposed in this pull request? Complex state-updating and/or timeout-handl

[GitHub] spark pull request #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add su...

2017-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19385 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19385 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r144678394 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2103,4 +2103,35 @@ class DataFrameSuite extends QueryTest with SharedSQL

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19385 **[Test build #82750 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82750/testReport)** for PR 19385 at commit [`ced3e58`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82750/ Test PASSed. ---

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19385 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19385 **[Test build #82750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82750/testReport)** for PR 19385 at commit [`ced3e58`](https://github.com/apache/spark/commit/ce

[GitHub] spark issue #16801: [SPARK-13619] [WEBUI] [CORE] Jobs page UI shows wrong nu...

2017-10-13 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/16801 Will identify better solution to fix this issue and create a new PR, closing it. --- - To unsubscribe, e-mail: reviews-un

[GitHub] spark pull request #16801: [SPARK-13619] [WEBUI] [CORE] Jobs page UI shows w...

2017-10-13 Thread devaraj-kavali
Github user devaraj-kavali closed the pull request at: https://github.com/apache/spark/pull/16801 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82749/testReport)** for PR 19467 at commit [`971f579`](https://github.com/apache/spark/commit/97

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19452 **[Test build #82748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82748/testReport)** for PR 19452 at commit [`0a753ed`](https://github.com/apache/spark/commit/0a

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82747/ Test FAILed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82747/testReport)** for PR 19467 at commit [`84ac2d8`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19396: [SPARK-22172][CORE] Worker hangs when the external shuff...

2017-10-13 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/19396 @jerryshao Please let me know if you don't convince with the above comment, I can make the changes to PR to make Worker do down on external shuffle service start failure. --- -

[GitHub] spark pull request #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add su...

2017-10-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19385#discussion_r144671613 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -174,6 +182,9 @@ p

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/19385 Thanks @vanzin for looking into this. > This only solves half the problem, right? What about cluster mode? Yes, it solves the Mesos/Client mode. For Mesos/Cluster mode, I think the

[GitHub] spark pull request #19435: [MINOR][SS] "keyWithIndexToNumValues" -> "keyWith...

2017-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19435 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r144668015 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -23,21 +23,37 @@ import org.apache.spark.sql.cat

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r144668103 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -84,25 +84,45 @@ private[sql] trait ColumnarBatchScan extends

[GitHub] spark issue #19385: [SPARK-11034] [LAUNCHER] [MESOS] Launcher: add support f...

2017-10-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19385 This only solves half the problem, right? What about cluster mode? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.

[GitHub] spark issue #19435: [MINOR][SS] "keyWithIndexToNumValues" -> "keyWithIndexTo...

2017-10-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19435 Thanks! Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82746/ Test PASSed. ---

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82746/testReport)** for PR 18747 at commit [`750b230`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #19452: [SPARK-22136][SS] Evaluate one-sided conditions e...

2017-10-13 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19452#discussion_r144664962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala --- @@ -349,12 +356,28 @@ case class Streaming

[GitHub] spark issue #18979: [SPARK-21762][SQL] FileFormatWriter/BasicWriteTaskStatsT...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82745/ Test PASSed. ---

[GitHub] spark issue #18979: [SPARK-21762][SQL] FileFormatWriter/BasicWriteTaskStatsT...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18979 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18979: [SPARK-21762][SQL] FileFormatWriter/BasicWriteTaskStatsT...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18979 **[Test build #82745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82745/testReport)** for PR 18979 at commit [`c0e81a1`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19467 LGTM, assuming tests pass. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82747/testReport)** for PR 19467 at commit [`84ac2d8`](https://github.com/apache/spark/commit/84

[GitHub] spark pull request #19488: [SPARK-22266][SQL] The same aggregate function wa...

2017-10-13 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/19488#discussion_r144656360 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -205,14 +205,17 @@ object PhysicalAggregation {

[GitHub] spark pull request #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks...

2017-10-13 Thread superbobry
Github user superbobry commented on a diff in the pull request: https://github.com/apache/spark/pull/19458#discussion_r144655974 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala --- @@ -100,7 +100,16 @@ private[spark] class DiskBlockManager(conf: SparkC

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82743/ Test PASSed. ---

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19452 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19452 **[Test build #82743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82743/testReport)** for PR 19452 at commit [`94dfa85`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #19488: [SPARK-22266][SQL] The same aggregate function wa...

2017-10-13 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/19488#discussion_r144651235 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -205,14 +205,17 @@ object PhysicalAggregation {

[GitHub] spark pull request #19452: [SPARK-22136][SS] Evaluate one-sided conditions e...

2017-10-13 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19452#discussion_r144651101 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingSymmetricHashJoinHelperSuite.scala --- @@ -0,0 +1,118 @@ +/* + * Licensed to

[GitHub] spark pull request #19235: [SPARK-14387][SPARK-19459][SQL] Enable Hive-1.x O...

2017-10-13 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/19235 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

  1   2   3   4   >