[GitHub] spark pull request #19924: [SPARK-22187][SS][REVERT] Revert change in state ...

2017-12-07 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/19924 [SPARK-22187][SS][REVERT] Revert change in state row format for mapGroupsWithState ## What changes were proposed in this pull request? #19416 changed the format in which rows were encoded in

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19920 **[Test build #84620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84620/testReport)** for PR 19920 at commit [`9d46804`](https://github.com/apache/spark/commit/9d

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84614/ Test PASSed. ---

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #84614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84614/testReport)** for PR 19683 at commit [`7cb9454`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155657473 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -17,16 +17,15 @@ package org.apache.spark.status -i

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155657274 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -30,7 +31,8 @@ private[history] class HistoryPage(parent: HistorySe

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155657306 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -88,4 +90,9 @@ private[history] class HistoryPage(parent: HistorySe

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155653359 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -30,7 +31,8 @@ private[history] class HistoryPage(parent: HistoryServe

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155640885 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -17,16 +17,15 @@ package org.apache.spark.status -impo

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155653408 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -88,4 +90,9 @@ private[history] class HistoryPage(parent: HistoryServe

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19864 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84612/ Test PASSed. ---

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19864 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19864 **[Test build #84612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84612/testReport)** for PR 19864 at commit [`c2a92d4`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19920 err, ignore that --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19920 @shaneknapp any idea why the PR builder is not testing this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org Fo

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155647982 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at == pa.float64(

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19923 yay! LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155647741 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at == pa.float64(

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19923 **[Test build #84619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84619/testReport)** for PR 19923 at commit [`12de708`](https://github.com/apache/spark/commit/12

[GitHub] spark issue #19923: [SPARK-22721] BytesToBytesMap peak memory not updated.

2017-12-07 Thread juliuszsompolski
Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19923 Sorry @hvanhovell for not getting it fully right the first time... --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark pull request #19923: [SPARK-22721] BytesToBytesMap peak memory not upd...

2017-12-07 Thread juliuszsompolski
GitHub user juliuszsompolski opened a pull request: https://github.com/apache/spark/pull/19923 [SPARK-22721] BytesToBytesMap peak memory not updated. ## What changes were proposed in this pull request? Follow-up to earlier commit. The peak memory of BytesToBytesMap is no

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84611/ Test FAILed. ---

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84611/testReport)** for PR 19811 at commit [`7872bb4`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19920 **[Test build #84618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84618/testReport)** for PR 19920 at commit [`29d8735`](https://github.com/apache/spark/commit/29

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19920 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #84617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84617/testReport)** for PR 19893 at commit [`644ee6a`](https://github.com/apache/spark/commit/64

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155639843 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155639821 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155639701 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155639667 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark issue #18995: [SPARK-21787][SQL] Support for pushing down filters for ...

2017-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18995 Hi, @cloud-fan . Could you review this PR again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19843: [SPARK-22644][ML][TEST][WIP] Make ML testsuite su...

2017-12-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19843#discussion_r155636618 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTest.scala --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #19906: [SPARK-22516][SQL] Bump up Univocity version to 2.5.9

2017-12-07 Thread smurakozi
Github user smurakozi commented on the issue: https://github.com/apache/spark/pull/19906 Thanks for your help and reviews @HyukjinKwon, @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155632345 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseri

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155631788 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -25,6 +25,7 @@ import scala.collection.JavaConverters._ import org.

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155631687 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -29,6 +29,7 @@ import org.apache.spark.executor.TaskMetrics impo

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155631377 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala --- @@ -188,11 +188,11 @@ class HistoryServer( } def

[GitHub] spark issue #18995: [SPARK-21787][SQL] Support for pushing down filters for ...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18995 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18995: [SPARK-21787][SQL] Support for pushing down filters for ...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84609/ Test PASSed. ---

[GitHub] spark issue #18995: [SPARK-21787][SQL] Support for pushing down filters for ...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18995 **[Test build #84609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84609/testReport)** for PR 18995 at commit [`18accdd`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #19843: [SPARK-22644][ML][TEST][WIP] Make ML testsuite su...

2017-12-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19843#discussion_r155629428 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTest.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #19843: [SPARK-22644][ML][TEST][WIP] Make ML testsuite su...

2017-12-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19843#discussion_r155629359 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTest.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84610/ Test FAILed. ---

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84610/testReport)** for PR 19811 at commit [`1648d91`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #84616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84616/testReport)** for PR 19884 at commit [`93b1eb3`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84616/ Test FAILed. ---

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155626333 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155626249 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at == pa.f

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #84616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84616/testReport)** for PR 19884 at commit [`93b1eb3`](https://github.com/apache/spark/commit/93

[GitHub] spark issue #19922: [SPARK-22688][SQL][HOTFIX] Upgrade Janino version to 3.0...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19922 **[Test build #84615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84615/testReport)** for PR 19922 at commit [`c3a0870`](https://github.com/apache/spark/commit/c3

[GitHub] spark pull request #19922: [SPARK-22688][SQL][HOTFIX] Upgrade Janino version...

2017-12-07 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/19922 [SPARK-22688][SQL][HOTFIX] Upgrade Janino version to 3.0.8 ## What changes were proposed in this pull request? Hotfix inadvertent change to xmlbuilder dep when updating Janino. See backp

[GitHub] spark pull request #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDi...

2017-12-07 Thread bolkedebruin
Github user bolkedebruin commented on a diff in the pull request: https://github.com/apache/spark/pull/19911#discussion_r155624133 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -48,5 +48,7 @@ private object DB2Dialect extends JdbcDialect {

[GitHub] spark pull request #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDi...

2017-12-07 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/19911#discussion_r155622687 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -48,5 +48,7 @@ private object DB2Dialect extends JdbcDialect {

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-07 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/19683 My guess is that it's safe to do so in our case because of the immediate projection that happens. In general, emitting JoinedRows where the RHS row is shared between all JoinedRows could be a problem

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-07 Thread uzadude
Github user uzadude commented on the issue: https://github.com/apache/spark/pull/19683 @henryr I understand what you're saying. I'm not sure why there is the UnsafeProject in the end of the function, but it's commented in this PR that fixes [SPARK-13476] without much elaboration. -

[GitHub] spark pull request #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDi...

2017-12-07 Thread bolkedebruin
Github user bolkedebruin commented on a diff in the pull request: https://github.com/apache/spark/pull/19911#discussion_r155619800 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -48,5 +48,7 @@ private object DB2Dialect extends JdbcDialect {

[GitHub] spark pull request #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDi...

2017-12-07 Thread bolkedebruin
Github user bolkedebruin commented on a diff in the pull request: https://github.com/apache/spark/pull/19911#discussion_r155619721 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -48,5 +48,7 @@ private object DB2Dialect extends JdbcDialect {

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19717 With some bug fixes in https://github.com/apache/spark/pull/19717/commits/83d0b9cc29eec2654308344ac8b2208e5e3f08df, I was able to run `SparkPi` on a GKE cluster. --- --

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #84614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84614/testReport)** for PR 19683 at commit [`7cb9454`](https://github.com/apache/spark/commit/7c

[GitHub] spark issue #19717: [SPARK-22646] [Submission] Spark on Kubernetes - basic s...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84613/testReport)** for PR 19717 at commit [`83d0b9c`](https://github.com/apache/spark/commit/83

[GitHub] spark issue #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDialect

2017-12-07 Thread danielvdende
Github user danielvdende commented on the issue: https://github.com/apache/spark/pull/19911 @dongjoon-hyun ok made the changes, also replaced the test that was in place for `isCascadingTruncateTable` with one for the `getTruncateQuery` method. Right now, I've left the method in `JdbcD

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19861 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19861 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84607/ Test PASSed. ---

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19861 **[Test build #84607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84607/testReport)** for PR 19861 at commit [`6b4fcab`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19864 **[Test build #84612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84612/testReport)** for PR 19864 at commit [`c2a92d4`](https://github.com/apache/spark/commit/c2

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155611800 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseriali

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155611738 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseri

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155610889 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseri

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155610181 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -29,6 +29,7 @@ import org.apache.spark.executor.TaskMetrics import

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155609975 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // scalasty

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84611/testReport)** for PR 19811 at commit [`7872bb4`](https://github.com/apache/spark/commit/78

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155609528 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -29,6 +29,7 @@ import org.apache.spark.executor.TaskMetrics impo

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155608281 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155606762 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // scalasty

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155605695 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19893 > All in all if you have an approach which solves context reuse feel free to share. I don't have one, but it feels sub-optimal to add code that will knowingly trigger false positives in a la

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155603886 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseriali

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155603905 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseriali

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155603357 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -29,6 +29,7 @@ import org.apache.spark.executor.TaskMetrics import

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155603588 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -25,6 +25,7 @@ import scala.collection.JavaConverters._ import org.apa

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155604279 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -24,27 +24,32 @@ import com.fasterxml.jackson.databind.annotation.JsonDeseriali

[GitHub] spark pull request #19920: [SPARK-21672][CORE] Remove SHS-specific applicati...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19920#discussion_r155603153 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala --- @@ -188,11 +188,11 @@ class HistoryServer( } def ge

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155603848 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/19893 Yeah, this is fully true. This enhancement is definitely will not solve the issues once and for all. The problems were hidden till now and we would like to make a step ahead and make it at lea

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155602584 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // scalasty

[GitHub] spark pull request #19912: [SPARK-22719][SQL]Refactor ConstantPropagation

2017-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19912 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155601696 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark issue #19912: [SPARK-22719][SQL]Refactor ConstantPropagation

2017-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19912 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155600929 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark issue #19912: [SPARK-22719][SQL]Refactor ConstantPropagation

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19912 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84605/ Test PASSed. ---

[GitHub] spark issue #19912: [SPARK-22719][SQL]Refactor ConstantPropagation

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19912 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19912: [SPARK-22719][SQL]Refactor ConstantPropagation

2017-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19912 **[Test build #84605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84605/testReport)** for PR 19912 at commit [`433587a`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155599671 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // scalasty

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2017-12-07 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r155598719 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -34,12 +36,53 @@ abstract class SparkFunSuite with Logging { // s

[GitHub] spark issue #19902: [SPARK-22452][SQL]Add getInt, getLong, getBoolean to Dat...

2017-12-07 Thread skambha
Github user skambha commented on the issue: https://github.com/apache/spark/pull/19902 Opened a new PR to add the getDouble method. https://github.com/apache/spark/pull/19921 --- - To unsubscribe, e-mail: reviews-

[GitHub] spark issue #19921: [SPARK-22452][SQL] Add getDouble to DataSourceV2Options

2017-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19921 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   >