[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21028 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188943212 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -870,6 +870,7 @@ case class

[GitHub] spark pull request #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.w...

2018-05-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21327 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r18894 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,113 @@ object JavaCode { }

[GitHub] spark pull request #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-05-17 Thread DylanGuedes
Github user DylanGuedes commented on a diff in the pull request: https://github.com/apache/spark/pull/21045#discussion_r188947896 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -90,6 +90,132 @@ case class

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188958139 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -388,7 +388,8 @@ case class

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90726/ Test PASSed. ---

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r188972342 --- Diff: python/pyspark/sql/tests.py --- @@ -3040,6 +3040,24 @@ def test_csv_sampling_ratio(self): .csv(rdd, samplingRatio=0.5).schema

[GitHub] spark pull request #21291: [SPARK-24242][SQL] RangeExec should have correct ...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21291#discussion_r188972061 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -39,7 +39,9 @@ class ConfigBehaviorSuite extends QueryTest with

[GitHub] spark issue #21353: [SPARK-24036][SS] Scheduler changes for continuous proce...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3301/

[GitHub] spark issue #21353: [SPARK-24036][SS] Scheduler changes for continuous proce...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21353 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21351: [SPARK-24002][SQL][BACKPORT-2.3] Task not serializable c...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21351 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3295/

[GitHub] spark issue #21351: [SPARK-24002][SQL][BACKPORT-2.3] Task not serializable c...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21351 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21106: [SPARK-23711][SQL] Add fallback generator for Uns...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21106#discussion_r188914060 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/UnsafeRowConverterSuite.scala --- @@ -24,25 +24,29 @@ import

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #90729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90729/testReport)** for PR 21317 at commit

[GitHub] spark issue #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.writeFul...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21327 **[Test build #90715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90715/testReport)** for PR 21327 at commit

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21282 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21282 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90721/ Test PASSed. ---

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21282 **[Test build #90721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90721/testReport)** for PR 21282 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90724/ Test PASSed. ---

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r188950746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,115 @@ object JavaCode {

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188952027 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -552,7 +553,8 @@ case class Slice(x:

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188954320 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1483,7 +1490,7 @@ case class

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188954781 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1324,15 +1329,17 @@ case class

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188953999 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1324,15 +1329,17 @@ case class

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188952482 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -902,6 +905,7 @@ case class

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188952300 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -837,6 +839,7 @@ case class

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188953863 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1126,9 +1130,9 @@ case class

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r188954468 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,115 @@ object JavaCode { }

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3199/ ---

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188952711 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1147,6 +1151,7 @@ case class

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188951710 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -388,7 +388,8 @@ case class

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21193 LGTM apart from a nit --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-05-17 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20894#discussion_r188966754 --- Diff: python/pyspark/sql/tests.py --- @@ -3040,6 +3040,24 @@ def test_csv_sampling_ratio(self): .csv(rdd, samplingRatio=0.5).schema

[GitHub] spark issue #21332: [SPARK-24236][SS] Continuous replacement for ShuffleExch...

2018-05-17 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21332 > As discussed in the other PR, I'm not sure about how we're integrating with the scheduler here, so I can't really give a more detailed review at this point. My bad, I'm preparing

[GitHub] spark pull request #21353: [SPARK-24036][SS] Scheduler changes for continuou...

2018-05-17 Thread xuanyuanking
GitHub user xuanyuanking opened a pull request: https://github.com/apache/spark/pull/21353 [SPARK-24036][SS] Scheduler changes for continuous processing shuffle support ## What changes were proposed in this pull request? This is the last part of the preview PRs, the mainly

[GitHub] spark pull request #21106: [SPARK-23711][SQL] Add fallback generator for Uns...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21106#discussion_r188913856 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -196,39 +197,33 @@ trait

[GitHub] spark pull request #21114: [SPARK-22371][CORE] Return None instead of throwi...

2018-05-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21114 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 @liyinan926 How can I trigger integration tests again? ``` Exception encountered when invoking run on a nested suite - Operation: [create] for kind: [Namespace] with name: [null] in

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21028 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization of priv...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21352 **[Test build #90733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90733/testReport)** for PR 21352 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #90734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90734/testReport)** for PR 21317 at commit

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-05-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r188950673 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ComplexTypeSuite.scala --- @@ -186,6 +186,37 @@ class ComplexTypeSuite

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r188950520 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,115 @@ object JavaCode {

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188960163 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -552,7 +553,8 @@ case class Slice(x:

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20894 I understand the default value of `enforceSchema` must be true to keep backward compatibility, but shall we suggest users to set it to false to avoid incorrect result in the document? ---

[GitHub] spark issue #21351: [SPARK-24002][SQL][BACKPORT-2.3] Task not serializable c...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21351 thanks, merging to 2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21353: [SPARK-24036][SS] Scheduler changes for continuou...

2018-05-17 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21353#discussion_r188974718 --- Diff: core/src/main/scala/org/apache/spark/SparkEnv.scala --- @@ -140,6 +140,7 @@ object SparkEnv extends Logging { private[spark] val

[GitHub] spark issue #21351: [SPARK-24002][SQL][BACKPORT-2.3] Task not serializable c...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21351 **[Test build #90727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90727/testReport)** for PR 21351 at commit

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21165 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization of priv...

2018-05-17 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21352 cc @ueshin @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization of priv...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21352 **[Test build #90730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90730/testReport)** for PR 21352 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90725/testReport)** for PR 21106 at commit

[GitHub] spark issue #21291: [SPARK-24242][SQL] RangeExec should have correct outputO...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21291 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90722/ Test PASSed. ---

[GitHub] spark issue #21291: [SPARK-24242][SQL] RangeExec should have correct outputO...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21291 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization of priv...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21352 adding `@transient` is no harm, I'm fine with it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r188940909 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -870,6 +870,7 @@ case class

[GitHub] spark issue #21329: [SPARK-24277][SQL] Code clean up in SQL module: HadoopMa...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21329 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3298/

[GitHub] spark issue #21329: [SPARK-24277][SQL] Code clean up in SQL module: HadoopMa...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21329 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90732/testReport)** for PR 21193 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3300/

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3199/ ---

[GitHub] spark issue #21351: [SPARK-24002][SQL][BACKPORT-2.3] Task not serializable c...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21351 **[Test build #90727 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90727/testReport)** for PR 21351 at commit

[GitHub] spark pull request #21353: [SPARK-24036][SS] Scheduler changes for continuou...

2018-05-17 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21353#discussion_r188974319 --- Diff: core/src/main/scala/org/apache/spark/Dependency.scala --- @@ -88,14 +96,53 @@ class ShuffleDependency[K: ClassTag, V: ClassTag, C:

[GitHub] spark pull request #21353: [SPARK-24036][SS] Scheduler changes for continuou...

2018-05-17 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21353#discussion_r188974568 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -213,6 +213,12 @@ private[spark] sealed trait MapOutputTrackerMessage

[GitHub] spark issue #21353: [SPARK-24036][SS] Scheduler changes for continuous proce...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21353 **[Test build #90736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90736/testReport)** for PR 21353 at commit

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90714/ Test FAILed. ---

[GitHub] spark issue #21343: [SPARK-24292][SQL] Proxy user cannot connect to HiveMeta...

2018-05-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21343 This problem seems to have been fixed, can you try [v2.3.1-rc1](https://github.com/apache/spark/releases/tag/v2.3.1-rc1)? --- -

[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21114 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21349 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21349 **[Test build #90728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90728/testReport)** for PR 21349 at commit

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21236 **[Test build #90720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90720/testReport)** for PR 21236 at commit

[GitHub] spark issue #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization of priv...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21352 in most of the cases these are lazy fields which are called only on the execution, so I don't think they are serialized. And in some cases (I am thinking to the ordering ones) with this patch we

[GitHub] spark issue #21329: [SPARK-24277][SQL] Code clean up in SQL module: HadoopMa...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21329 **[Test build #90731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90731/testReport)** for PR 21329 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90724/testReport)** for PR 21106 at commit

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90734/ Test PASSed. ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21291: [SPARK-24242][SQL] RangeExec should have correct ...

2018-05-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21291#discussion_r188953639 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -39,7 +39,9 @@ class ConfigBehaviorSuite extends QueryTest with

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #90734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90734/testReport)** for PR 21317 at commit

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r188956140 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,115 @@ object JavaCode {

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-05-17 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r188960491 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -344,6 +344,36 @@ case class

[GitHub] spark pull request #21353: [SPARK-24036][SS] Scheduler changes for continuou...

2018-05-17 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21353#discussion_r188975680 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ContinuousShuffleMapTask.scala --- @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21349 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21349 **[Test build #90714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90714/testReport)** for PR 21349 at commit

[GitHub] spark pull request #21165: [Spark-20087][CORE] Attach accumulators / metrics...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21165#discussion_r188925916 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -1868,15 +1868,26 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #21165: [Spark-20087][CORE] Attach accumulators / metrics...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21165#discussion_r188925889 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -1868,15 +1868,26 @@ class DAGSchedulerSuite extends

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3197/ ---

[GitHub] spark issue #21316: [SPARK-20538][SQL] Wrap Dataset.reduce with withNewRddEx...

2018-05-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21316 **[Test build #90718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90718/testReport)** for PR 21316 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21106: [SPARK-23711][SQL] Add fallback generator for UnsafeProj...

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90725/ Test FAILed. ---

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21236 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21236 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90720/ Test PASSed. ---

[GitHub] spark issue #21342: [SPARK-24294] Throw SparkException when OOM in Broadcast...

2018-05-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21342 Basically, I agree with this two-step direction. However, I think that it is too risky to simply throw and catch `SparkException`. This is because other `SparkException` may be caught in step 2. For

[GitHub] spark issue #21327: [SPARK-24107][CORE][followup] ChunkedByteBuffer.writeFul...

2018-05-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21327 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r188955270 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,115 @@ object JavaCode { }

<    1   2   3   4   5   6   >