[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20930 **[Test build #89870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89870/testReport)** for PR 20930 at commit

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184276403 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184274946 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #21138: [SPARK-24062][Thrift Server] Fix SASL encryption ...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21138 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21138 Merging to master and branch 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21160: [SPARK-24094][SS][MINOR] Change description strings of v...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21160 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89866/ Test PASSed. ---

[GitHub] spark issue #21160: [SPARK-24094][SS][MINOR] Change description strings of v...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21160 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21160: [SPARK-24094][SS][MINOR] Change description strings of v...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21160 **[Test build #89866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89866/testReport)** for PR 21160 at commit

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #89869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89869/testReport)** for PR 21156 at commit

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21138 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89864/ Test PASSed. ---

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21138 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21138 **[Test build #89864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89864/testReport)** for PR 21138 at commit

[GitHub] spark pull request #21011: [SPARK-23916][SQL] Add array_join function

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21011 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21011: [SPARK-23916][SQL] Add array_join function

2018-04-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21011 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2685/

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #89868 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89868/testReport)** for PR 20146 at commit

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r184266872 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -288,6 +288,114 @@ case class

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184266100 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184265979 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #19404: [SPARK-21760] [Streaming] Fix for Structured stre...

2018-04-25 Thread rekhajoshm
Github user rekhajoshm closed the pull request at: https://github.com/apache/spark/pull/19404 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19404: [SPARK-21760] [Streaming] Fix for Structured streaming t...

2018-04-25 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/19404 Thanks for the good inputs.Closing this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21162: shaded guava is not used anywhere, seems guava is not sh...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21162 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21162: shaded guava is not used anywhere, seems guava is not sh...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21162 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21162: shaded guava is not used anywhere, seems guava is...

2018-04-25 Thread yileic
GitHub user yileic opened a pull request: https://github.com/apache/spark/pull/21162 shaded guava is not used anywhere, seems guava is not shaded anymore ## What changes were proposed in this pull request? remove unused code to reduce confusion (Please fill in changes

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184262594 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -117,47 +118,18 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184262552 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -300,6 +333,49 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184262545 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -191,24 +163,85 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184261968 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -168,9 +140,9 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184261902 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -191,24 +163,85 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184262013 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -191,24 +163,85 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184262044 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -191,24 +163,85 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184261996 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -168,9 +140,9 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184262129 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -117,47 +118,18 @@ case class

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r184261947 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -300,6 +333,49 @@ case class

[GitHub] spark pull request #21008: [SPARK-23902][SQL] Add roundOff flag to months_be...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21008 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21008: [SPARK-23902][SQL] Add roundOff flag to months_between

2018-04-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21008 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184260815 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184260597 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184260210 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #21040: [SPARK-23930][SQL] Add slice function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21040#discussion_r184257279 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -105,4 +105,28 @@ class

[GitHub] spark pull request #21040: [SPARK-23930][SQL] Add slice function

2018-04-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21040#discussion_r184257274 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -287,3 +287,101 @@ case class

[GitHub] spark issue #21161: [SPARK-21645]left outer join synchronize the condition f...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21161 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21118 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89863/ Test PASSed. ---

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21118 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21161: left outer join synchronize the condition from left tabl...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21161 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21118 **[Test build #89863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89863/testReport)** for PR 21118 at commit

[GitHub] spark pull request #21161: left outer join synchronize the condition from le...

2018-04-25 Thread shining1989
GitHub user shining1989 opened a pull request: https://github.com/apache/spark/pull/21161 left outer join synchronize the condition from left table to right ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #89867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89867/testReport)** for PR 21156 at commit

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184255020 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20930: [SPARK-23811][Core] FetchFailed comes before Succ...

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20930#discussion_r184254856 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2399,84 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark issue #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20560 @rxin It seems you are talking about the followup PR: https://github.com/apache/spark/pull/21072 I think this is the way we do back propagation in catalyst: match a specific node,

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 Solid -1 if it breaks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21160: [SPARK-24094][SS][MINOR] Change description strings of v...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21160 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21160: [SPARK-24094][SS][MINOR] Change description strings of v...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21160 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2684/

[GitHub] spark issue #21160: [SPARK-24094][SS][MINOR] Change description strings of v...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21160 **[Test build #89866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89866/testReport)** for PR 21160 at commit

[GitHub] spark pull request #21160: [SPARK-24094][MINOR][SS] Change description strin...

2018-04-25 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/21160 [SPARK-24094][MINOR][SS] Change description strings of v2 streaming sources to reflect the change ## What changes were proposed in this pull request? This makes it easy to understand at

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89865/ Test FAILed. ---

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #89865 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89865/testReport)** for PR 21157 at commit

[GitHub] spark pull request #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PyS...

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21141#discussion_r184250774 --- Diff: python/pyspark/sql/tests.py --- @@ -3021,6 +3021,17 @@ def test_sort_with_nulls_order(self): class

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89862/ Test PASSed. ---

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21050 **[Test build #89862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89862/testReport)** for PR 21050 at commit

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #89865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89865/testReport)** for PR 21157 at commit

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 Why don't we try to fix it rather than removing out? Does the test even pass? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-04-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21070 I just run `TPCDSQueryBenchmark` in an ec2 instance (m4.2xlarge). The job to check performance regression is too heavy, so we just check if `tpcds`, `tpch`, and `ssb` can be correctly compiled in

[GitHub] spark issue #21144: [SPARK-24043][SQL] Interpreted Predicate should initiali...

2018-04-25 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21144 @hvanhovell I couldn't find another path that had this particular issue, except possibly in the aggregation area (where a grouping projection is created but not initialized). But I couldn't

[GitHub] spark pull request #21056: [SPARK-23849][SQL] Tests for samplingRatio of jso...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21056 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21142: [SPARK-24069][R] Add array_min / array_max functi...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21142 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21143 This is a very good point! Unfortunately Spark SQL doesn't support change the physical plan at a per-split fashion, and I'd say this feature is non-trivial to implement, and needs a design doc.

[GitHub] spark issue #21142: [SPARK-24069][R] Add array_min / array_max functions

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21142 Merged to master. Thanks for reviewing this @felixcheung. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21056: [SPARK-23849][SQL] Tests for samplingRatio of json datas...

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21056 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PyS...

2018-04-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21141#discussion_r184247014 --- Diff: python/pyspark/sql/tests.py --- @@ -3021,6 +3021,17 @@ def test_sort_with_nulls_order(self): class

[GitHub] spark issue #21155: SPARK-23927: Add "sequence" expression

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21155 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21056: [SPARK-23849][SQL] Tests for samplingRatio of json datas...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21056 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21056: [SPARK-23849][SQL] Tests for samplingRatio of json datas...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89861/ Test PASSed. ---

[GitHub] spark issue #21155: SPARK-23927: Add "sequence" expression

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21155 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89860/ Test PASSed. ---

[GitHub] spark issue #21056: [SPARK-23849][SQL] Tests for samplingRatio of json datas...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21056 **[Test build #89861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89861/testReport)** for PR 21056 at commit

[GitHub] spark issue #21155: SPARK-23927: Add "sequence" expression

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21155 **[Test build #89860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89860/testReport)** for PR 21155 at commit

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21138 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21138 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2683/

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21138 **[Test build #89864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89864/testReport)** for PR 21138 at commit

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21138 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21138: [SPARK-24062][Thrift Server] Fix SASL encryption cannot ...

2018-04-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21138 Thanks for the review @mridulm @vanzin . Let me test again. I will merge the code when test is passed. --- - To unsubscribe,

[GitHub] spark issue #21143: [SPARK-24072][SQL] clearly define pushed filters

2018-04-25 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21143 Thanks for working on this, @cloud-fan! I was thinking about needing it just recently so that data sources can delegate to Spark when needed. I'll have a thorough look at it tomorrow, but

[GitHub] spark pull request #21063: [SPARK-23886][Structured Streaming] Update query ...

2018-04-25 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21063#discussion_r184241625 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala --- @@ -111,7 +112,12 @@ trait ProgressReporter

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-04-25 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/21070 yes, thanks @maropu! +1 to the idea of making this a jenkins job. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-04-25 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21070 Thank you @maropu! What resources does the run require? Is it something we could create a Jenkins job to run? --- - To

[GitHub] spark pull request #21141: [SPARK-23853][PYSPARK][TEST] Run Hive-related PyS...

2018-04-25 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21141#discussion_r184241270 --- Diff: python/pyspark/sql/tests.py --- @@ -3021,6 +3021,17 @@ def test_sort_with_nulls_order(self): class

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-04-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21070 @gatorsmile @rdblue I checked the numbers and I found no regression at least in TPC-DS: to check the actual numbers, see https://issues.apache.org/jira/browse/SPARK-24070. ---

[GitHub] spark pull request #21152: [SPARK-23688][SS] Refactor tests away from rate s...

2018-04-25 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21152#discussion_r184236837 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -66,157 +66,115 @@ class ContinuousSuite

[GitHub] spark pull request #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to R...

2018-04-25 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21145#discussion_r184235329 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala --- @@ -299,13 +299,13 @@ private[kafka010]

[GitHub] spark pull request #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to R...

2018-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21145#discussion_r184232476 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala --- @@ -299,13 +299,13 @@

[GitHub] spark issue #21088: [SPARK-24003][CORE] Add support to provide spark.executo...

2018-04-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21088 It would be nice to add this to YARN's AM too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21118 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2682/

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21118 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-04-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21118 **[Test build #89863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89863/testReport)** for PR 21118 at commit

[GitHub] spark pull request #21122: [SPARK-24017] [SQL] Refactor ExternalCatalog to b...

2018-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21122#discussion_r184228752 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -31,10 +30,16 @@ import

  1   2   3   4   5   6   >