[GitHub] spark pull request #21805: [SPARK-24850][SQL] fix str representation of Cach...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21805#discussion_r204378903 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala --- @@ -206,4 +206,20 @@ class DatasetCacheSuite extends QueryTest with Share

[GitHub] spark pull request #21805: [SPARK-24850][SQL] fix str representation of Cach...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21805#discussion_r204379093 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala --- @@ -207,4 +207,7 @@ case class InMemoryRelation(

[GitHub] spark issue #21846: [SPARK-24887][SQL]Avro: use SerializableConfiguration in...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21846 **[Test build #93441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93441/testReport)** for PR 21846 at commit [`a8dd96d`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21846: [SPARK-24887][SQL]Avro: use SerializableConfiguration in...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21846 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21846: [SPARK-24887][SQL]Avro: use SerializableConfiguration in...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21846 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93441/ Test PASSed. ---

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21789 **[Test build #93442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93442/testReport)** for PR 21789 at commit [`05cbdc8`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21789 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93442/ Test FAILed. ---

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21789 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21789 **[Test build #93440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93440/testReport)** for PR 21789 at commit [`05cbdc8`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21789 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21789 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93440/ Test FAILed. ---

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21805 **[Test build #93444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93444/testReport)** for PR 21805 at commit [`2a21c80`](https://github.com/apache/spark/commit/2a

[GitHub] spark pull request #16374: [SPARK-18925][STREAMING] Reduce memory usage of m...

2018-07-23 Thread vpchelko
Github user vpchelko closed the pull request at: https://github.com/apache/spark/pull/16374 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-23 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204390597 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-23 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204391134 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1411,6 +1420,76 @@ class DAGScheduler( }

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-23 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204394561 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1411,6 +1420,76 @@ class DAGScheduler( } }

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1235/

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93445/testReport)** for PR 21758 at commit [`9ae56d1`](https://github.com/apache/spark/commit/9a

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1236/

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93446/testReport)** for PR 21758 at commit [`c16a47f`](https://github.com/apache/spark/commit/c1

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21653 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21653 **[Test build #93447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93447/testReport)** for PR 21653 at commit [`b6585da`](https://github.com/apache/spark/commit/b6

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21811 **[Test build #4221 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4221/testReport)** for PR 21811 at commit [`8b46534`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21635: [SPARK-24594][YARN] Introducing metrics for YARN

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21635 **[Test build #93439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93439/testReport)** for PR 21635 at commit [`0b86788`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21635: [SPARK-24594][YARN] Introducing metrics for YARN

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21635 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21635: [SPARK-24594][YARN] Introducing metrics for YARN

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21635 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93439/ Test PASSed. ---

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-07-23 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r204429892 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -94,36 +95,59 @@ object ExtractPythonUDFFromAggre

[GitHub] spark issue #21828: Update regression.py

2018-07-23 Thread woodthom2
Github user woodthom2 commented on the issue: https://github.com/apache/spark/pull/21828 OK thank you. I will close --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21828: Update regression.py

2018-07-23 Thread woodthom2
Github user woodthom2 closed the pull request at: https://github.com/apache/spark/pull/21828 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-23 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21811 > I like it, but this will still create the byte channel right? is there a way to reuse it? we could create a pool, though management becomes a bit more complex. would you ever shrink the p

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 I want to hear opinion of others about the order of a result. cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21805 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-23 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21811 > Does it make sense to release byteChannel at deallocate()? you could, just to let GC kick in a *bit* earlier, but I don't think its going to make a big difference. (Netty's ByteBufs must b

[GitHub] spark issue #15071: [SPARK-17517][SQL]Improve generated Code for BroadcastHa...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15071 @yaooqinn Can you close this because it's not long time for a long time. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark issue #18954: [SPARK-17654] [SQL] Enable populating hive bucketed tabl...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18954 @tejasapatil Can you close this for now because it's not active for a long time. --- - To unsubscribe, e-mail: reviews-unsubscr..

[GitHub] spark issue #15334: [SPARK-10367][SQL] Support Parquet logical type INTERVAL

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15334 IIUC we have no plan to expose Interval types now, so can we close this for now? cc: @gatorsmile --- - To unsubscribe, e-mail: r

[GitHub] spark issue #15334: [SPARK-10367][SQL] Support Parquet logical type INTERVAL

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15334 oh, I noticed the jira ticket has already been closed as later, so can you close this? @dilipbiswal --- - To unsubscribe, e-mail

[GitHub] spark issue #18697: [SPARK-16683][SQL] Repeated joins to same table can leak...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18697 @aray Can you close this for now because it's not active for a long time? (I'm not sure the current master still has this issue..., so you should check it first) --- --

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21821 @hvanhovell The question is whether `HandleNullInputsForUDF ` is the only rule that caused the issue. If not, we still need to add an AnalysisBarrier. --- -

[GitHub] spark issue #15970: [SPARK-18134][SQL] Comparable MapTypes [POC]

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15970 @hvanhovell We still need to keep this pr open? Either way, we need rework based on this pr. If so, can you close this for now? --- -

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21764 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19745: [SPARK-2926][Core][Follow Up] Sort shuffle reader for Sp...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19745 @xuanyuanking Can you close this for now because it's not active for a long time. --- - To unsubscribe, e-mail: reviews-unsubscr.

[GitHub] spark pull request #21764: [SPARK-24802][SQL] Add a new config for Optimizat...

2018-07-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21764 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20699: [SPARK-23544][SQL]Remove redundancy ShuffleExchange in t...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20699 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19773 @xuanyuanking Any update? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21839 **[Test build #93448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93448/testReport)** for PR 21839 at commit [`2cf131f`](https://github.com/apache/spark/commit/2c

[GitHub] spark pull request #21839: [SPARK-24339][SQL] Prunes the unused columns from...

2018-07-23 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21839#discussion_r204447671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -450,13 +450,16 @@ object ColumnPruning extends R

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-07-23 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r204447950 --- Diff: python/pyspark/sql/tests.py --- @@ -5060,6 +5049,144 @@ def test_type_annotation(self): df = self.spark.range(1).select(pandas_udf(

[GitHub] spark issue #20699: [SPARK-23544][SQL]Remove redundancy ShuffleExchange in t...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20699 **[Test build #93449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93449/testReport)** for PR 20699 at commit [`be96e39`](https://github.com/apache/spark/commit/be

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-23 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21839 @gatorsmile Thanks for your advice, added ut in ScriptTransformationSuite. --- - To unsubscribe, e-mail: reviews-unsubscr...

[GitHub] spark pull request #21846: [SPARK-24887][SQL]Avro: use SerializableConfigura...

2018-07-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21846 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21839 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1237/

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21839 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20854: [SPARK-23712][SQL] Interpreted UnsafeRowJoiner [WIP]

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20854 @hvanhovell What's the status of this pr? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comman

[GitHub] spark issue #19745: [SPARK-2926][Core][Follow Up] Sort shuffle reader for Sp...

2018-07-23 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/19745 No problem. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark pull request #19745: [SPARK-2926][Core][Follow Up] Sort shuffle reader...

2018-07-23 Thread xuanyuanking
Github user xuanyuanking closed the pull request at: https://github.com/apache/spark/pull/19745 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21805 **[Test build #93443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93443/testReport)** for PR 21805 at commit [`de3f63e`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2018-07-23 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/19773 I'll resolve the conflicts today, thanks for ping me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21805 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21805 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93443/ Test PASSed. ---

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21839 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark issue #19434: [SPARK-21785][SQL]Support create table from a parquet fi...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19434 @CrazyJacky Can you close this for now cuz it's not active for a long time? --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #17174: [SPARK-19145][SQL] Timestamp to String casting is slowin...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17174 @tanejagagan Can you update? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21650 **[Test build #93450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93450/testReport)** for PR 21650 at commit [`78f2ebf`](https://github.com/apache/spark/commit/78

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1238/

[GitHub] spark pull request #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO ...

2018-07-23 Thread lindblombr
GitHub user lindblombr opened a pull request: https://github.com/apache/spark/pull/21847 [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support should support specified schema on write ## What changes were proposed in this pull request? Allows `avroSchema` option to be sp

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21805 **[Test build #93444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93444/testReport)** for PR 21805 at commit [`2a21c80`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21805 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21805 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93444/ Test PASSed. ---

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21845 This helps, but it is not sustainable to keep increasing the threshold. What we need to do is to look at test time distribution and figure out what test suites are unnecessarily long and actually cut d

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-23 Thread maryannxue
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/21821 I just ran a test with once-strategy check and found out that a few batches/rules do not stop, e.g. AggregatePushDown, "Convert to Spark client exec", PartitionPruning. I believe most of them are

[GitHub] spark issue #21826: [SPARK-24872] Remove the symbol “||” of the “OR”...

2018-07-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21826 No we can't because you can still use string concat in filters, e.g. colA || colB == "ab" What is "||" here? --- -

[GitHub] spark pull request #21840: [WIP] New copy() method for Column of StructType

2018-07-23 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204476440 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3858,3 +3858,29 @@ object ArrayUnion

[GitHub] spark issue #21805: [SPARK-24850][SQL] fix str representation of CachedRDDBu...

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21805 LGTM Thanks! Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream forma...

2018-07-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r204479024 --- Diff: python/pyspark/sql/dataframe.py --- @@ -2095,9 +2095,11 @@ def toPandas(self): _check_dataframe_localize_timestamp

[GitHub] spark pull request #21805: [SPARK-24850][SQL] fix str representation of Cach...

2018-07-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21805 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21845 of course i am as usual. I actually already have been being taking care of it. Thing is the tests are just being added even if they are duplicated of something. I feel like it's a bit excessive

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-07-23 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21118 @cloud-fan, any update on merging this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93445/testReport)** for PR 21758 at commit [`9ae56d1`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21650 **[Test build #93451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93451/testReport)** for PR 21650 at commit [`4c9c007`](https://github.com/apache/spark/commit/4c

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93445/ Test FAILed. ---

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93446/testReport)** for PR 21758 at commit [`c16a47f`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-07-23 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r204482591 --- Diff: python/pyspark/sql/tests.py --- @@ -5060,6 +5049,144 @@ def test_type_annotation(self): df = self.spark.range(1).select(pandas_udf(

[GitHub] spark pull request #21823: [SPARK-24870][SQL]Cache can't work normally if th...

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21823#discussion_r204482993 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SameResultSuite.scala --- @@ -58,4 +61,16 @@ class SameResultSuite extends QueryTest wit

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93446/ Test FAILed. ---

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1239/

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21758 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream forma...

2018-07-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r204484728 --- Diff: python/pyspark/serializers.py --- @@ -184,27 +184,67 @@ def loads(self, obj): raise NotImplementedError -class Arr

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1240/

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93452/testReport)** for PR 21758 at commit [`c16a47f`](https://github.com/apache/spark/commit/c1

<    1   2   3   4   5   >