[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20850 This test is consistently failed. While I did not change file reader, I am investigating the reason in my environment. --- - To

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175407178 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -40,29 +37,45 @@ */

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88373/ Test FAILed. ---

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88372/ Test FAILed. ---

[GitHub] spark issue #20847: [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path ...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20847: [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path ...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88371/ Test FAILed. ---

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175393443 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -31,24 +31,24 @@ * for each

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20701 **[Test build #88374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88374/testReport)** for PR 20701 at commit

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20827 **[Test build #88376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88376/testReport)** for PR 20827 at commit

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88373/testReport)** for PR 20850 at commit

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20827 **[Test build #88368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88368/testReport)** for PR 20827 at commit

[GitHub] spark pull request #18355: [SPARK-21145][SS] Added StateStoreProviderId with...

2018-03-19 Thread andrusha
Github user andrusha commented on a diff in the pull request: https://github.com/apache/spark/pull/18355#discussion_r175379493 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala --- @@ -364,12 +398,21 @@ object StateStore extends

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20433 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20827 **[Test build #88370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88370/testReport)** for PR 20827 at commit

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1609/

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1610/

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175392909 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -86,11 +86,17 @@ public void grow(int

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175393169 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -47,126 +40,119 @@ private void

[GitHub] spark issue #20847: [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path ...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20847 **[Test build #88371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88371/testReport)** for PR 20847 at commit

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175407382 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriter.java --- @@ -36,8 +90,92 @@ public abstract

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1612/

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175363603 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -86,11 +86,17 @@ public void grow(int

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175372993 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriter.java --- @@ -17,17 +17,71 @@ package

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175372362 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -86,11 +86,17 @@ public void grow(int

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20850 @kiszk this is a good start! This is very performance critical code, can you please extend/update/run the existing `UnsafeProjectionBenchmark`? ---

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1611/

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88368/ Test FAILed. ---

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20433 **[Test build #88372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88372/testReport)** for PR 20433 at commit

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18982 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88369/ Test PASSed. ---

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175391993 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -111,27 +111,27 @@ object

[GitHub] spark issue #20818: [SPARK-23675][WEB-UI]Title add spark logo, use spark log...

2018-03-19 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20818 @ajbozarth @srowen Help to review the code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18982 **[Test build #88369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88369/testReport)** for PR 18982 at commit

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20719 **[Test build #88375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88375/testReport)** for PR 20719 at commit

[GitHub] spark issue #20846: [SPARK-5498][SQL][FOLLOW] add schema to table partition

2018-03-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20846 Right, @gatorsmile . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20849: [SPARK-23723] New charset option for json datasou...

2018-03-19 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20849#discussion_r175439887 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -85,6 +85,12 @@ private[sql] class JSONOptions(

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88378/testReport)** for PR 20850 at commit

[GitHub] spark issue #20818: [SPARK-23675][WEB-UI]Title add spark logo, use spark log...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20818 **[Test build #4140 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4140/testReport)** for PR 20818 at commit

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Implementation of the concat_a...

2018-03-19 Thread mn-mikke
GitHub user mn-mikke opened a pull request: https://github.com/apache/spark/pull/20858 [SPARK-23736][SQL] Implementation of the concat_arrays function concatenating multiple array columns into one. ## What changes were proposed in this pull request? The PR adds a logic for easy

[GitHub] spark issue #20858: [SPARK-23736][SQL] Implementation of the concat_arrays f...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20858 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175474010 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriter.java --- @@ -17,17 +17,71 @@ package

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-03-19 Thread brad-kaiser
Github user brad-kaiser commented on the issue: https://github.com/apache/spark/pull/19041 Hi @squito, The back and forth communication between CacheRecoveryManager and the BlockManagerMasterEndpoint is so that we always have an up to date view of what executors are

[GitHub] spark pull request #20849: [SPARK-23723] New charset option for json datasou...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20849#discussion_r175442491 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -85,6 +85,12 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175445800 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -111,27 +111,27 @@ object

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88375/ Test PASSed. ---

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20858: [SPARK-23736][SQL] Implementation of the concat_arrays f...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20858 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18666: [SPARK-21449][SQL][Hive]Close HiveClient's SessionState ...

2018-03-19 Thread samartinucci
Github user samartinucci commented on the issue: https://github.com/apache/spark/pull/18666 Do we have a workaround for this? Will this PR get merged? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1613/

[GitHub] spark pull request #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorr...

2018-03-19 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/20763 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20844: [SPARK-23707][SQL] Fresh 'initRange' name to avoi...

2018-03-19 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/20844#discussion_r175658889 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -396,9 +396,11 @@ case class RangeExec(range:

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20579 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r175651639 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -72,6 +82,14 @@ private[parquet]

[GitHub] spark pull request #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpar...

2018-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20830 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20830 Hmm, it looks like the conflict is just one block with group agg tests, probably not a big deal - you want to take a look? ---

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20830 Sure, will open a PR soon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20579 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88396/ Test PASSed. ---

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20830 The cherry pick to branch-2.3 did have some conflicts. Just to check for the reason to backport, even though this isn't a bug it's pretty safe and will help keep things inline so less

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20830 Thanks for reviewing and merging this @ueshin, @felixcheung, @BryanCutler and @dongjoon-hyun. (Just FYI, I usually manually resolve JIRAs when I accidentally failed to take an action

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20830 But I am willing to do this if you think it's better to do this. No objection. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20863: [SPARK-23691][PYTHON][BRANCH-2.3] Use sql_conf util in P...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20863 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20863: [SPARK-23691][PYTHON][BRANCH-2.3] Use sql_conf util in P...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20863 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1631/

[GitHub] spark issue #20863: [SPARK-23691][PYTHON][BRANCH-2.3] Use sql_conf util in P...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20863 **[Test build #88404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88404/testReport)** for PR 20863 at commit

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20579 **[Test build #88396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88396/testReport)** for PR 20579 at commit

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20830 Merged to master! (I think it went ok..) Thanks @HyukjinKwon !! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r17565 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -86,11 +88,39 @@ public void grow(int

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-19 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r175657704 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedUnsafeProjection.scala --- @@ -178,81 +171,76 @@ object

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20774 **[Test build #88403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88403/testReport)** for PR 20774 at commit

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r175649724 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -76,7 +77,9 @@ class

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20830 Yup, that was exactly what I thought. I think it's fine to not bother backport too since it has conflicts. --- - To

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20433 **[Test build #88400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88400/testReport)** for PR 20433 at commit

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20433: [SPARK-23264][SQL] Make INTERVAL keyword optional in INT...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20433 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88400/ Test FAILed. ---

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20774 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20774: [SPARK-23549][SQL] Cast to timestamp when comparing time...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1630/

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1632/

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20816 **[Test build #88405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88405/testReport)** for PR 20816 at commit

[GitHub] spark issue #19819: [SPARK-22606][Streaming]Add threadId to the CachedKafkaC...

2018-03-19 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/19819 It will create a new consumer for each thread. This could be quite resource consuming when several topics shared with thread pools. ---

[GitHub] spark issue #20852: [SPARK-23728][BRANCH-2.3] Fix ML tests with expected exc...

2018-03-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20852 (@attilapiros, just in case it should be manually closed) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20853 **[Test build #88385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88385/testReport)** for PR 20853 at commit

[GitHub] spark issue #20829: [SPARK-23690][ML] Add handleinvalid to VectorAssembler

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20829 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88392/ Test PASSed. ---

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88385/ Test PASSed. ---

[GitHub] spark issue #20829: [SPARK-23690][ML] Add handleinvalid to VectorAssembler

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20829 **[Test build #88392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88392/testReport)** for PR 20829 at commit

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r175629947 --- Diff: core/src/main/scala/org/apache/spark/ui/WebUI.scala --- @@ -126,7 +126,11 @@ private[spark] abstract class WebUI( def bind(): Unit = {

[GitHub] spark issue #20853: [SPARK-23729][CORE] Respect URI fragment when resolving ...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20829: [SPARK-23690][ML] Add handleinvalid to VectorAssembler

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20829 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20579 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20579 **[Test build #88396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88396/testReport)** for PR 20579 at commit

[GitHub] spark issue #19108: [SPARK-21898][ML] Feature parity for KolmogorovSmirnovTe...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19108 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1623/

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18666: [SPARK-21449][SQL][Hive]Close HiveClient's SessionState ...

2018-03-19 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/18666 @samartinucci thanks for reminding of this, i have fixed the conflicts. --- - To unsubscribe, e-mail:

  1   2   3   4   5   >