[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132103945 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -261,5 +261,37 @@ class DataFrameNaFunctionsSuite extends Q

[GitHub] spark issue #17373: [SPARK-12664] Expose probability in mlp model

2017-08-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17373 I think this will change the output of `summary` on `spark.mlp` in R right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18880 **[Test build #80444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80444/testReport)** for PR 18880 at commit [`7bd2904`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18648 **[Test build #80442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80442/testReport)** for PR 18648 at commit [`6c0bf70`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18709 **[Test build #80445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80445/testReport)** for PR 18709 at commit [`f5a26cc`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132104291 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -261,5 +261,37 @@ class DataFrameNaFunctionsSuite extends Q

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132104328 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -261,5 +261,37 @@ class DataFrameNaFunctionsSuite extends Q

[GitHub] spark issue #15435: [SPARK-17139][ML] Add model summary for MultinomialLogis...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15435 **[Test build #80441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80441/testReport)** for PR 15435 at commit [`403680c`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18846 **[Test build #80443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80443/testReport)** for PR 18846 at commit [`afc07ee`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18846 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18880 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15435: [SPARK-17139][ML] Add model summary for MultinomialLogis...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15435 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15435: [SPARK-17139][ML] Add model summary for MultinomialLogis...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15435 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80441/ Test FAILed. ---

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18846 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80443/ Test FAILed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #80439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80439/testReport)** for PR 18869 at commit [`bca2b0b`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18709 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80445/ Test FAILed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80440/ Test FAILed. ---

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18709 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18648 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80442/ Test FAILed. ---

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18648 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #80440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80440/testReport)** for PR 18869 at commit [`b64c9e6`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18880 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80444/ Test FAILed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80439/ Test FAILed. ---

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18820 LGTM except a few minor comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18846 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #18867: [SPARK-21663][TESTS]test("remote fetch below max RPC mes...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18867 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132105592 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,13 @@ object SQLConf { "disable logging o

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18846 **[Test build #80446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80446/testReport)** for PR 18846 at commit [`afc07ee`](https://github.com/apache/spark/commit/af

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-09 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/18756 You can see this picture,my table has three columns,and I insert only two columns, then the last column is null. @maropu @gatorsmile ![insertinto](https://user-images.githubusercontent.com/256

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132105955 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,18 @@ class CodegenContex

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132106108 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,18 @@ class CodegenContex

[GitHub] spark issue #18890: [SPARK-21596][SS] Ensure places calling HDFSMetadataLog....

2017-08-09 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/18890 Thanks @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18890: [SPARK-21596][SS] Ensure places calling HDFSMetad...

2017-08-09 Thread tdas
Github user tdas closed the pull request at: https://github.com/apache/spark/pull/18890 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark issue #18891: updateMerge pull request #1 from apache/master

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18891 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18891: updateMerge pull request #1 from apache/master

2017-08-09 Thread xiaosannimei
GitHub user xiaosannimei opened a pull request: https://github.com/apache/spark/pull/18891 updateMerge pull request #1 from apache/master new update ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was

[GitHub] spark issue #18891: updateMerge pull request #1 from apache/master

2017-08-09 Thread xiaosannimei
Github user xiaosannimei commented on the issue: https://github.com/apache/spark/pull/18891 nothing to do --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18709 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132107421 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -301,6 +301,61 @@ class AggregateBenchmark ex

[GitHub] spark pull request #18849: [SPARK-21617][SQL] Store correct table metadata w...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18849#discussion_r132107646 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/Hive_2_1_DDLSuite.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the Apac

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18709 **[Test build #80447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80447/testReport)** for PR 18709 at commit [`f5a26cc`](https://github.com/apache/spark/commit/f5

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18879 I'm not convinced this deserves a separated PR because it's neither bugfix nor any improvement. Further more, the method `Utils.localHostName()` is used in many other places, we can't change thi

[GitHub] spark issue #18474: [SPARK-21235][TESTS] UTest should clear temp results whe...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18474 Yea, I agree we should find the root cause why these temp directories are not deleted, this should be a minor issue though. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18879 The method name is misleading internally, but we can't change this logic. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark issue #18891: updateMerge pull request #1 from apache/master

2017-08-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18891 Close this @xiaosannimei --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/18879 @srowen @jiangxb1987 If we can't change this logic,I think we should change the annotation,because it says "If user doesn't specify the appName, we want to get [SparkSQL::localHostName]" ,

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-09 Thread IngoSchuster
Github user IngoSchuster commented on the issue: https://github.com/apache/spark/pull/18499 Yes I believe it still makes sense to limit the number of selector threads. Jetty's default is tuned for a real web server and by default it creates (number of cpus / 2) selector threads. Right

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132115069 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1446,21 +1457,21 @@ def all_of_(xs): if isinstance(to_replace, (float, int, long, basestring)):

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132115201 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1446,21 +1457,21 @@ def all_of_(xs): if isinstance(to_replace, (float, int, long, basestring)):

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/18648 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18648 **[Test build #80448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80448/testReport)** for PR 18648 at commit [`51fac11`](https://github.com/apache/spark/commit/51

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18818#discussion_r132119567 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -465,7 +475,7 @@ abstract class BinaryComparison ext

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/18879 @srowen I have modified the annotation,would you like to review it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request #18879: [SPARK-21662] modify the appname to [SparkSQL::lo...

2017-08-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/18879#discussion_r132123263 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLEnv.scala --- @@ -35,7 +35,7 @@ private[hive] object SparkSQLE

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18879 I think @srowen means you should change the method name, not only the comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #18855: [SPARK-3151] [Block Manager] DiskStore.getBytes fails fo...

2017-08-09 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/18855 Agreed, though I can't really understand it... Ran SBT locally with --mem 12000 and provided the magical jvm flag that prints jvm provided CLI args, it seems SBT is indeed running with 12gb and

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18818#discussion_r132124067 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -465,7 +475,7 @@ abstract class BinaryComparison ext

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132125663 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1446,21 +1457,21 @@ def all_of_(xs): if isinstance(to_replace, (float, int, long, basestri

[GitHub] spark pull request #18879: [SPARK-21662] modify the appname to [SparkSQL::lo...

2017-08-09 Thread liu-zhaokun
Github user liu-zhaokun commented on a diff in the pull request: https://github.com/apache/spark/pull/18879#discussion_r132126001 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLEnv.scala --- @@ -35,7 +35,7 @@ private[hive] object Spar

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132126123 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1446,21 +1457,21 @@ def all_of_(xs): if isinstance(to_replace, (float, int, long, basestring)):

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/18879 Thanks for your patience.Would you @srowen agree with @jiangxb1987 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18820 Please add the suggested tests then LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18820 Yea, looks much safer. LGTM too except the comments above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread vinodkc
Github user vinodkc commented on the issue: https://github.com/apache/spark/pull/18880 retest this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if th

[GitHub] spark pull request #18725: [SPARK-21520][SQL]Hivetable scan for all the colu...

2017-08-09 Thread heary-cao
Github user heary-cao closed the pull request at: https://github.com/apache/spark/pull/18725 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18892: Improvement a special case for non-deterministic ...

2017-08-09 Thread heary-cao
GitHub user heary-cao opened a pull request: https://github.com/apache/spark/pull/18892 Improvement a special case for non-deterministic projects and filters in optimizer ## What changes were proposed in this pull request? Currently, Did a lot of special handling for non-de

[GitHub] spark issue #18892: Improvement a special case for non-deterministic project...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18892 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18893: [SPARK-21675][WebUI]Add a navigation bar at the b...

2017-08-09 Thread yaooqinn
GitHub user yaooqinn opened a pull request: https://github.com/apache/spark/pull/18893 [SPARK-21675][WebUI]Add a navigation bar at the bottom of the Details for Stage Page ## What changes were proposed in this pull request? 1. In Spark Web UI, the Details for Stage Page don

[GitHub] spark issue #18893: [SPARK-21675][WebUI]Add a navigation bar at the bottom o...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18893 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18736: [SPARK-21481][ML] Add indexOf method for ml.featu...

2017-08-09 Thread facaiy
Github user facaiy commented on a diff in the pull request: https://github.com/apache/spark/pull/18736#discussion_r132131171 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala --- @@ -90,10 +92,22 @@ class HashingTF @Since("1.4.0") (@Since("1.4.0") override

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/18879 @jiangxb1987 BTW,how to find the release branch of Hive-0.13.1a which spark1.4.1 compiled with? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18709 **[Test build #80447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80447/testReport)** for PR 18709 at commit [`f5a26cc`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18709 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80447/ Test FAILed. ---

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18846 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80446/ Test FAILed. ---

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18846 **[Test build #80446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80446/testReport)** for PR 18846 at commit [`afc07ee`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18846 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18709 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18880 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18880 **[Test build #80449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80449/testReport)** for PR 18880 at commit [`7bd2904`](https://github.com/apache/spark/commit/7b

[GitHub] spark issue #18843: [SPARK-21595] Separate thresholds for buffering and spil...

2017-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18843 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18892: [SPARK-21520][SQL]Improvement a special case for non-det...

2017-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18892 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18843: [SPARK-21595] Separate thresholds for buffering and spil...

2017-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18843 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #18843: [SPARK-21595] Separate thresholds for buffering and spil...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18843 **[Test build #80451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80451/testReport)** for PR 18843 at commit [`398ccaf`](https://github.com/apache/spark/commit/39

[GitHub] spark issue #18892: [SPARK-21520][SQL]Improvement a special case for non-det...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18892 **[Test build #80450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80450/testReport)** for PR 18892 at commit [`a1c7559`](https://github.com/apache/spark/commit/a1

[GitHub] spark issue #18892: [SPARK-21520][SQL]Improvement a special case for non-det...

2017-08-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18892 @heary-cao can you improve the PR description, and explain what special cases you are improving. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132146376 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -31,16 +31,16 @@ import org.apache.spark.s

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132148739 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -844,24 +844,39 @@ object SQLConf { .stringConf

[GitHub] spark pull request #18884: [SPARK-21669] Internal API for collecting metrics...

2017-08-09 Thread adrian-ionescu
Github user adrian-ionescu commented on a diff in the pull request: https://github.com/apache/spark/pull/18884#discussion_r132130615 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSink.scala --- @@ -128,6 +128,7 @@ class FileStreamSink(

[GitHub] spark pull request #18884: [SPARK-21669] Internal API for collecting metrics...

2017-08-09 Thread adrian-ionescu
Github user adrian-ionescu commented on a diff in the pull request: https://github.com/apache/spark/pull/18884#discussion_r132142337 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/WriteStatsTracker.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed

[GitHub] spark pull request #18884: [SPARK-21669] Internal API for collecting metrics...

2017-08-09 Thread adrian-ionescu
Github user adrian-ionescu commented on a diff in the pull request: https://github.com/apache/spark/pull/18884#discussion_r132147389 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -269,50 +278,57 @@ object FileFormatWrite

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132148869 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -844,24 +844,39 @@ object SQLConf { .stringConf

[GitHub] spark issue #18867: [SPARK-21663][TESTS]test("remote fetch below max RPC mes...

2017-08-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18867 thanks, merging to master/2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18867: [SPARK-21663][TESTS]test("remote fetch below max ...

2017-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18867 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18843: [SPARK-21595] Separate thresholds for buffering a...

2017-08-09 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18843#discussion_r132156490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -844,24 +844,39 @@ object SQLConf { .stringConf

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r132157043 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -105,107 +105,69 @@ private[hive] class HiveClientImpl(

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r132150474 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -136,8 +136,15 @@ class SparkHadoopUtil extends Logging {

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r132155005 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -105,107 +105,69 @@ private[hive] class HiveClientImpl(

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r132156338 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -229,6 +230,17 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18648 **[Test build #80448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80448/testReport)** for PR 18648 at commit [`51fac11`](https://github.com/apache/spark/commit/5

  1   2   3   4   >