[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18874 Going to 0 executors is not a bug, if you set the min to 0. A deadlock is a bug. But, nothing in the JIRA or here suggests there's a deadlock -- what do you mean? --- If your project is set up

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r132246475 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java --- @@ -65,15 +65,35 @@ final Row row;

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-08-09 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/16774 Thanks for taking a look @WeichenXu123! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-09 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18874 I'm saying you have a stage running that has > 0 tasks to run. If dynamic allocation has already got all the executors it originally thought it needed and they all idle timeout then you have 0

[GitHub] spark issue #18893: [SPARK-21675][WebUI]Add a navigation bar at the bottom o...

2017-08-09 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/18893 Though I think these should have been separate PRs both these changes LGTM, I'm not sure why the original choice was to only display the pagination nabber at the top and though I'm not usually a

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132262219 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132262064 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132262101 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132262287 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132262559 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/DriverWrapper.scala --- @@ -66,4 +70,24 @@ object DriverWrapper { System.exit(-1)

[GitHub] spark issue #17455: [Spark-20044][Web UI] Support Spark UI behind front-end ...

2017-08-09 Thread sayeaud
Github user sayeaud commented on the issue: https://github.com/apache/spark/pull/17455 Would love to get this in for the next release, currently I'm using a variant of https://github.com/aseigneurin/spark-ui-proxy to work around this issue... --- If your project is set up for it,

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18648 **[Test build #80458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80458/testReport)** for PR 18648 at commit

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80458/ Test PASSed. ---

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18648 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80460/ Test PASSed. ---

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18648 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18883: [SPARK-21276][CORE] Update lz4-java to the latest...

2017-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18883 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18752: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-08-09 Thread peay
Github user peay commented on the issue: https://github.com/apache/spark/pull/18752 Great, thanks @JoshRosen! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r132252558 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java --- @@ -65,15 +65,35 @@ final Row row;

[GitHub] spark issue #18849: [SPARK-21617][SQL] Store correct table metadata when alt...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18849 **[Test build #80463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80463/testReport)** for PR 18849 at commit

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-09 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18874 I suggest you go understand the code. I've already explained this multiple times. You get 0 executors by there being delays when an executors doesn't have a task scheduled. say you

[GitHub] spark issue #17455: [Spark-20044][Web UI] Support Spark UI behind front-end ...

2017-08-09 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17455 #18499 was just merged, you'll need to update this to match the new code if you hope to get reinvigorate interest --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 @rxin . Could you make some decision for this PR? Do we need to put this into `sql/hive` still for some reasons? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #80465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80465/testReport)** for PR 1 at commit

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18640 **[Test build #80466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80466/testReport)** for PR 18640 at commit

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18648 **[Test build #80460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80460/testReport)** for PR 18648 at commit

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18896 **[Test build #80462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80462/testReport)** for PR 18896 at commit

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18896 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80462/ Test PASSed. ---

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18820 **[Test build #80459 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80459/testReport)** for PR 18820 at commit

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18820 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18820 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80459/ Test FAILed. ---

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18820 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18820 **[Test build #80464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80464/testReport)** for PR 18820 at commit

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132274194 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -300,28 +298,14 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r132360643 --- Diff: python/pyspark/ml/classification.py --- @@ -1325,7 +1325,7 @@ def __init__(self, featuresCol="features", labelCol="label", predictionCol="pred

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-09 Thread debugger87
Github user debugger87 commented on the issue: https://github.com/apache/spark/pull/18900 @cloud-fan could you please help me to review this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132360895 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,19 @@ class

[GitHub] spark pull request #18900: [SPARK-21687][SQL] Spark SQL should set createTim...

2017-08-09 Thread debugger87
GitHub user debugger87 opened a pull request: https://github.com/apache/spark/pull/18900 [SPARK-21687][SQL] Spark SQL should set createTime for Hive partition ## What changes were proposed in this pull request? Set createTime for every hive partition created in Spark SQL,

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132363994 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,19 @@ class

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132367041 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132368484 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,14 @@ case class

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80470/testReport)** for PR 18810 at commit

[GitHub] spark issue #17972: [SPARK-20723][ML]Add intermediate storage level to tree ...

2017-08-09 Thread phatak-dev
Github user phatak-dev commented on the issue: https://github.com/apache/spark/pull/17972 @MLnick Any updates on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132359678 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,15 @@ case class

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r132361043 --- Diff: python/pyspark/ml/tests.py --- @@ -1572,7 +1588,8 @@ def test_java_params(self): for name, cls in inspect.getmembers(module,

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80472/testReport)** for PR 18810 at commit

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132367400 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80470/ Test PASSed. ---

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r132359369 --- Diff: python/pyspark/ml/wrapper.py --- @@ -144,7 +158,9 @@ def _transfer_params_from_java(self): if self._java_obj.hasParam(param.name):

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r132363283 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r132364612 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132366187 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,14 @@ case class

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18810 Btw, can you change `[sql]` to `[SQL]` in title? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r132358656 --- Diff: python/pyspark/ml/wrapper.py --- @@ -263,7 +284,8 @@ def _fit_java(self, dataset): def _fit(self, dataset): java_model

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80472/ Test FAILed. ---

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r132361425 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132365401 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,19 @@ class CodegenContext

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132365436 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,19 @@ class CodegenContext

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132365359 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,13 @@ object SQLConf { "disable logging or

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80475/testReport)** for PR 18810 at commit

[GitHub] spark pull request #17995: [SPARK-20762][ML]Make String Params Case-Insensit...

2017-08-09 Thread zhengruifeng
Github user zhengruifeng closed the pull request at: https://github.com/apache/spark/pull/17995 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132366896 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala --- @@ -89,6 +89,14 @@ object CodeFormatter {

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132368646 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or

[GitHub] spark issue #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Param Val...

2017-08-09 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17849 Sorry, let me try and take a look tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17849: [SPARK-10931][ML][PYSPARK] PySpark Models Copy Pa...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17849#discussion_r132360069 --- Diff: python/pyspark/ml/tests.py --- @@ -417,6 +417,54 @@ def test_logistic_regression_check_thresholds(self): LogisticRegression,

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80471/testReport)** for PR 18810 at commit

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80471/ Test FAILed. ---

[GitHub] spark issue #18810: [SPARK-21603][sql]The wholestage codegen will be much sl...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132360710 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,13 @@ object SQLConf { "disable logging

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r132363687 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18869 @viirya Could you hold this PR at first? There is a related PR https://github.com/apache/spark/pull/18818 that is changing the underlying semantics. We also need to update the description after

[GitHub] spark pull request #18421: [SPARK-21213][SQL] Support collecting partition-l...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18421#discussion_r132099286 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -1033,25 +998,126 @@ private[spark] class

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18499 @aosagie do we still need the previous fix for SPARK-21176? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-09 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18797 Merged into master and branch-2.2. Thanks for all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an ...

2017-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18797 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r132104291 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala --- @@ -261,5 +261,37 @@ class DataFrameNaFunctionsSuite extends

[GitHub] spark issue #17373: [SPARK-12664] Expose probability in mlp model

2017-08-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17373 I think this will change the output of `summary` on `spark.mlp` in R right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18880 **[Test build #80444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80444/testReport)** for PR 18880 at commit

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18648 **[Test build #80442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80442/testReport)** for PR 18648 at commit

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18709 **[Test build #80445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80445/testReport)** for PR 18709 at commit

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18820 LGTM except a few minor comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132105955 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -356,6 +356,18 @@ class

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-09 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/18756 You can see this picture,my table has three columns,and I insert only two columns, then the last column is null. @maropu @gatorsmile

[GitHub] spark pull request #18810: [SPARK-21603][sql]The wholestage codegen will be ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132107421 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -301,6 +301,61 @@ class AggregateBenchmark

[GitHub] spark pull request #18849: [SPARK-21617][SQL] Store correct table metadata w...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18849#discussion_r132107646 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/Hive_2_1_DDLSuite.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark issue #18879: [SPARK-21662] modify the appname to [SparkSQL::localHost...

2017-08-09 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18879 I'm not convinced this deserves a separated PR because it's neither bugfix nor any improvement. Further more, the method `Utils.localHostName()` is used in many other places, we can't change

[GitHub] spark issue #18891: updateMerge pull request #1 from apache/master

2017-08-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18891 Close this @xiaosannimei --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-09 Thread IngoSchuster
Github user IngoSchuster commented on the issue: https://github.com/apache/spark/pull/18499 Yes I believe it still makes sense to limit the number of selector threads. Jetty's default is tuned for a real web server and by default it creates (number of cpus / 2) selector threads.

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-09 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/18648 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #80435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80435/testReport)** for PR 18869 at commit

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80435/ Test PASSed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 @gatorsmile Sure. Let's wait after #18818. I'll look at it too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18818 Also add `NULL` in the test case? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet t...

2017-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18499 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

  1   2   3   4   >