[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98011972 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -263,17 +288,21 @@ class

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16700 **[Test build #72025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72025/testReport)** for PR 16700 at commit

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98018738 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -578,6 +578,79 @@ class

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2017-01-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r98026636 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,128 @@ +/* + * Licensed to the

[GitHub] spark pull request #16138: [SPARK-16609] Add to_date/to_timestamp with forma...

2017-01-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r97976102 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1048,64 @@ case class

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r97980819 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogUtils.scala --- @@ -120,6 +120,24 @@ object

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98013552 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -263,17 +288,21 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98014650 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -736,15 +765,27 @@ class

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16700 **[Test build #72025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72025/testReport)** for PR 16700 at commit

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98015797 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -736,15 +765,27 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98016201 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -263,17 +288,21 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98019881 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -578,6 +578,79 @@ class

[GitHub] spark issue #16680: [SPARK-16101][SQL] Refactoring CSV schema inference path...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16680 **[Test build #72024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72024/testReport)** for PR 16680 at commit

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98019350 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -578,6 +578,79 @@ class

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98012237 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Instance.scala --- @@ -27,3 +27,25 @@ import org.apache.spark.ml.linalg.Vector *

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98012093 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Instance.scala --- @@ -27,3 +27,25 @@ import org.apache.spark.ml.linalg.Vector *

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16700 **[Test build #72026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72026/testReport)** for PR 16700 at commit

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98018075 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -753,14 +794,15 @@ class

[GitHub] spark issue #16654: [SPARK-19303][ML][WIP] Add evaluate method in clustering...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16654 @zhengruifeng don't most ML libraries have separate clustering evaluators? For example, WEKA has ClusterEvalution class. Scikit-learn just has a metrics class and functions you can call,

[GitHub] spark issue #16680: [SPARK-16101][SQL] Refactoring CSV schema inference path...

2017-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16680 Sure! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98010924 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -94,6 +94,17 @@ private[regression] trait

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98017514 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -753,14 +794,15 @@ class

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16699 This looks really good. I made a couple more comments. I'm not sure about the class hierarchy of Instance/OffsetInstance. Maybe committers can take a look/comment on this PR? @jkbradley

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r98020407 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -578,6 +578,79 @@ class

[GitHub] spark issue #16557: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16557 @srowen @yanboliang might you be able to take a look at this PR? Is it possibly too large and I should break it up into 3 PRs, one per evaluator/metrics class? --- If your project is set

[GitHub] spark issue #16693: [SPARK-19152][SQL][followup] simplify CreateHiveTableAsS...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16693 **[Test build #72022 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72022/testReport)** for PR 16693 at commit

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r97977055 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogUtils.scala --- @@ -120,6 +120,24 @@ object

[GitHub] spark issue #16680: [SPARK-16101][SQL] Refactoring CSV schema inference path...

2017-01-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16680 shall we rename `InferSchema` to `JsonInferSchema`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72023/testReport)** for PR 16620 at commit

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16536 **[Test build #72021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72021/testReport)** for PR 16536 at commit

[GitHub] spark issue #16693: [SPARK-19152][SQL][followup] simplify CreateHiveTableAsS...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16693 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16693: [SPARK-19152][SQL][followup] simplify CreateHiveTableAsS...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16693 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72022/ Test FAILed. ---

[GitHub] spark issue #16693: [SPARK-19152][SQL][followup] simplify CreateHiveTableAsS...

2017-01-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16693 @gatorsmile thanks for adding comments about why the cleanup is safe! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72021/ Test PASSed. ---

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16536 **[Test build #72021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72021/testReport)** for PR 16536 at commit

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16575: [SPARK-19213] DatasourceScanExec uses runtime sparksessi...

2017-01-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16575 I suppose it makes sense. The only thing I am wondering if this doesn't subtly breaks things. I am certain that some weirdness is to be expected with respect to session local configurations. For

[GitHub] spark issue #16308: [SPARK-18936][SQL] Infrastructure for session local time...

2017-01-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16308 Sure! I'll send follow-up prs as soon as possible. Thanks a lot for your review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16693: [SPARK-19152][SQL][followup] simplify CreateHiveTableAsS...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16693 **[Test build #72022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72022/testReport)** for PR 16693 at commit

[GitHub] spark issue #16557: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16557 ping @sethah @Lewuathe @thunterdb @WeichenXu123 @jkbradley would you be able to take a look at the changes to add a weight column to binary/multiclass/regression evaluators/metrics classes?

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-01-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r98002931 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -727,37 +728,18 @@ case class

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16620 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16620 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72023/ Test PASSed. ---

[GitHub] spark pull request #16710: [SPARK-18872][SQL][TESTS] New test cases for EXIS...

2017-01-26 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16710#discussion_r98048517 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/exists-subquery/exists-basic.sql.out --- @@ -0,0 +1,201 @@ +-- Automatically generated

[GitHub] spark issue #16695: [SPARK-19277][yarn] Localize topology scripts inside Had...

2017-01-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16695 Actually this can be considered kind of a continuation of SPARK-2669. The issue this is trying to solve is that the NMs don't necessarily have a client Hadoop configuration in place. (Or

[GitHub] spark issue #16494: [SPARK-17975][MLLIB] Fix EMLDAOptimizer failing with Cla...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16494 I'll go ahead and close this review. We can always reopen later, but it really does seem that it is fixed now. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98056351 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -839,6 +839,25 @@ private[spark] class

[GitHub] spark issue #16695: [SPARK-19277][yarn] Localize topology scripts inside Had...

2017-01-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16695 It's kind of a side-effect of how we deploy client configurations, I think. We've had some issues with the topology scripts not being available together with Spark client configs in certain

[GitHub] spark issue #16582: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16582 I wasn't really planning to backport this, unless someone is really interested in this functionality. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-01-26 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r98043010 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -718,6 +703,21 @@ private[spark] class TaskSetManager( "

[GitHub] spark issue #16680: [SPARK-16101][SQL] Refactoring CSV schema inference path...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16680 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16710: [SPARK-18872][SQL][TESTS] New test cases for EXISTS subq...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16710 **[Test build #72029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72029/testReport)** for PR 16710 at commit

[GitHub] spark pull request #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initializa...

2017-01-26 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/16536#discussion_r98048118 --- Diff: python/pyspark/sql/functions.py --- @@ -1826,19 +1826,27 @@ class UserDefinedFunction(object): def __init__(self, func, returnType,

[GitHub] spark pull request #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initializa...

2017-01-26 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/16536#discussion_r98049505 --- Diff: python/pyspark/sql/functions.py --- @@ -1826,19 +1826,27 @@ class UserDefinedFunction(object): def __init__(self, func, returnType,

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98056608 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -839,6 +839,25 @@ private[spark] class

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98057702 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -899,6 +918,21 @@ private[spark] class

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72028/testReport)** for PR 16620 at commit

[GitHub] spark pull request #16710: [SPARK-18872][SQL][TESTS] New test cases for EXIS...

2017-01-26 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16710#discussion_r98048551 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/exists-subquery/exists-within-and-or.sql.out --- @@ -0,0 +1,156 @@ +-- Automatically

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98055435 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -839,6 +839,25 @@ private[spark] class

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98055321 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -839,6 +839,25 @@ private[spark] class

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98057316 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -899,6 +918,21 @@ private[spark] class

[GitHub] spark issue #16557: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-01-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16557 I wouldn't ping that frequently, please. I don't feel qualified to review this myself. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16680: [SPARK-16101][SQL] Refactoring CSV schema inference path...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72024/ Test PASSed. ---

[GitHub] spark issue #16494: [SPARK-17975][MLLIB] Fix EMLDAOptimizer failing with Cla...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16494 **[Test build #72030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72030/testReport)** for PR 16494 at commit

[GitHub] spark issue #16707: [SPARK-19338][SQL] Add UDF names in explain

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16707 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16707: [SPARK-19338][SQL] Add UDF names in explain

2017-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16707 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16711: [SPARK-19220][UI] Make redirection to HTTPS apply...

2017-01-26 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/16711 [SPARK-19220][UI] Make redirection to HTTPS apply to all URIs. (branch-2.1) The redirect handler was installed only for the root of the server; any other context ended up being served directly

[GitHub] spark issue #16494: [SPARK-17975][MLLIB] Fix EMLDAOptimizer failing with Cla...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16494 @QQshu1 1.) It seems that we are creating the wrong type of object when getting edges, EdgeRDDImpl instead of edge partitions, when setting the checkpoint 2.) I was able to repro the issue

[GitHub] spark issue #16695: [SPARK-19277][yarn] Localize topology scripts inside Had...

2017-01-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16695 > then overlay the user's config on top of it Just tried this for kicks and doing it the naive way (with `addResource`) would result in a bunch of warnings: WARN

[GitHub] spark issue #16693: [SPARK-19152][SQL][followup] simplify CreateHiveTableAsS...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16693 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initializa...

2017-01-26 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/16536#discussion_r98055966 --- Diff: python/pyspark/sql/functions.py --- @@ -1826,19 +1826,27 @@ class UserDefinedFunction(object): def __init__(self, func, returnType,

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98055867 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -839,6 +839,25 @@ private[spark] class

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72026/ Test PASSed. ---

[GitHub] spark pull request #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initializa...

2017-01-26 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/16536#discussion_r98043331 --- Diff: python/pyspark/sql/functions.py --- @@ -1826,19 +1826,27 @@ class UserDefinedFunction(object): def __init__(self, func, returnType,

[GitHub] spark issue #16711: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16711 **[Test build #72031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72031/testReport)** for PR 16711 at commit

[GitHub] spark pull request #16494: [SPARK-17975][MLLIB] Fix EMLDAOptimizer failing w...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft closed the pull request at: https://github.com/apache/spark/pull/16494 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16043 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16043 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72033/ Test FAILed. ---

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16043 **[Test build #72033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72033/testReport)** for PR 16043 at commit

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16690 **[Test build #72027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72027/testReport)** for PR 16690 at commit

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72023/testReport)** for PR 16620 at commit

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16695: [SPARK-19277][yarn] Localize topology scripts inside Had...

2017-01-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/16695 this seems really really specific to the scripts being in the hadoop conf directory and the user using default mapping. I assume the hadoop confs on the nodemanagers have a different config then

[GitHub] spark issue #16710: [SPARK-18872][SQL][TESTS] New test cases for EXISTS subq...

2017-01-26 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/16710 cc @hvanhovell @gatorsmile @nsyca --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16497: [SPARK-19118] [SQL] Percentile support for frequency dis...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16497 **[Test build #72032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72032/testReport)** for PR 16497 at commit

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98056076 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -839,6 +839,25 @@ private[spark] class

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16536 **[Test build #72034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72034/testReport)** for PR 16536 at commit

[GitHub] spark pull request #16695: [SPARK-19277][yarn] Localize topology scripts ins...

2017-01-26 Thread vanzin
Github user vanzin closed the pull request at: https://github.com/apache/spark/pull/16695 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16700 **[Test build #72026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72026/testReport)** for PR 16700 at commit

[GitHub] spark pull request #16710: [SPARK-18872] New test cases for EXISTS subquery

2017-01-26 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/16710 [SPARK-18872] New test cases for EXISTS subquery ## What changes were proposed in this pull request? This PR adds the first set of tests for EXISTS subquery. - exists-basic.sql

[GitHub] spark issue #16680: [SPARK-16101][SQL] Refactoring CSV schema inference path...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16680 **[Test build #72024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72024/testReport)** for PR 16680 at commit

[GitHub] spark issue #16494: [SPARK-17975][MLLIB] Fix EMLDAOptimizer failing with Cla...

2017-01-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16494 @jkbradley I verified that this bug is fixed after rebasing to latest. Should I go ahead and close this PR? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16043 **[Test build #72033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72033/testReport)** for PR 16043 at commit

[GitHub] spark issue #16695: [SPARK-19277][yarn] Localize topology scripts inside Had...

2017-01-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/16695 So I'm just curious, in the specific case you saw this issue, what were the configs? The configs on NM had the correct path or the ones on the gateways were only pointing to gateway. If any

[GitHub] spark pull request #16700: [SPARK-19359][SQL]clear useless path after rename...

2017-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16700#discussion_r98056949 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -899,6 +918,21 @@ private[spark] class

[GitHub] spark issue #16706: [SPARK-19365][Core]Optimize RequestMessage serialization

2017-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16706 **[Test build #72036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72036/testReport)** for PR 16706 at commit

[GitHub] spark pull request #16706: [SPARK-19365][Core]Optimize RequestMessage serial...

2017-01-26 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16706#discussion_r98061320 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEndpointAddress.scala --- @@ -25,10 +27,11 @@ import org.apache.spark.SparkException * The

[GitHub] spark issue #16712: [SPARK-18873][SQL][TEST] New test cases for scalar subqu...

2017-01-26 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/16712 Attached are a slightly modified version of the submitted test file to adapt to IBM DB2 syntax, and the result of the run. [Modified version of the test

[GitHub] spark issue #16712: [SPARK-18873][SQL][TEST] New test cases for scalar subqu...

2017-01-26 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/16712 @kevinyu, @gatorsmile. Also FYI to @hvanhovell. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16536: [SPARK-19163][PYTHON][SQL] Delay _judf initialization to...

2017-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72034/ Test PASSed. ---

  1   2   3   4   5   >