[GitHub] spark issue #13508: [SPARK-15766][SparkR]:R should export is.nan

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13508 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13508: [SPARK-15766][SparkR]:R should export is.nan

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13508 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59984/ Test PASSed. ---

[GitHub] spark issue #13508: [SPARK-15766][SparkR]:R should export is.nan

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13508 **[Test build #59984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59984/consoleFull)** for PR 13508 at commit

[GitHub] spark pull request #12258: [SPARK-14485][CORE] ignore task finished for exec...

2016-06-03 Thread zhonghaihua
Github user zhonghaihua commented on a diff in the pull request: https://github.com/apache/spark/pull/12258#discussion_r65797802 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13494 Can you try to write a design doc on this? Would be great to discuss the reasons why we might want this, the kind of queries that can be answered, corner cases, and how it should be implemented.

[GitHub] spark pull request #12258: [SPARK-14485][CORE] ignore task finished for exec...

2016-06-03 Thread zhonghaihua
Github user zhonghaihua commented on a diff in the pull request: https://github.com/apache/spark/pull/12258#discussion_r6579 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #12258: [SPARK-14485][CORE] ignore task finished for exec...

2016-06-03 Thread zhonghaihua
Github user zhonghaihua commented on a diff in the pull request: https://github.com/apache/spark/pull/12258#discussion_r65797773 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -343,17 +343,31 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13508: [SPARK-15766][SparkR]:R should export is.nan

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13508 **[Test build #59984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59984/consoleFull)** for PR 13508 at commit

[GitHub] spark pull request #13500: [SPARK-15756] [SQL] Support command 'create table...

2016-06-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13505 hm probably shouldn't happen in this pr but i'm wondering if it'd make sense to generalize AttributeSeq and use it everywhere, rather than Seq[Attribute]. --- If your project is set up for it, you

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65797603 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/package.scala --- @@ -86,11 +86,31 @@ package object expressions { /**

[GitHub] spark issue #13500: [SPARK-15756] [SQL] Support command 'create table stored...

2016-06-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13500 Thanks - merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13508: [SPARK-15766][SparkR]:R should export is.nan

2016-06-03 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/13508 [SPARK-15766][SparkR]:R should export is.nan ## What changes were proposed in this pull request? When reviewing SPARK-15545, we found that is.nan is not exported, which should be

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13504 LGTM other than that small comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13504: [SPARK-15762][SQL] Cache Metadata & StructType ha...

2016-06-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13504#discussion_r65797565 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Metadata.scala --- @@ -104,7 +104,8 @@ sealed class Metadata private[types] (private[types]

[GitHub] spark pull request #13506: [SPARK-15763][SQL] Support DELETE FILE command na...

2016-06-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13506#discussion_r65797550 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1441,6 +1441,32 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark issue #13507: [SPARK-15765][SQL][Streaming] Make continuous Parquet wr...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13507 **[Test build #59983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59983/consoleFull)** for PR 13507 at commit

[GitHub] spark pull request #13507: [SPARK-15765][SQL][Streaming] Make continuous Par...

2016-06-03 Thread lw-lin
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/13507 [SPARK-15765][SQL][Streaming] Make continuous Parquet writing consistent with non-consistent Parquet writing ## What changes were proposed in this pull request? Currently there are some

[GitHub] spark issue #13442: [SPARK-15654][SQL] Check if all the input files are spli...

2016-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/13442 @rxin plz check this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13442: [SPARK-15654][SQL] Check if all the input files are spli...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13442 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59982/ Test PASSed. ---

[GitHub] spark issue #13442: [SPARK-15654][SQL] Check if all the input files are spli...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13442 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13442: [SPARK-15654][SQL] Check if all the input files are spli...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13442 **[Test build #59982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59982/consoleFull)** for PR 13442 at commit

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r65796528 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -88,14 +106,6 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r65796510 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -58,15 +59,32 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #13413: [SPARK-15663][SQL] SparkSession.catalog.listFunct...

2016-06-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/13413#discussion_r65796505 --- Diff: python/pyspark/sql/tests.py --- @@ -1481,17 +1481,7 @@ def test_list_functions(self): spark.sql("CREATE DATABASE some_db")

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13504 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59981/ Test PASSed. ---

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13504 **[Test build #59981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59981/consoleFull)** for PR 13504 at commit

[GitHub] spark issue #13461: [SPARK-15721][ML] Make DefaultParamsReadable, DefaultPar...

2016-06-03 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/13461 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13403: [SPARK-15660][CORE] RDD and Dataset should show the cons...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59979/ Test PASSed. ---

[GitHub] spark issue #13403: [SPARK-15660][CORE] RDD and Dataset should show the cons...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13403 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13403: [SPARK-15660][CORE] RDD and Dataset should show the cons...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13403 **[Test build #59979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59979/consoleFull)** for PR 13403 at commit

[GitHub] spark issue #13442: [SPARK-15654][SQL] Check if all the input files are spli...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13442 **[Test build #59982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59982/consoleFull)** for PR 13442 at commit

[GitHub] spark issue #13444: [SPARK-15530][SQL] Set #parallelism for file listing in ...

2016-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/13444 @yhuai ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13486: [SPARK-15743][SQL] Prevent saving with all-column partit...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13486 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13486: [SPARK-15743][SQL] Prevent saving with all-column partit...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13486 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59977/ Test PASSed. ---

[GitHub] spark issue #13486: [SPARK-15743][SQL] Prevent saving with all-column partit...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13486 **[Test build #59977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59977/consoleFull)** for PR 13486 at commit

[GitHub] spark issue #12173: [SPARK-13792][SQL] Limit logging of bad records in CSVRe...

2016-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/12173 @falaki ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13436: [SPARK-15696][SQL] Improve `crosstab` to have a consiste...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13436 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59978/ Test PASSed. ---

[GitHub] spark issue #13436: [SPARK-15696][SQL] Improve `crosstab` to have a consiste...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13436 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13505 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13436: [SPARK-15696][SQL] Improve `crosstab` to have a consiste...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13436 **[Test build #59978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59978/consoleFull)** for PR 13436 at commit

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13505 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59976/ Test PASSed. ---

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13505 **[Test build #59976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59976/consoleFull)** for PR 13505 at commit

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13505 Here's a flame graph of bindReferences dominating the CPU used for a 10k column query: [profile](https://github.com/apache/spark/files/298644/slow-bind-refs.svg.zip) --- If your project is

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13504 [hash code profile](https://github.com/apache/spark/files/298642/hashcode.svg.zip) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13505 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59980/ Test FAILed. ---

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13505 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13505 **[Test build #59980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59980/consoleFull)** for PR 13505 at commit

[GitHub] spark issue #13500: [SPARK-15756] [SQL] Support command 'create table stored...

2016-06-03 Thread lianhuiwang
Github user lianhuiwang commented on the issue: https://github.com/apache/spark/pull/13500 @rxin I have updated PR description. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/13505 @rxin, @ericl has some new benchmarks which operate on even wider schemas and which uncovered this bottleneck. Adding the caching of the map here resulted in a huge scalability improvement. Maybe

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/13504 @rxin, I don't have the specific performance numbers handy but in an optimizer stress-test benchmark run by @ericl, these hashCode calls accounted for roughly 40% of the total CPU time and this

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59974/ Test PASSed. ---

[GitHub] spark issue #13407: [SPARK-15665] [CORE] spark-submit --kill and --status ar...

2016-06-03 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/13407 Thanks @vanzin for review and merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13504 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13504 **[Test build #59974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59974/consoleFull)** for PR 13504 at commit

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13504 **[Test build #59981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59981/consoleFull)** for PR 13504 at commit

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65795007 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -296,7 +296,7 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65794996 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/package.scala --- @@ -86,11 +86,31 @@ package object expressions {

[GitHub] spark issue #13248: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

2016-06-03 Thread MechCoder
Github user MechCoder commented on the issue: https://github.com/apache/spark/pull/13248 @praveendareddy21 Just made a first pass. Also please run PEP8 on your code --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13496: [SPARK-15753][SQL] Move Analyzer stuff to Analyzer from ...

2016-06-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13496 cc @cloud-fan @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794904 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794836 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794779 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #13503: [SPARK-15761] [MLlib] [PySpark] Load ipython when defaul...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13503 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13503: [SPARK-15761] [MLlib] [PySpark] Load ipython when defaul...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59970/ Test PASSed. ---

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13061 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13061 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59972/ Test PASSed. ---

[GitHub] spark issue #13503: [SPARK-15761] [MLlib] [PySpark] Load ipython when defaul...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13503 **[Test build #59970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59970/consoleFull)** for PR 13503 at commit

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13061 **[Test build #59972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59972/consoleFull)** for PR 13061 at commit

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794427 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65794395 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -296,7 +296,7 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794325 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794293 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59971/ Test PASSed. ---

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13504 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13504: [SPARK-15762][SQL] Cache Metadata & StructType hashCodes...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13504 **[Test build #59971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59971/consoleFull)** for PR 13504 at commit

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794126 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #13472: [SPARK-15735] Allow specifying min time to run in microb...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59973/ Test FAILed. ---

[GitHub] spark issue #13472: [SPARK-15735] Allow specifying min time to run in microb...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13472 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13472: [SPARK-15735] Allow specifying min time to run in microb...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13472 **[Test build #59973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59973/consoleFull)** for PR 13472 at commit

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13505 **[Test build #59980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59980/consoleFull)** for PR 13505 at commit

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65794056 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65793951 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #13506: [SPARK-15763][SQL] Support DELETE FILE command natively

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13506 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #13506: [SPARK-15763][SQL] Support DELETE FILE command na...

2016-06-03 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/13506 [SPARK-15763][SQL] Support DELETE FILE command natively ## What changes were proposed in this pull request? Hive supports these cli commands to manage the resource [Hive

[GitHub] spark pull request #13248: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-06-03 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/13248#discussion_r65793865 --- Diff: python/pyspark/ml/stat/distribution.py --- @@ -0,0 +1,267 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13505 **[Test build #59976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59976/consoleFull)** for PR 13505 at commit

[GitHub] spark issue #13403: [SPARK-15660][CORE] RDD and Dataset should show the cons...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13403 **[Test build #59979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59979/consoleFull)** for PR 13403 at commit

[GitHub] spark issue #13436: [SPARK-15696][SQL] Improve `crosstab` to have a consiste...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13436 **[Test build #59978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59978/consoleFull)** for PR 13436 at commit

[GitHub] spark issue #13486: [SPARK-15743][SQL] Prevent saving with all-column partit...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13486 **[Test build #59977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59977/consoleFull)** for PR 13486 at commit

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65793015 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -84,17 +84,27 @@ object BindReferences

[GitHub] spark pull request #12938: [SPARK-15162][SPARK-15164][PySpark][DOCS][ML] upd...

2016-06-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12938#discussion_r65792982 --- Diff: python/pyspark/ml/classification.py --- @@ -183,7 +191,7 @@ def getThresholds(self): If :py:attr:`thresholds` is set, return its value.

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13505 **[Test build #59975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59975/consoleFull)** for PR 13505 at commit

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13505 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59975/ Test FAILed. ---

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13505 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65792855 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -84,17 +84,27 @@ object BindReferences

[GitHub] spark issue #13505: [SPARK-15764][SQL] Replace N^2 loop in BindReferences

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13505 **[Test build #59975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59975/consoleFull)** for PR 13505 at commit

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13505#discussion_r65792692 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -84,17 +84,27 @@ object BindReferences

[GitHub] spark pull request #13505: [SPARK-15764][SQL] Replace N^2 loop in BindRefere...

2016-06-03 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/13505 [SPARK-15764][SQL] Replace N^2 loop in BindReferences BindReferences contains a n^2 loop which causes performance issues when operating over large schemas: to determine the ordinal of an

  1   2   3   4   5   6   >