[jira] [Created] (SPARK-14580) HiveTypeCoercion.IfCoercion should preserve original predicates.

2016-04-12 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-14580: - Summary: HiveTypeCoercion.IfCoercion should preserve original predicates. Key: SPARK-14580 URL: https://issues.apache.org/jira/browse/SPARK-14580 Project: Spark

[jira] [Commented] (SPARK-14579) Fix a race condition in StreamExecution.processAllAvailable

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238089#comment-15238089 ] Apache Spark commented on SPARK-14579: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14579) Fix a race condition in StreamExecution.processAllAvailable

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14579: Assignee: Shixiong Zhu (was: Apache Spark) > Fix a race condition in

[jira] [Assigned] (SPARK-14579) Fix a race condition in StreamExecution.processAllAvailable

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14579: Assignee: Apache Spark (was: Shixiong Zhu) > Fix a race condition in

[jira] [Created] (SPARK-14579) Fix a race condition in StreamExecution.processAllAvailable

2016-04-12 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-14579: Summary: Fix a race condition in StreamExecution.processAllAvailable Key: SPARK-14579 URL: https://issues.apache.org/jira/browse/SPARK-14579 Project: Spark

[jira] [Resolved] (SPARK-14544) Spark UI is very slow in recent Chrome

2016-04-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14544. Resolution: Fixed Fix Version/s: 2.0.0 > Spark UI is very slow in recent Chrome >

[jira] [Updated] (SPARK-13753) Column nullable is derived incorrectly

2016-04-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13753: - Description: There is a problem in spark sql to derive nullable column and used in

[jira] [Updated] (SPARK-13753) Column nullable is derived incorrectly

2016-04-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13753: - Description: There is a problem in spark sql to derive nullable column and used in

[jira] [Updated] (SPARK-13753) Column nullable is derived incorrectly

2016-04-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13753: - Target Version/s: 2.0.0 Priority: Critical (was: Major) > Column nullable

[jira] [Assigned] (SPARK-14578) Can't load a json dataset with nested wide schema

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14578: Assignee: Davies Liu (was: Apache Spark) > Can't load a json dataset with nested wide

[jira] [Commented] (SPARK-14578) Can't load a json dataset with nested wide schema

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238067#comment-15238067 ] Apache Spark commented on SPARK-14578: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14578) Can't load a json dataset with nested wide schema

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14578: Assignee: Apache Spark (was: Davies Liu) > Can't load a json dataset with nested wide

[jira] [Created] (SPARK-14578) Can't load a json dataset with nested wide schema

2016-04-12 Thread Davies Liu (JIRA)
Davies Liu created SPARK-14578: -- Summary: Can't load a json dataset with nested wide schema Key: SPARK-14578 URL: https://issues.apache.org/jira/browse/SPARK-14578 Project: Spark Issue Type:

[jira] [Created] (SPARK-14577) spark.sql.codegen.maxCaseBranches config option

2016-04-12 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-14577: --- Summary: spark.sql.codegen.maxCaseBranches config option Key: SPARK-14577 URL: https://issues.apache.org/jira/browse/SPARK-14577 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14577) spark.sql.codegen.maxCaseBranches config option

2016-04-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238057#comment-15238057 ] Reynold Xin commented on SPARK-14577: - cc [~dongjoon] want to do this? >

[jira] [Assigned] (SPARK-14414) Make error messages consistent across DDLs

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14414: Assignee: Apache Spark (was: Andrew Or) > Make error messages consistent across DDLs >

[jira] [Reopened] (SPARK-14414) Make error messages consistent across DDLs

2016-04-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-14414: --- > Make error messages consistent across DDLs > -- > >

[jira] [Assigned] (SPARK-14414) Make error messages consistent across DDLs

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14414: Assignee: Andrew Or (was: Apache Spark) > Make error messages consistent across DDLs >

[jira] [Closed] (SPARK-14575) Make spark.ml GaussianMixture.probabilityCol output optional

2016-04-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-14575. - Resolution: Won't Fix > Make spark.ml GaussianMixture.probabilityCol output optional >

[jira] [Commented] (SPARK-14575) Make spark.ml GaussianMixture.probabilityCol output optional

2016-04-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238042#comment-15238042 ] Joseph K. Bradley commented on SPARK-14575: --- Actually, I'm closing this. I'd like to do it for

[jira] [Assigned] (SPARK-14573) Python docs Makefile overrides shell environment variables breaking linting

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14573: Assignee: Apache Spark > Python docs Makefile overrides shell environment variables

[jira] [Assigned] (SPARK-14573) Python docs Makefile overrides shell environment variables breaking linting

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14573: Assignee: (was: Apache Spark) > Python docs Makefile overrides shell environment

[jira] [Commented] (SPARK-14573) Python docs Makefile overrides shell environment variables breaking linting

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238004#comment-15238004 ] Apache Spark commented on SPARK-14573: -- User 'holdenk' has created a pull request for this issue:

[jira] [Created] (SPARK-14576) Spark console should display Web UI url

2016-04-12 Thread Ergin Seyfe (JIRA)
Ergin Seyfe created SPARK-14576: --- Summary: Spark console should display Web UI url Key: SPARK-14576 URL: https://issues.apache.org/jira/browse/SPARK-14576 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-11321) Allow addition of non-nullable UDFs

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11321: Assignee: (was: Apache Spark) > Allow addition of non-nullable UDFs >

[jira] [Commented] (SPARK-11321) Allow addition of non-nullable UDFs

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237978#comment-15237978 ] Apache Spark commented on SPARK-11321: -- User 'kevincox' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11321) Allow addition of non-nullable UDFs

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11321: Assignee: Apache Spark > Allow addition of non-nullable UDFs >

[jira] [Commented] (SPARK-14516) Clustering evaluator

2016-04-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237976#comment-15237976 ] Joseph K. Bradley commented on SPARK-14516: --- [~podongfeng] [~akamal] could you please

[jira] [Updated] (SPARK-14574) Pure Java modules should not have _2.xx suffixes in their package names

2016-04-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14574: --- Summary: Pure Java modules should not have _2.xx suffixes in their package names (was: Scala-free

[jira] [Updated] (SPARK-14516) Clustering evaluator

2016-04-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14516: -- Description: MLlib does not have any general purposed clustering metrics with a ground

[jira] [Updated] (SPARK-14516) Clustering evaluator

2016-04-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14516: -- Component/s: (was: MLlib) > Clustering evaluator > > >

[jira] [Updated] (SPARK-14516) Clustering evaluator

2016-04-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14516: -- Summary: Clustering evaluator (was: What about adding general clustering metrics?) >

[jira] [Assigned] (SPARK-14574) Scala-free modules should not have _2.xx suffixes in their package names

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14574: Assignee: Josh Rosen (was: Apache Spark) > Scala-free modules should not have _2.xx

[jira] [Commented] (SPARK-14574) Scala-free modules should not have _2.xx suffixes in their package names

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237969#comment-15237969 ] Apache Spark commented on SPARK-14574: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-14575) Make spark.ml GaussianMixture.probabilityCol output optional

2016-04-12 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-14575: - Summary: Make spark.ml GaussianMixture.probabilityCol output optional Key: SPARK-14575 URL: https://issues.apache.org/jira/browse/SPARK-14575 Project:

[jira] [Assigned] (SPARK-14574) Scala-free modules should not have _2.xx suffixes in their package names

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14574: Assignee: Apache Spark (was: Josh Rosen) > Scala-free modules should not have _2.xx

[jira] [Created] (SPARK-14574) Scala-free modules should not have _2.xx suffixes in their package names

2016-04-12 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-14574: -- Summary: Scala-free modules should not have _2.xx suffixes in their package names Key: SPARK-14574 URL: https://issues.apache.org/jira/browse/SPARK-14574 Project: Spark

[jira] [Resolved] (SPARK-14513) Threads left behind after stopping SparkContext

2016-04-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14513. - Resolution: Fixed Assignee: Terence Yim Fix Version/s: 2.0.0 > Threads left

[jira] [Resolved] (SPARK-14414) Make error messages consistent across DDLs

2016-04-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14414. - Resolution: Fixed Fix Version/s: 2.0.0 > Make error messages consistent across DDLs >

[jira] [Commented] (SPARK-14572) Update Config Doc to specify -Xms in extraJavaOptions

2016-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237937#comment-15237937 ] Sean Owen commented on SPARK-14572: --- I don't think it's necessary to explicitly tell people they can

[jira] [Commented] (SPARK-14311) Model persistence in SparkR

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237918#comment-15237918 ] Xiangrui Meng commented on SPARK-14311: --- I think we can implement a generic load in a Scalar

[jira] [Created] (SPARK-14573) Python docs Makefile overrides shell environment variables breaking linting

2016-04-12 Thread holdenk (JIRA)
holdenk created SPARK-14573: --- Summary: Python docs Makefile overrides shell environment variables breaking linting Key: SPARK-14573 URL: https://issues.apache.org/jira/browse/SPARK-14573 Project: Spark

[jira] [Created] (SPARK-14572) Update Config Doc to specify -Xms in extraJavaOptions

2016-04-12 Thread Dhruve Ashar (JIRA)
Dhruve Ashar created SPARK-14572: Summary: Update Config Doc to specify -Xms in extraJavaOptions Key: SPARK-14572 URL: https://issues.apache.org/jira/browse/SPARK-14572 Project: Spark Issue

[jira] [Commented] (SPARK-14531) Flume streaming should respect maxRate (and backpressure)

2016-04-12 Thread Yong Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237864#comment-15237864 ] Yong Tang commented on SPARK-14531: --- Thanks [~hermansc], I noticed that my previous understanding may

[jira] [Resolved] (SPARK-14562) Improve constraints propagation in Union

2016-04-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14562. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12328

[jira] [Assigned] (SPARK-14568) Log instrumentation in logistic regression as a first task

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14568: Assignee: (was: Apache Spark) > Log instrumentation in logistic regression as a first

[jira] [Assigned] (SPARK-14568) Log instrumentation in logistic regression as a first task

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14568: Assignee: Apache Spark > Log instrumentation in logistic regression as a first task >

[jira] [Commented] (SPARK-14568) Log instrumentation in logistic regression as a first task

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237770#comment-15237770 ] Apache Spark commented on SPARK-14568: -- User 'thunterdb' has created a pull request for this issue:

[jira] [Resolved] (SPARK-14556) Code clean-ups for package o.a.s.sql.execution.streaming.state

2016-04-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-14556. -- Resolution: Fixed Assignee: Liwei Lin Fix Version/s: 2.0.0 > Code clean-ups

[jira] [Updated] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

2016-04-12 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Hunter updated SPARK-14567: --- Description: In order to debug performance issues when training mllib algorithms, it is

[jira] [Updated] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

2016-04-12 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Hunter updated SPARK-14567: --- Description: In order to debug performance issues when training mllib algorithms, it is

[jira] [Created] (SPARK-14571) Log instrumentation in ALS

2016-04-12 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-14571: -- Summary: Log instrumentation in ALS Key: SPARK-14571 URL: https://issues.apache.org/jira/browse/SPARK-14571 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-14549) Copy the Vector and Matrix classes from mllib to ml in mllib-local

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14549: -- Shepherd: Xiangrui Meng > Copy the Vector and Matrix classes from mllib to ml in mllib-local >

[jira] [Created] (SPARK-14570) Log instrumentation in Random forests

2016-04-12 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-14570: -- Summary: Log instrumentation in Random forests Key: SPARK-14570 URL: https://issues.apache.org/jira/browse/SPARK-14570 Project: Spark Issue Type:

[jira] [Created] (SPARK-14569) Log instrumentation in KMeans

2016-04-12 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-14569: -- Summary: Log instrumentation in KMeans Key: SPARK-14569 URL: https://issues.apache.org/jira/browse/SPARK-14569 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-14147) SparkR - ML predictors return features with vector datatype, however SparkR doesn't support it

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14147. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11958

[jira] [Created] (SPARK-14568) Log instrumentation in logistic regression as a first task

2016-04-12 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-14568: -- Summary: Log instrumentation in logistic regression as a first task Key: SPARK-14568 URL: https://issues.apache.org/jira/browse/SPARK-14568 Project: Spark

[jira] [Updated] (SPARK-14147) SparkR - ML predictors return features with vector datatype, however SparkR doesn't support it

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14147: -- Assignee: Yanbo Liang > SparkR - ML predictors return features with vector datatype, however

[jira] [Created] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

2016-04-12 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-14567: -- Summary: Add instrumentation logs to MLlib training algorithms Key: SPARK-14567 URL: https://issues.apache.org/jira/browse/SPARK-14567 Project: Spark

[jira] [Assigned] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14566: Assignee: Cheng Lian (was: Apache Spark) > When appending to partitioned persisted

[jira] [Commented] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237721#comment-15237721 ] Apache Spark commented on SPARK-14566: -- User 'liancheng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14566: Assignee: Apache Spark (was: Cheng Lian) > When appending to partitioned persisted

[jira] [Commented] (SPARK-14564) Python Word2Vec missing setWindowSize method

2016-04-12 Thread Jason C Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237719#comment-15237719 ] Jason C Lee commented on SPARK-14564: - Looks straightforward enough. I will give it a shot! > Python

[jira] [Resolved] (SPARK-14563) SQLTransformer.transformSchema is not implemented correctly

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14563. --- Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by

[jira] [Updated] (SPARK-13597) Python API for GeneralizedLinearRegression

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13597: -- Assignee: Kai Jiang > Python API for GeneralizedLinearRegression >

[jira] [Resolved] (SPARK-13597) Python API for GeneralizedLinearRegression

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13597. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11468

[jira] [Resolved] (SPARK-13322) AFTSurvivalRegression should support feature standardization

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-13322. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11365

[jira] [Updated] (SPARK-13590) Document the behavior of spark.ml logistic regression and AFT survival regression when there are constant features

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13590: -- Summary: Document the behavior of spark.ml logistic regression and AFT survival regression

[jira] [Updated] (SPARK-13590) Document the behavior of spark.ml logistic regression when there are constant features

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-13590: -- Assignee: Yanbo Liang > Document the behavior of spark.ml logistic regression when there are

[jira] [Comment Edited] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237690#comment-15237690 ] Cheng Lian edited comment on SPARK-14566 at 4/12/16 6:25 PM: - This bug is

[jira] [Commented] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237690#comment-15237690 ] Cheng Lian commented on SPARK-14566: This bug is exposed after fixing SPARK-14458. These two bugs

[jira] [Created] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14566: -- Summary: When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema Key: SPARK-14566 URL:

[jira] [Reopened] (SPARK-14154) Simplify the implementation for Kolmogorov–Smirnov test

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-14154: --- Re-open this issue to continue discussion. > Simplify the implementation for Kolmogorov–Smirnov

[jira] [Updated] (SPARK-14565) RandomForest should use parseInt and parseDouble for feature subset size instead of regexes

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14565: -- Description: Using regex is not robust and hard to maintain. > RandomForest should use

[jira] [Created] (SPARK-14565) RandomForest should use parseInt and parseDouble for feature subset size instead of regexes

2016-04-12 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14565: - Summary: RandomForest should use parseInt and parseDouble for feature subset size instead of regexes Key: SPARK-14565 URL: https://issues.apache.org/jira/browse/SPARK-14565

[jira] [Commented] (SPARK-14154) Simplify the implementation for Kolmogorov–Smirnov test

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237657#comment-15237657 ] Xiangrui Meng commented on SPARK-14154: --- [~yuhaoyan] The main purpose of the initial implementation

[jira] [Resolved] (SPARK-14474) Move FileSource offset log into checkpointLocation

2016-04-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-14474. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12247

[jira] [Resolved] (SPARK-14324) Refactor GLMs code in SparkRWrappers

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14324. --- Resolution: Fixed Fix Version/s: 2.0.0 > Refactor GLMs code in SparkRWrappers >

[jira] [Resolved] (SPARK-12566) GLM model family, link function support in SparkR:::glm

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12566. --- Resolution: Fixed Fix Version/s: 2.0.0 > GLM model family, link function support in

[jira] [Assigned] (SPARK-14563) SQLTransformer.transformSchema is not implemented correctly

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14563: Assignee: Apache Spark (was: Xiangrui Meng) > SQLTransformer.transformSchema is not

[jira] [Assigned] (SPARK-14563) SQLTransformer.transformSchema is not implemented correctly

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14563: Assignee: Xiangrui Meng (was: Apache Spark) > SQLTransformer.transformSchema is not

[jira] [Commented] (SPARK-14563) SQLTransformer.transformSchema is not implemented correctly

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237611#comment-15237611 ] Apache Spark commented on SPARK-14563: -- User 'mengxr' has created a pull request for this issue:

[jira] [Updated] (SPARK-14563) SQLTransformer.transformSchema is not implemented correctly

2016-04-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-14563: -- Description: `transformSchema` uses `__THIS__` as a temp table name, which would cause errors

[jira] [Created] (SPARK-14564) Python Word2Vec missing setWindowSize method

2016-04-12 Thread Brad Willard (JIRA)
Brad Willard created SPARK-14564: Summary: Python Word2Vec missing setWindowSize method Key: SPARK-14564 URL: https://issues.apache.org/jira/browse/SPARK-14564 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-14563) SQLTransformer.transformSchema is not implemented correctly

2016-04-12 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14563: - Summary: SQLTransformer.transformSchema is not implemented correctly Key: SPARK-14563 URL: https://issues.apache.org/jira/browse/SPARK-14563 Project: Spark

[jira] [Created] (SPARK-14561) History Server does not see new logs in S3

2016-04-12 Thread Miles Crawford (JIRA)
Miles Crawford created SPARK-14561: -- Summary: History Server does not see new logs in S3 Key: SPARK-14561 URL: https://issues.apache.org/jira/browse/SPARK-14561 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14562) Improve constraints propagation in Union

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237528#comment-15237528 ] Apache Spark commented on SPARK-14562: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14562) Improve constraints propagation in Union

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14562: Assignee: Davies Liu (was: Apache Spark) > Improve constraints propagation in Union >

[jira] [Assigned] (SPARK-14562) Improve constraints propagation in Union

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14562: Assignee: Apache Spark (was: Davies Liu) > Improve constraints propagation in Union >

[jira] [Commented] (SPARK-14561) History Server does not see new logs in S3

2016-04-12 Thread Miles Crawford (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237525#comment-15237525 ] Miles Crawford commented on SPARK-14561: Steve Loughran on the user list says: {quote} s3 isn't a

[jira] [Created] (SPARK-14562) Improve constraints propagation in Union

2016-04-12 Thread Davies Liu (JIRA)
Davies Liu created SPARK-14562: -- Summary: Improve constraints propagation in Union Key: SPARK-14562 URL: https://issues.apache.org/jira/browse/SPARK-14562 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-14503) spark.ml API for FPGrowth

2016-04-12 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237407#comment-15237407 ] Gayathri Murali edited comment on SPARK-14503 at 4/12/16 3:44 PM: --

[jira] [Commented] (SPARK-14503) spark.ml API for FPGrowth

2016-04-12 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237407#comment-15237407 ] Gayathri Murali commented on SPARK-14503: - [~josephkb] [~yuhaoyan] and I can work on this. Will

[jira] [Commented] (SPARK-14154) Simplify the implementation for Kolmogorov–Smirnov test

2016-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237346#comment-15237346 ] Apache Spark commented on SPARK-14154: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Resolved] (SPARK-3724) RandomForest: More options for feature subset size

2016-04-12 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-3724. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11989

[jira] [Updated] (SPARK-3724) RandomForest: More options for feature subset size

2016-04-12 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-3724: -- Shepherd: Nick Pentreath > RandomForest: More options for feature subset size >

[jira] [Updated] (SPARK-3724) RandomForest: More options for feature subset size

2016-04-12 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-3724: -- Assignee: Yong Tang > RandomForest: More options for feature subset size >

[jira] [Resolved] (SPARK-14493) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14493. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12303

[jira] [Resolved] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14488. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12303

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-04-12 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237239#comment-15237239 ] Imran Rashid commented on SPARK-11293: -- For users that are hitting *real* OOMs from similar error

<    1   2   3   >