[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: The current accumulator API has a few problems: 1. Its type hierarchy is very complic

[jira] [Closed] (SPARK-14629) Add support for custom scala style rules

2016-04-14 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin closed SPARK-14629. - Resolution: Won't Fix > Add support for custom scala style rules > --

[jira] [Commented] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242540#comment-15242540 ] Reynold Xin commented on SPARK-14654: - cc [~holdenk] and [~imranr] > New accumulator

[jira] [Commented] (SPARK-14629) Add support for custom scala style rules

2016-04-14 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242537#comment-15242537 ] Liwei Lin commented on SPARK-14629: --- As is discussed in https://github.com/apache/spark

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: The current accumulator API has a few problems: 1. Its type hierarchy is very complic

[jira] [Commented] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242536#comment-15242536 ] Reynold Xin commented on SPARK-14654: - Note that one challenge is that we most likel

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Assignee: (was: Reynold Xin) > New accumulator API > --- > > Ke

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: The current accumulator API has a few problems: 1. Its type hierarchy is very complic

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: The current accumulator API has a few problems: 1. Its type hierarchy is very complic

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: The current accumulator API has a few problems: 1. Its type hierarchy is very complic

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: This is a proposed new API -- that is substantially simpler than the existing one, an

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: This is a proposed new API -- that is substantially simpler than the existing one, an

[jira] [Assigned] (SPARK-14650) Compile Spark REPL for Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14650: -- Assignee: Josh Rosen > Compile Spark REPL for Scala 2.12 > - >

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: This is a proposed new API -- that is substantially simpler than the existing one, an

[jira] [Commented] (SPARK-14650) Compile Spark REPL for Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242529#comment-15242529 ] Josh Rosen commented on SPARK-14650: Fortunately, this is going to be really easy. I

[jira] [Updated] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14654: Description: This is a proposed new API -- that is substantially simpler than the existing one, an

[jira] [Created] (SPARK-14654) New accumulator API

2016-04-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-14654: --- Summary: New accumulator API Key: SPARK-14654 URL: https://issues.apache.org/jira/browse/SPARK-14654 Project: Spark Issue Type: Sub-task Components:

[jira] [Updated] (SPARK-14623) add label binarizer

2016-04-14 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-14623: - Description: It relates to https://issues.apache.org/jira/browse/SPARK-7445 Map the labels to 0/1. For

[jira] [Closed] (SPARK-14640) @DeveloperApi on PythonUserDefinedType

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-14640. --- Resolution: Not A Problem I'm closing this. PythonUserDefinedType is actually an internal class. > @

[jira] [Commented] (SPARK-13801) DataFrame.col should return unresolved attribute

2016-04-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242436#comment-15242436 ] Takeshi Yamamuro commented on SPARK-13801: -- I got this cause; wrong nullability

[jira] [Created] (SPARK-14653) Remove NumericParser and jackson dependency from mllib-local

2016-04-14 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14653: - Summary: Remove NumericParser and jackson dependency from mllib-local Key: SPARK-14653 URL: https://issues.apache.org/jira/browse/SPARK-14653 Project: Spark

[jira] [Resolved] (SPARK-14374) PySpark ml GBTClassifier, Regressor support export/import

2016-04-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14374. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12383 [https://g

[jira] [Created] (SPARK-14652) pyspark streaming driver unable to cleanup metadata for cached RDDs leading to driver OOM

2016-04-14 Thread Wei Deng (JIRA)
Wei Deng created SPARK-14652: Summary: pyspark streaming driver unable to cleanup metadata for cached RDDs leading to driver OOM Key: SPARK-14652 URL: https://issues.apache.org/jira/browse/SPARK-14652 Pro

[jira] [Resolved] (SPARK-14275) Reimplement TypedAggregateExpression to DeclarativeAggregate

2016-04-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-14275. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12067 [https://githu

[jira] [Updated] (SPARK-14447) Speed up TungstenAggregate w/ keys using AggregateHashMap

2016-04-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14447: - Assignee: Sameer Agarwal > Speed up TungstenAggregate w/ keys using AggregateHashMap > --

[jira] [Resolved] (SPARK-14447) Speed up TungstenAggregate w/ keys using AggregateHashMap

2016-04-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-14447. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12345 [https://github.com/

[jira] [Closed] (SPARK-14432) Add API to calculate the approximate quantiles for multiple columns

2016-04-14 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-14432. --- Resolution: Duplicate > Add API to calculate the approximate quantiles for multiple columns >

[jira] [Commented] (SPARK-14649) DagScheduler runs duplicate tasks on fetch failure

2016-04-14 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242376#comment-15242376 ] Sital Kedia commented on SPARK-14649: - [~kayousterhout] - Any idea how to handle this

[jira] [Created] (SPARK-14651) CREATE TEMPORARY TABLE is not supported yet

2016-04-14 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-14651: --- Summary: CREATE TEMPORARY TABLE is not supported yet Key: SPARK-14651 URL: https://issues.apache.org/jira/browse/SPARK-14651 Project: Spark Issue Type:

[jira] [Created] (SPARK-14650) Compile Spark REPL for Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-14650: -- Summary: Compile Spark REPL for Scala 2.12 Key: SPARK-14650 URL: https://issues.apache.org/jira/browse/SPARK-14650 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242366#comment-15242366 ] Apache Spark commented on SPARK-14622: -- User 'yang0228' has created a pull request f

[jira] [Assigned] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14622: Assignee: (was: Apache Spark) > Retain lost executors status > ---

[jira] [Assigned] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14622: Assignee: Apache Spark > Retain lost executors status > > >

[jira] [Updated] (SPARK-14104) All Python param setters should use the `_set` method.

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14104: -- Target Version/s: 2.0.0 > All Python param setters should use the `_set` method. >

[jira] [Updated] (SPARK-14104) All Python param setters should use the `_set` method.

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14104: -- Shepherd: Joseph K. Bradley Assignee: Seth Hendrickson > All Python param setters s

[jira] [Commented] (SPARK-14460) DataFrameWriter JDBC doesn't Quote/Escape column names

2016-04-14 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242330#comment-15242330 ] Bo Meng commented on SPARK-14460: - I have added the test case that is using "order" as co

[jira] [Updated] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Qingyang Hong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingyang Hong updated SPARK-14622: -- Target Version/s: 2.0.0 (was: 1.6.0) Fix Version/s: (was: 1.6.0)

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242300#comment-15242300 ] Josh Rosen commented on SPARK-14540: It looks like the Scala PR has largely fixed thi

[jira] [Created] (SPARK-14649) DagScheduler runs duplicate tasks on fetch failure

2016-04-14 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-14649: --- Summary: DagScheduler runs duplicate tasks on fetch failure Key: SPARK-14649 URL: https://issues.apache.org/jira/browse/SPARK-14649 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14649) DagScheduler runs duplicate tasks on fetch failure

2016-04-14 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-14649: Description: When running a job we found out that there are many duplicate tasks running after fetc

[jira] [Resolved] (SPARK-14601) Minor doc/usage changes related to removal of Spark assembly

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14601. - Resolution: Fixed Assignee: Mark Grover Fix Version/s: 2.0.0 > Minor doc/usage ch

[jira] [Commented] (SPARK-14346) SHOW CREATE TABLE command (Native)

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242281#comment-15242281 ] Apache Spark commented on SPARK-14346: -- User 'xwu0226' has created a pull request fo

[jira] [Created] (SPARK-14648) Spark EC2 script creates cluster but spark is not installed properly.

2016-04-14 Thread Nikhil (JIRA)
Nikhil created SPARK-14648: -- Summary: Spark EC2 script creates cluster but spark is not installed properly. Key: SPARK-14648 URL: https://issues.apache.org/jira/browse/SPARK-14648 Project: Spark Is

[jira] [Commented] (SPARK-14489) RegressionEvaluator returns NaN for ALS in Spark ml

2016-04-14 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242251#comment-15242251 ] Seth Hendrickson commented on SPARK-14489: -- [~mlnick] I am skeptical that [SPAR

[jira] [Commented] (SPARK-14609) LOAD DATA

2016-04-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242252#comment-15242252 ] Xiao Li commented on SPARK-14609: - BTW, in the implementation, we need to read the follow

[jira] [Assigned] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14647: Assignee: Andrew Or (was: Apache Spark) > Group SQLContext/HiveContext state into Persist

[jira] [Assigned] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14647: Assignee: Apache Spark (was: Andrew Or) > Group SQLContext/HiveContext state into Persist

[jira] [Commented] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242250#comment-15242250 ] Apache Spark commented on SPARK-14647: -- User 'andrewor14' has created a pull request

[jira] [Created] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14647: - Summary: Group SQLContext/HiveContext state into PersistentState Key: SPARK-14647 URL: https://issues.apache.org/jira/browse/SPARK-14647 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14609) LOAD DATA

2016-04-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242238#comment-15242238 ] Xiao Li commented on SPARK-14609: - https://issues.apache.org/jira/browse/SPARK-14603 is t

[jira] [Commented] (SPARK-14460) DataFrameWriter JDBC doesn't Quote/Escape column names

2016-04-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242231#comment-15242231 ] Andrew Ash commented on SPARK-14460: I also had a problem with a column named "order"

[jira] [Created] (SPARK-14646) k-means save/load should put one cluster per row

2016-04-14 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14646: - Summary: k-means save/load should put one cluster per row Key: SPARK-14646 URL: https://issues.apache.org/jira/browse/SPARK-14646 Project: Spark Issue Type

[jira] [Resolved] (SPARK-12869) Optimize conversion from BlockMatrix to IndexedRowMatrix

2016-04-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12869. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10839 [https://g

[jira] [Resolved] (SPARK-14565) RandomForest should use parseInt and parseDouble for feature subset size instead of regexes

2016-04-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14565. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12360 [https://g

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting poin

[jira] [Assigned] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14644: Assignee: (was: Apache Spark) > Binary param can be a shared param with rewording > --

[jira] [Assigned] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14644: Assignee: Apache Spark > Binary param can be a shared param with rewording > -

[jira] [Commented] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242180#comment-15242180 ] Apache Spark commented on SPARK-14644: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14645: Assignee: (was: Apache Spark) > non local Python resource doesn't work with Mesos clus

[jira] [Assigned] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14645: Assignee: Apache Spark > non local Python resource doesn't work with Mesos cluster mode >

[jira] [Commented] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242172#comment-15242172 ] Apache Spark commented on SPARK-14645: -- User 'tnachen' has created a pull request fo

[jira] [Created] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Timothy Chen (JIRA)
Timothy Chen created SPARK-14645: Summary: non local Python resource doesn't work with Mesos cluster mode Key: SPARK-14645 URL: https://issues.apache.org/jira/browse/SPARK-14645 Project: Spark

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242160#comment-15242160 ] Josh Rosen commented on SPARK-14540: There's now a Scala PR to fix this: https://gith

[jira] [Updated] (SPARK-14583) SparkSQL doesn't apply TBLPROPERTIES('serialization.null.format'='') when Hive Table has partitions

2016-04-14 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephane Maarek updated SPARK-14583: Summary: SparkSQL doesn't apply TBLPROPERTIES('serialization.null.format'='') when Hive Tab

[jira] [Updated] (SPARK-14583) SparkSQL doesn't read TBLPROPERTIES('serialization.null.format'='') when Hive Table has partitions

2016-04-14 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephane Maarek updated SPARK-14583: Summary: SparkSQL doesn't read TBLPROPERTIES('serialization.null.format'='') when Hive Tabl

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Since the PipelineMLWriter/PipelineMLReader/PipelineModelMLWriter/PipelineModelMLReader

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Since the PipelineMLWriter/PipelineMLReader/PipelineModelMLWriter/PipelineModelMLReader

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting poin

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Since the PipelineMLWriter/PipelineMLReader/PipelineModelMLWriter/PipelineModelMLReader

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting poin

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting poin

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting poin

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Remove * PipelineMLWriter * PipelineMLReader * PipelineModelMLWriter * PipelineModelMLRea

[jira] [Commented] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242147#comment-15242147 ] Xusen Yin commented on SPARK-14440: --- Sorry for the late response, I'll update it soon.

[jira] [Updated] (SPARK-14130) [Table related commands] Alter column

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14130: -- Assignee: Yin Huai (was: Andrew Or) > [Table related commands] Alter column >

[jira] [Commented] (SPARK-12741) DataFrame count method return wrong size.

2016-04-14 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242132#comment-15242132 ] Stephane Maarek commented on SPARK-12741: - Hi Sean, What do you mean by the behav

[jira] [Commented] (SPARK-14519) Cross-publish Kafka for Scala 2.12.0-M4

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242130#comment-15242130 ] Josh Rosen commented on SPARK-14519: I emailed the Kafka dev list: https://mail-arch

[jira] [Created] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread holdenk (JIRA)
holdenk created SPARK-14644: --- Summary: Binary param can be a shared param with rewording Key: SPARK-14644 URL: https://issues.apache.org/jira/browse/SPARK-14644 Project: Spark Issue Type: Improveme

[jira] [Created] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-14643: -- Summary: Remove overloaded methods which become ambiguous in Scala 2.12 Key: SPARK-14643 URL: https://issues.apache.org/jira/browse/SPARK-14643 Project: Spark I

[jira] [Assigned] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14643: -- Assignee: Josh Rosen > Remove overloaded methods which become ambiguous in Scala 2.12 > --

[jira] [Commented] (SPARK-14434) User guide doc and examples for GaussianMixture in spark.ml

2016-04-14 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242080#comment-15242080 ] Miao Wang commented on SPARK-14434: --- Sent pull request to SPARK-14433. Start working on

[jira] [Assigned] (SPARK-14433) PySpark ml GaussianMixture

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14433: Assignee: (was: Apache Spark) > PySpark ml GaussianMixture > -

[jira] [Commented] (SPARK-14433) PySpark ml GaussianMixture

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241981#comment-15241981 ] Apache Spark commented on SPARK-14433: -- User 'wangmiao1981' has created a pull reque

[jira] [Assigned] (SPARK-14433) PySpark ml GaussianMixture

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14433: Assignee: Apache Spark > PySpark ml GaussianMixture > -- > >

[jira] [Updated] (SPARK-14642) import org.apache.spark.sql.expressions._ breaks udf under functions

2016-04-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14642: - Description: The following code works {code} scala> import org.apache.spark.sql.functions._ import org.ap

[jira] [Created] (SPARK-14642) import org.apache.spark.sql.expressions._ breaks udf under functions

2016-04-14 Thread Yin Huai (JIRA)
Yin Huai created SPARK-14642: Summary: import org.apache.spark.sql.expressions._ breaks udf under functions Key: SPARK-14642 URL: https://issues.apache.org/jira/browse/SPARK-14642 Project: Spark

[jira] [Commented] (SPARK-10179) LogisticRegressionWithSGD does not multiclass

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241933#comment-15241933 ] DB Tsai commented on SPARK-10179: - Since we're moving to ML api, we will not add new feat

[jira] [Closed] (SPARK-10179) LogisticRegressionWithSGD does not multiclass

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai closed SPARK-10179. --- Resolution: Won't Fix > LogisticRegressionWithSGD does not multiclass >

[jira] [Created] (SPARK-14641) Specify worker log dir separately from scratch space dir

2016-04-14 Thread Philipp Hanslovsky (JIRA)
Philipp Hanslovsky created SPARK-14641: -- Summary: Specify worker log dir separately from scratch space dir Key: SPARK-14641 URL: https://issues.apache.org/jira/browse/SPARK-14641 Project: Spark

[jira] [Closed] (SPARK-14499) Add tests to make sure drop partitions of an external table will not delete data

2016-04-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-14499. --- Resolution: Resolved > Add tests to make sure drop partitions of an external table will not delete > data >

[jira] [Updated] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-7159: --- Issue Type: New Feature (was: Improvement) > Support multiclass logistic regression in spark.ml > ---

[jira] [Commented] (SPARK-14306) PySpark ml.classification OneVsRest support export/import

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241899#comment-15241899 ] Joseph K. Bradley commented on SPARK-14306: --- Right! Thanks > PySpark ml.class

[jira] [Commented] (SPARK-14638) Threads of Spark Streaming (with Kafka) looses sight of the executor classpath

2016-04-14 Thread Younos Aboulnaga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241893#comment-15241893 ] Younos Aboulnaga commented on SPARK-14638: -- But I have shown that the worker dir

[jira] [Commented] (SPARK-14638) Threads of Spark Streaming (with Kafka) looses sight of the executor classpath

2016-04-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241885#comment-15241885 ] Sean Owen commented on SPARK-14638: --- The stack trace you show demonstrates it's an init

[jira] [Updated] (SPARK-14374) PySpark ml GBTClassifier, Regressor support export/import

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14374: -- Shepherd: Joseph K. Bradley Assignee: Yanbo Liang > PySpark ml GBTClassifier, Regre

[jira] [Closed] (SPARK-14616) TreeNodeException running Q44 and 58 on Parquet tables

2016-04-14 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN closed SPARK-14616. -- Resolution: Not A Problem > TreeNodeException running Q44 and 58 on Parquet tables > --

[jira] [Resolved] (SPARK-14545) Improve `LikeSimplification` by adding `a%b` rule

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14545. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 > Improve `LikeSim

[jira] [Commented] (SPARK-14616) TreeNodeException running Q44 and 58 on Parquet tables

2016-04-14 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241860#comment-15241860 ] JESSE CHEN commented on SPARK-14616: Build from yesterday did not have this problem.

[jira] [Resolved] (SPARK-14238) Add binary toggle Param to PySpark HashingTF in ML & MLlib

2016-04-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-14238. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12079 [https:/

[jira] [Resolved] (SPARK-14618) RegressionEvaluator doc out of date

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14618. --- Resolution: Fixed Fix Version/s: 1.6.2 1.5.3

  1   2   3   >