[jira] [Assigned] (SPARK-16619) Add shuffle service metrics entry in monitoring docs

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16619: Assignee: (was: Apache Spark) > Add shuffle service metrics entry in monitoring docs

[jira] [Commented] (SPARK-16619) Add shuffle service metrics entry in monitoring docs

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383617#comment-15383617 ] Apache Spark commented on SPARK-16619: -- User 'lovexi' has created a pull request for this issue:

[jira] [Commented] (SPARK-16216) CSV data source does not write date and timestamp correctly

2016-07-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383616#comment-15383616 ] Reynold Xin commented on SPARK-16216: - How does this work with JSON? > CSV data source does not

[jira] [Assigned] (SPARK-16619) Add shuffle service metrics entry in monitoring docs

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16619: Assignee: Apache Spark > Add shuffle service metrics entry in monitoring docs >

[jira] [Created] (SPARK-16619) Add shuffle service metrics entry in monitoring docs

2016-07-18 Thread YangyangLiu (JIRA)
YangyangLiu created SPARK-16619: --- Summary: Add shuffle service metrics entry in monitoring docs Key: SPARK-16619 URL: https://issues.apache.org/jira/browse/SPARK-16619 Project: Spark Issue

[jira] [Created] (SPARK-16618) Binary classification using Naive bayes

2016-07-18 Thread mahendra singh (JIRA)
mahendra singh created SPARK-16618: -- Summary: Binary classification using Naive bayes Key: SPARK-16618 URL: https://issues.apache.org/jira/browse/SPARK-16618 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16495) Add ADMM optimizer in mllib package

2016-07-18 Thread zunwen you (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383477#comment-15383477 ] zunwen you commented on SPARK-16495: I am working on it. I am going to use the implemented ADMM in

[jira] [Comment Edited] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-07-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383471#comment-15383471 ] Hyukjin Kwon edited comment on SPARK-16610 at 7/19/16 2:02 AM: --- Thank you

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-07-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383471#comment-15383471 ] Hyukjin Kwon commented on SPARK-16610: -- Yea, actually I pointed this out in the PR. Check out this

[jira] [Commented] (SPARK-13514) Spark Shuffle Service 1.6.0 issue in Yarn

2016-07-18 Thread Kevin Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383437#comment-15383437 ] Kevin Zhang commented on SPARK-13514: - yes file:// does exsit and this is definitely a work around.

[jira] [Resolved] (SPARK-16615) Expose sqlContext in SparkSession

2016-07-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16615. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Expose sqlContext in

[jira] [Resolved] (SPARK-16590) Improve LogicalPlanToSQLSuite to check generated SQL directly

2016-07-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16590. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Improve

[jira] [Commented] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

2016-07-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383342#comment-15383342 ] Joseph K. Bradley commented on SPARK-14816: --- SGTM I just sent

[jira] [Assigned] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

2016-07-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-14816: - Assignee: Joseph K. Bradley > Update MLlib, GraphX, SparkR websites for 2.0 >

[jira] [Updated] (SPARK-16579) Add a spark install function

2016-07-18 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16579: -- Assignee: Junyang Qian > Add a spark install function > > >

[jira] [Assigned] (SPARK-16617) Upgrade to Avro 1.8.x

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16617: Assignee: Apache Spark > Upgrade to Avro 1.8.x > - > >

[jira] [Assigned] (SPARK-16617) Upgrade to Avro 1.8.x

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16617: Assignee: (was: Apache Spark) > Upgrade to Avro 1.8.x > - > >

[jira] [Commented] (SPARK-16617) Upgrade to Avro 1.8.x

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383262#comment-15383262 ] Apache Spark commented on SPARK-16617: -- User 'benmccann' has created a pull request for this issue:

[jira] [Created] (SPARK-16617) Upgrade to Avro 1.8.x

2016-07-18 Thread Ben McCann (JIRA)
Ben McCann created SPARK-16617: -- Summary: Upgrade to Avro 1.8.x Key: SPARK-16617 URL: https://issues.apache.org/jira/browse/SPARK-16617 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383216#comment-15383216 ] Reynold Xin commented on SPARK-16321: - Can you try again and set spark.sql.parquet.filterPushdown to

[jira] [Commented] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383207#comment-15383207 ] Reynold Xin commented on SPARK-16321: - hm I wonder if this is related to predicate pushdown in

[jira] [Commented] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2016-07-18 Thread Weiqiang Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383206#comment-15383206 ] Weiqiang Zhuang commented on SPARK-16611: - To answer @shivaram's question: we are calling lapply

[jira] [Created] (SPARK-16616) Allow Catalyst to take Advantage of Hash Partitioned DataSources

2016-07-18 Thread Russell Spitzer (JIRA)
Russell Spitzer created SPARK-16616: --- Summary: Allow Catalyst to take Advantage of Hash Partitioned DataSources Key: SPARK-16616 URL: https://issues.apache.org/jira/browse/SPARK-16616 Project:

[jira] [Commented] (SPARK-16615) Expose sqlContext in SparkSession

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383193#comment-15383193 ] Apache Spark commented on SPARK-16615: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16615) Expose sqlContext in SparkSession

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16615: Assignee: Apache Spark (was: Reynold Xin) > Expose sqlContext in SparkSession >

[jira] [Assigned] (SPARK-16615) Expose sqlContext in SparkSession

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16615: Assignee: Reynold Xin (was: Apache Spark) > Expose sqlContext in SparkSession >

[jira] [Created] (SPARK-16615) Expose sqlContext in SparkSession

2016-07-18 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16615: --- Summary: Expose sqlContext in SparkSession Key: SPARK-16615 URL: https://issues.apache.org/jira/browse/SPARK-16615 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-16602) Spark2.0-error occurs when execute the sql statement which includes "nvl" function while spark1.6 supports

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16602: Assignee: Apache Spark > Spark2.0-error occurs when execute the sql statement which

[jira] [Assigned] (SPARK-16602) Spark2.0-error occurs when execute the sql statement which includes "nvl" function while spark1.6 supports

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16602: Assignee: (was: Apache Spark) > Spark2.0-error occurs when execute the sql statement

[jira] [Commented] (SPARK-16602) Spark2.0-error occurs when execute the sql statement which includes "nvl" function while spark1.6 supports

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383177#comment-15383177 ] Apache Spark commented on SPARK-16602: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-16614) DirectJoin with DataSource for SparkSQL

2016-07-18 Thread Russell Spitzer (JIRA)
Russell Spitzer created SPARK-16614: --- Summary: DirectJoin with DataSource for SparkSQL Key: SPARK-16614 URL: https://issues.apache.org/jira/browse/SPARK-16614 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16602) Spark2.0-error occurs when execute the sql statement which includes "nvl" function while spark1.6 supports

2016-07-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383136#comment-15383136 ] Dongjoon Hyun commented on SPARK-16602: --- Right. I get the same result with you for the following

[jira] [Updated] (SPARK-16613) A bug in RDD pipe operation

2016-07-18 Thread Alex Krasnyansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Krasnyansky updated SPARK-16613: - Description: Suppose we have such Spark code {code} object PipeExample { def

[jira] [Updated] (SPARK-16613) A bug in RDD pipe operation

2016-07-18 Thread Alex Krasnyansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Krasnyansky updated SPARK-16613: - Description: Suppose we have such Spark code {code} object PipeExample { def

[jira] [Created] (SPARK-16613) A bug in RDD pipe operation

2016-07-18 Thread Alex Krasnyansky (JIRA)
Alex Krasnyansky created SPARK-16613: Summary: A bug in RDD pipe operation Key: SPARK-16613 URL: https://issues.apache.org/jira/browse/SPARK-16613 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-18 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383081#comment-15383081 ] Ruben Janssen edited comment on SPARK-9120 at 7/18/16 9:25 PM: --- Bumping this

[jira] [Comment Edited] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-18 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383081#comment-15383081 ] Ruben Janssen edited comment on SPARK-9120 at 7/18/16 9:18 PM: --- Bumping this

[jira] [Comment Edited] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-18 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383081#comment-15383081 ] Ruben Janssen edited comment on SPARK-9120 at 7/18/16 9:16 PM: --- Bumping this

[jira] [Comment Edited] (SPARK-16605) Spark2.0 cannot "select" data from a table stored as an orc file which has been created by hive while hive or spark1.6 supports

2016-07-18 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383082#comment-15383082 ] Xin Wu edited comment on SPARK-16605 at 7/18/16 9:17 PM: - The current issue for

[jira] [Commented] (SPARK-16605) Spark2.0 cannot "select" data from a table stored as an orc file which has been created by hive while hive or spark1.6 supports

2016-07-18 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383082#comment-15383082 ] Xin Wu commented on SPARK-16605: The current issue for dealing with ORC data inserted by Hive is that the

[jira] [Commented] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-18 Thread Ruben Janssen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383081#comment-15383081 ] Ruben Janssen commented on SPARK-9120: -- Bumping this JIRA because of the recent PR for JIRA

[jira] [Resolved] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16515. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 14249

[jira] [Updated] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16515: - Assignee: Adrian Wang > [SPARK][SQL] transformation script got failure for python script >

[jira] [Commented] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383050#comment-15383050 ] Shivaram Venkataraman commented on SPARK-16611: --- I think its a bit different as SPARK-16581

[jira] [Commented] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2016-07-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383047#comment-15383047 ] Felix Cheung commented on SPARK-16611: -- Is this SPARK-16581? > Expose several hidden DataFrame/RDD

[jira] [Resolved] (SPARK-16612) Introduce a way for users to easily add support for new services that need delegation tokens

2016-07-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16612. Resolution: Duplicate > Introduce a way for users to easily add support for new services

[jira] [Commented] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382834#comment-15382834 ] Apache Spark commented on SPARK-16515: -- User 'yhuai' has created a pull request for this issue:

[jira] [Commented] (SPARK-16581) Making JVM backend calling functions public

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382812#comment-15382812 ] Shivaram Venkataraman commented on SPARK-16581: --- Importing comment from SPARK-16608 {code}

[jira] [Resolved] (SPARK-16608) Expose JVM SparkR API functions

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-16608. --- Resolution: Duplicate > Expose JVM SparkR API functions >

[jira] [Commented] (SPARK-16608) Expose JVM SparkR API functions

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382810#comment-15382810 ] Shivaram Venkataraman commented on SPARK-16608: --- [~olarayej] I'm marking this as duplicate

[jira] [Updated] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar D. Lara Yejas updated SPARK-16611: Description: Expose the following functions: - lapply or map - lapplyPartition or

[jira] [Updated] (SPARK-16608) Expose JVM SparkR API functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar D. Lara Yejas updated SPARK-16608: Description: Expose the following functions: - invokeJava - callJStatic -

[jira] [Created] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
Oscar D. Lara Yejas created SPARK-16611: --- Summary: Expose several hidden DataFrame/RDD functions Key: SPARK-16611 URL: https://issues.apache.org/jira/browse/SPARK-16611 Project: Spark

[jira] [Updated] (SPARK-16608) Expose JVM SparkR API functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar D. Lara Yejas updated SPARK-16608: Description: Expose the following functions: - invokeJava - callJStatic -

[jira] [Updated] (SPARK-16608) Expose JVM SparkR API functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar D. Lara Yejas updated SPARK-16608: Description: - invokeJava - callJStatic - callJMethod  - cleanup.jobj  - broadcast

[jira] [Updated] (SPARK-16608) Expose JVM SparkR API functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar D. Lara Yejas updated SPARK-16608: Summary: Expose JVM SparkR API functions (was: Expose some low-level SparkR

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382782#comment-15382782 ] Yin Huai commented on SPARK-16610: -- [~hyukjin.kwon] Will you have time to take a look at this bug? Thank

[jira] [Created] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-07-18 Thread Yin Huai (JIRA)
Yin Huai created SPARK-16610: Summary: When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options Key: SPARK-16610 URL:

[jira] [Updated] (SPARK-15581) MLlib 2.1 Roadmap

2016-07-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-15581: Description: This is a master list for MLlib improvements we are working on for the next release. Please

[jira] [Created] (SPARK-16609) Single function for parsing timestamps/dates

2016-07-18 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-16609: Summary: Single function for parsing timestamps/dates Key: SPARK-16609 URL: https://issues.apache.org/jira/browse/SPARK-16609 Project: Spark Issue

[jira] [Updated] (SPARK-16609) Single function for parsing timestamps/dates

2016-07-18 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-16609: - Target Version/s: 2.1.0 > Single function for parsing timestamps/dates >

[jira] [Created] (SPARK-16608) Expose some low-level SparkR functions

2016-07-18 Thread Oscar D. Lara Yejas (JIRA)
Oscar D. Lara Yejas created SPARK-16608: --- Summary: Expose some low-level SparkR functions Key: SPARK-16608 URL: https://issues.apache.org/jira/browse/SPARK-16608 Project: Spark Issue

[jira] [Commented] (SPARK-16394) Timestamp conversion error in pyspark.sql.Row because of timezones

2016-07-18 Thread Martin Tapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382716#comment-15382716 ] Martin Tapp commented on SPARK-16394: - We found it also happens when you take a python datetime and

[jira] [Commented] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382684#comment-15382684 ] Shivaram Venkataraman commented on SPARK-14816: --- Yeah I'm fine with spending more effort on

[jira] [Resolved] (SPARK-16301) Analyzer rule for resolving using joins should respect case sensitivity setting

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16301. -- Resolution: Fixed Fix Version/s: 2.0.0 > Analyzer rule for resolving using joins should respect

[jira] [Commented] (SPARK-16301) Analyzer rule for resolving using joins should respect case sensitivity setting

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382674#comment-15382674 ] Yin Huai commented on SPARK-16301: -- oh yes. Thank you for pinging me. > Analyzer rule for resolving

[jira] [Commented] (SPARK-16418) DataFrame.filter fails if it references a window function

2016-07-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382668#comment-15382668 ] Dongjoon Hyun commented on SPARK-16418: --- Thanks. > DataFrame.filter fails if it references a

[jira] [Commented] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-07-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382662#comment-15382662 ] Dongjoon Hyun commented on SPARK-16589: --- For me, it looks good to me. In the PR, you can get

[jira] [Commented] (SPARK-16418) DataFrame.filter fails if it references a window function

2016-07-18 Thread Erik Wright (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382657#comment-15382657 ] Erik Wright commented on SPARK-16418: - Sorry, perhaps I confused things by showing something that

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382658#comment-15382658 ] Shivaram Venkataraman commented on SPARK-15799: --- Yeah - so the assumption is that the user

[jira] [Assigned] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16589: Assignee: (was: Apache Spark) > Chained cartesian produces incorrect number of

[jira] [Commented] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382620#comment-15382620 ] Apache Spark commented on SPARK-16589: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-07-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16589: Assignee: Apache Spark > Chained cartesian produces incorrect number of records >

[jira] [Commented] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-07-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382614#comment-15382614 ] Maciej Szymkiewicz commented on SPARK-16589: [~dongjoon] I'll work on that but I am not

[jira] [Commented] (SPARK-16607) Aggregator with null initialisation will result in null

2016-07-18 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382602#comment-15382602 ] Amit Sela commented on SPARK-16607: --- Copy of the thread discussing this in the user mailing list:

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-18 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382586#comment-15382586 ] Amit Sela commented on SPARK-15810: --- opened https://issues.apache.org/jira/browse/SPARK-16607 >

[jira] [Updated] (SPARK-16607) Aggregator with null initialisation will result in null

2016-07-18 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Sela updated SPARK-16607: -- Description: Java code example: {code} SparkSession session = SparkSession.builder()

[jira] [Resolved] (SPARK-16055) sparkR.init() can not load sparkPackages when executing an R file

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-16055. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue

[jira] [Created] (SPARK-16607) Aggregator with null initialisation will result in null

2016-07-18 Thread Amit Sela (JIRA)
Amit Sela created SPARK-16607: - Summary: Aggregator with null initialisation will result in null Key: SPARK-16607 URL: https://issues.apache.org/jira/browse/SPARK-16607 Project: Spark Issue

[jira] [Commented] (SPARK-16421) Improve output from ML examples

2016-07-18 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382579#comment-15382579 ] Bryan Cutler commented on SPARK-16421: -- Yeah, I'm working on it now > Improve output from ML

[jira] [Updated] (SPARK-16351) Avoid record-per type dispatch in JSON when writing

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16351: - Assignee: Hyukjin Kwon > Avoid record-per type dispatch in JSON when writing >

[jira] [Comment Edited] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

2016-07-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381438#comment-15381438 ] Joseph K. Bradley edited comment on SPARK-14816 at 7/18/16 4:49 PM:

[jira] [Resolved] (SPARK-16351) Avoid record-per type dispatch in JSON when writing

2016-07-18 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16351. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14028

[jira] [Updated] (SPARK-16055) sparkR.init() can not load sparkPackages when executing an R file

2016-07-18 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-16055: -- Assignee: Krishna Kalyan > sparkR.init() can not load sparkPackages when

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:54 PM: - OK.

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:48 PM: - OK.

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:44 PM: - OK.

[jira] [Commented] (SPARK-14699) Driver is marked as failed even it runs successfully

2016-07-18 Thread Daniel Bruckner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382470#comment-15382470 ] Daniel Bruckner commented on SPARK-14699: - Any plans to backport to 1.6? Was resolved in April

[jira] [Updated] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-16321: --- Attachment: Spark16.nps Spark2.nps > Pyspark 2.0 performance drop vs pyspark

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:17 PM: - OK.

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:14 PM: - OK.

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:14 PM: - OK.

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 3:13 PM: - OK. I

[jira] [Updated] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-16321: --- Attachment: spark16._trace.png spark2_trace.png > Pyspark 2.0 performance

[jira] [Commented] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382429#comment-15382429 ] Maciej Bryński commented on SPARK-16321: OK. I think that I found difference. 1) I tested

[jira] [Commented] (SPARK-16532) Provide a REST API for submitting and tracking status of jobs

2016-07-18 Thread Joao Vasques (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382408#comment-15382408 ] Joao Vasques commented on SPARK-16532: -- [~apachespark] any hints on this? Thanks > Provide a REST

[jira] [Commented] (SPARK-14464) Logistic regression performs poorly for very large vectors, even when the number of non-zero features is small

2016-07-18 Thread Daniel Siegmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382382#comment-15382382 ] Daniel Siegmann commented on SPARK-14464: - Nick, thanks for pointing out this issue. Looks like

[jira] [Commented] (SPARK-16606) Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

2016-07-18 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382346#comment-15382346 ] Jacek Laskowski commented on SPARK-16606: - On it. Thanks. > Misleading warning for

[jira] [Comment Edited] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382113#comment-15382113 ] Maciej Bryński edited comment on SPARK-16321 at 7/18/16 2:12 PM: - OK. I

[jira] [Updated] (SPARK-16321) Pyspark 2.0 performance drop vs pyspark 1.6

2016-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-16321: --- Attachment: visualvm_spark2_G1GC.png > Pyspark 2.0 performance drop vs pyspark 1.6 >

  1   2   >