[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Created] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15547: -- Summary: Encoder validation is too strict for inner nested structs Key: SPARK-15547 URL: https://issues.apache.org/jira/browse/SPARK-15547 Project: Spark Issue

[jira] [Commented] (SPARK-12550) sbt-launch-lib.bash: line 72: 2404 Killed "$@"

2016-05-25 Thread Pete Clapham (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301569#comment-15301569 ] Pete Clapham commented on SPARK-12550: -- Thanks for your reply Sean,I followed the online docs for

[jira] [Resolved] (SPARK-10372) Add end-to-end tests for the scheduling code

2016-05-25 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-10372. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 8559

[jira] [Issue Comment Deleted] (SPARK-5682) Add encrypted shuffle in spark

2016-05-25 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-5682: Comment: was deleted (was: Steps were added to encode and decode the data, the performance will not be

[jira] [Issue Comment Deleted] (SPARK-5682) Add encrypted shuffle in spark

2016-05-25 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-5682: Comment: was deleted (was: The solution relied on hadoop API and maybe downgrade the performance.  The AES

[jira] [Issue Comment Deleted] (SPARK-5682) Add encrypted shuffle in spark

2016-05-25 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-5682: Comment: was deleted (was: Since the encrypted shuffle in spark is focus on the common module, it maybe

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301528#comment-15301528 ] Yin Huai commented on SPARK-15034: -- Seems it is better to explicitly set local filesystem as the scheme

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301518#comment-15301518 ] Yin Huai commented on SPARK-15034: -- Seems your default file system is hdfs? Since we need a warehouse

[jira] [Closed] (SPARK-12893) RM redirects to incorrect URL in Spark HistoryServer for yarn-cluster mode

2016-05-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-12893. --- Resolution: Not A Problem > RM redirects to incorrect URL in Spark HistoryServer for yarn-cluster

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301508#comment-15301508 ] koert kuipers commented on SPARK-15034: --- this just hit me today. i build spark 2.0.0-SNAPSHOT

[jira] [Created] (SPARK-15546) HiveContext : Connecting to MySQL metastore db - Always creates a DERBY database in first attempt

2016-05-25 Thread Jais Sebastian (JIRA)
Jais Sebastian created SPARK-15546: -- Summary: HiveContext : Connecting to MySQL metastore db - Always creates a DERBY database in first attempt Key: SPARK-15546 URL:

[jira] [Updated] (SPARK-15546) HiveContext : Connecting to MySQL metastore db - Always creates a DERBY database in first attempt

2016-05-25 Thread Jais Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jais Sebastian updated SPARK-15546: --- Attachment: log.txt hive-site.xml > HiveContext : Connecting to MySQL

[jira] [Resolved] (SPARK-15439) Failed to run unit test in SparkR

2016-05-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-15439. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Updated] (SPARK-15439) Failed to run unit test in SparkR

2016-05-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-15439: -- Assignee: Miao Wang > Failed to run unit test in SparkR >

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301468#comment-15301468 ] Yi Zhou commented on SPARK-15345: - I'm confused that it worked well with Apache Spark 1.6 and can't work

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301462#comment-15301462 ] Jeff Zhang commented on SPARK-15345: This seems hadoop distributor specific, you may need to ask in

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301456#comment-15301456 ] Yi Zhou commented on SPARK-15345: - I issued 'show databases;' , ' use XXX' and 'show tables;' and found

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301457#comment-15301457 ] Sean Owen commented on SPARK-15541: --- Yes, this is a subtle Java version issue. It means this was

[jira] [Updated] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12504: -- Assignee: Suresh Thalamati (was: Apache Spark) > JDBC data source credentials are not masked in the

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301437#comment-15301437 ] Jeff Zhang commented on SPARK-15345: bq. but it can't work after deploying the Apache Spark 2.0 and

[jira] [Updated] (SPARK-15545) R remove non-exported unused methods, like jsonRDD

2016-05-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-15545: - Description: Need to review what should be removed. one reason to not remove this right away is

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301427#comment-15301427 ] Yi Zhou commented on SPARK-15345: - Hi [~jeffzhang] In my test environment, i installed CDH 5.7.0 Hive(

[jira] [Created] (SPARK-15545) R remove non-exported unused methods, like jsonRDD

2016-05-25 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-15545: Summary: R remove non-exported unused methods, like jsonRDD Key: SPARK-15545 URL: https://issues.apache.org/jira/browse/SPARK-15545 Project: Spark Issue

[jira] [Commented] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2016-05-25 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301392#comment-15301392 ] Suresh Thalamati commented on SPARK-12504: -- [~srowen] I worked on resolving this issue. For

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-05-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301352#comment-15301352 ] Saisai Shao commented on SPARK-15541: - This method which returns {{KeySetView}} only exists in jdk

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301342#comment-15301342 ] Yi Zhou commented on SPARK-13955: - Thanks a lot [~jerryshao]! > Spark in yarn mode fails >

[jira] [Assigned] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15129: Assignee: (was: Apache Spark) > Clarify conventions for calling Spark and MLlib from

[jira] [Commented] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301322#comment-15301322 ] Apache Spark commented on SPARK-15129: -- User 'GayathriMurali' has created a pull request for this

[jira] [Assigned] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15129: Assignee: Apache Spark > Clarify conventions for calling Spark and MLlib from R >

[jira] [Resolved] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15533. - Resolution: Fixed Assignee: Sameer Agarwal Fix Version/s: 2.0.0 > Deprecate

[jira] [Reopened] (SPARK-15496) Spill metrics not updated when off-heap memory is enabled

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15496: --- > Spill metrics not updated when off-heap memory is enabled >

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15285: -- Assignee: Kazuaki Ishizaki > Generated SpecificSafeProjection.apply method grows beyond 64 KB >

[jira] [Resolved] (SPARK-15496) Spill metrics not updated when off-heap memory is enabled

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15496. --- Resolution: Cannot Reproduce Fix Version/s: (was: 2.0.0) > Spill metrics not updated when

[jira] [Updated] (SPARK-15315) CSV datasource writes garbage for complex types instead of rasing error.

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15315: -- Assignee: Suresh Thalamati > CSV datasource writes garbage for complex types instead of rasing

[jira] [Updated] (SPARK-14292) TPC-DS Query 23 Fails with compile error

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14292: -- Assignee: Herman van Hovell > TPC-DS Query 23 Fails with compile error >

[jira] [Commented] (SPARK-14292) TPC-DS Query 23 Fails with compile error

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301308#comment-15301308 ] Sean Owen commented on SPARK-14292: --- We should really link this to the JIRA/change that resolved it. I

[jira] [Updated] (SPARK-13820) TPC-DS Query 10 fails to compile

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13820: -- Assignee: Davies Liu > TPC-DS Query 10 fails to compile > > >

[jira] [Updated] (SPARK-12071) Programming guide should explain NULL in JVM translate to NA in R

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12071: -- Assignee: Krishna Kalyan > Programming guide should explain NULL in JVM translate to NA in R >

[jira] [Updated] (SPARK-10043) Add window functions into SparkR

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10043: -- Assignee: Sun Rui > Add window functions into SparkR > > >

[jira] [Commented] (SPARK-15526) Shade JPMML

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301298#comment-15301298 ] Sean Owen commented on SPARK-15526: --- I've also shaded JPMML in an app in order to work around the fact

[jira] [Updated] (SPARK-6009) IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND ()

2016-05-25 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-6009: -- Fix Version/s: 1.5.0 > IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND () >

[jira] [Resolved] (SPARK-15527) Duplicate column names with different case after join of DataFrames

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15527. --- Resolution: Not A Problem > Duplicate column names with different case after join of DataFrames >

[jira] [Reopened] (SPARK-15527) Duplicate column names with different case after join of DataFrames

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15527: --- Oops, I think maybe we use "Not A Problem" in this case, but the meaning is clear here > Duplicate

[jira] [Resolved] (SPARK-15527) Duplicate column names with different case after join of DataFrames

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15527. --- Resolution: Fixed OK, did you mean this is resolved? normally we wouldn't make a JIRA for it but I

[jira] [Resolved] (SPARK-15332) OutOfMemory in TimSort

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15332. --- Resolution: Duplicate > OutOfMemory in TimSort > --- > > Key:

[jira] [Commented] (SPARK-12550) sbt-launch-lib.bash: line 72: 2404 Killed "$@"

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301273#comment-15301273 ] Sean Owen commented on SPARK-12550: --- I can't reproduce this. How are you building; what are you

[jira] [Resolved] (SPARK-15524) Strange issue starting spark-shell: screen block

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15524. --- Resolution: Not A Problem This is specific to your environment. Your config is specifying a

[jira] [Updated] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

2016-05-25 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Lowenthal updated SPARK-15544: - Description: Shutting Down a single zookeeper node caused spark master to exit. The

[jira] [Updated] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

2016-05-25 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Lowenthal updated SPARK-15544: - Description: Shutting Down a single zookeeper node caused spark master to exit. The

[jira] [Created] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

2016-05-25 Thread Steven Lowenthal (JIRA)
Steven Lowenthal created SPARK-15544: Summary: Bouncing Zookeeper node causes Active spark master to exit Key: SPARK-15544 URL: https://issues.apache.org/jira/browse/SPARK-15544 Project: Spark

[jira] [Commented] (SPARK-15539) DROP TABLE should throw exceptions, not logError

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301233#comment-15301233 ] Apache Spark commented on SPARK-15539: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15536) Disallow TRUNCATE TABLE with external tables and views

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-15536: - Assignee: Andrew Or (was: Suresh Thalamati) > Disallow TRUNCATE TABLE with external tables and

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > Truncate table does not work on data source

[jira] [Updated] (SPARK-15536) Disallow TRUNCATE TABLE with external tables and views

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15536: -- Summary: Disallow TRUNCATE TABLE with external tables and views (was: Disallow TRUNCATE TABLE with

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Description: Truncate table does not seem to work on data source tables. Repro: {code} val df =

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Description: Truncate table does not seems to work on data source table. It returns success without

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301223#comment-15301223 ] Joseph K. Bradley commented on SPARK-15457: --- I just sent a PR. It does not clear up all of the

[jira] [Commented] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301222#comment-15301222 ] Apache Spark commented on SPARK-15538: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15457: Assignee: Joseph K. Bradley (was: Apache Spark) > Eliminate MLlib 2.0 build warnings

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Summary: Truncate table does not work on data source table (was: Truncate table does not work on data

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301218#comment-15301218 ] Apache Spark commented on SPARK-15457: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-15538: - Assignee: Andrew Or (was: Suresh Thalamati) > Truncate table does not work on data source

[jira] [Assigned] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15457: Assignee: Apache Spark (was: Joseph K. Bradley) > Eliminate MLlib 2.0 build warnings

[jira] [Updated] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15457: -- Description: Several classes and methods have been deprecated and are creating lots of

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301215#comment-15301215 ] Apache Spark commented on SPARK-15533: -- User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301208#comment-15301208 ] Takeshi Yamamuro commented on SPARK-13184: -- okay > Support minPartitions parameter for JSON and

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301205#comment-15301205 ] Reynold Xin commented on SPARK-13184: - Please go for it. > Support minPartitions parameter for JSON

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301202#comment-15301202 ] Takeshi Yamamuro commented on SPARK-13184: -- [~rxin] Is this ticket worth a try? If so, I'll take

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301196#comment-15301196 ] Apache Spark commented on SPARK-15533: -- User 'sameeragarwal' has created a pull request for this

[jira] [Assigned] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15533: Assignee: (was: Apache Spark) > Deprecate Dataset.explode > -

[jira] [Assigned] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15533: Assignee: Apache Spark > Deprecate Dataset.explode > - > >

[jira] [Closed] (SPARK-15480) Some InputInfo missed with window operation applied

2016-05-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu closed SPARK-15480. Resolution: Duplicate > Some InputInfo missed with window operation applied >

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301188#comment-15301188 ] Sameer Agarwal commented on SPARK-15533: [~WeichenXu123] I just created a PR for it. Sorry! >

[jira] [Comment Edited] (SPARK-14377) Review spark.ml parity for classification, except trees

2016-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244059#comment-15244059 ] Joseph K. Bradley edited comment on SPARK-14377 at 5/26/16 12:13 AM: -

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301179#comment-15301179 ] Weichen Xu commented on SPARK-15533: I will work on it and send a PR soon. Thanks! > Deprecate

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301178#comment-15301178 ] Weichen Xu commented on SPARK-15533: I will work on it. Thanks! > Deprecate Dataset.explode >

[jira] [Assigned] (SPARK-15332) OutOfMemory in TimSort

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15332: -- Assignee: Davies Liu > OutOfMemory in TimSort > --- > >

[jira] [Assigned] (SPARK-15391) Spark executor OOM during TimSort

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15391: -- Assignee: Davies Liu > Spark executor OOM during TimSort > -

[jira] [Commented] (SPARK-13445) Selecting "data" with window function does not work unless aliased (using PARTITION BY)

2016-05-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301166#comment-15301166 ] Yin Huai commented on SPARK-13445: -- [~simeons] Can you still be able to reproduce it? I am wondering if

[jira] [Commented] (SPARK-13445) Selecting "data" with window function does not work unless aliased (using PARTITION BY)

2016-05-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301161#comment-15301161 ] Yin Huai commented on SPARK-13445: -- So, if I understand it correctly, it is an issue with 1.6 instead of

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301114#comment-15301114 ] koert kuipers commented on SPARK-13184: --- agreed, there should be a general way for data sources to

[jira] [Assigned] (SPARK-15532) Add SQLConf.ALLOW_MULTIPLE_CONTEXTS back and make the error message configurable

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15532: Assignee: (was: Apache Spark) > Add SQLConf.ALLOW_MULTIPLE_CONTEXTS back and make the

[jira] [Updated] (SPARK-15543) Rename DefaultSources to make them more self-describing

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15543: Summary: Rename DefaultSources to make them more self-describing (was: Rename various

[jira] [Assigned] (SPARK-15532) Add SQLConf.ALLOW_MULTIPLE_CONTEXTS back and make the error message configurable

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15532: Assignee: Apache Spark > Add SQLConf.ALLOW_MULTIPLE_CONTEXTS back and make the error

[jira] [Commented] (SPARK-15532) Add SQLConf.ALLOW_MULTIPLE_CONTEXTS back and make the error message configurable

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301117#comment-15301117 ] Apache Spark commented on SPARK-15532: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15543) Rename DefaultSources to make them more self-describing

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15543: Assignee: Apache Spark (was: Reynold Xin) > Rename DefaultSources to make them more

[jira] [Commented] (SPARK-15520) SparkSession builder in python should also allow overriding confs of existing sessions

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301107#comment-15301107 ] Apache Spark commented on SPARK-15520: -- User 'ericl' has created a pull request for this issue:

[jira] [Created] (SPARK-15543) Rename various DefaultSources to more self-describing names

2016-05-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15543: --- Summary: Rename various DefaultSources to more self-describing names Key: SPARK-15543 URL: https://issues.apache.org/jira/browse/SPARK-15543 Project: Spark

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301058#comment-15301058 ] Reynold Xin commented on SPARK-13184: - I'm moving this out of the CSV umbrella ticket. > Support

[jira] [Updated] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13184: Issue Type: Improvement (was: Sub-task) Parent: (was: SPARK-12420) > Support

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301057#comment-15301057 ] Reynold Xin commented on SPARK-13184: - Rather than doing this specifically for JSON or CSV, it might

[jira] [Assigned] (SPARK-15542) Make error message clear for script './R/install-dev.sh' when R is missing on Mac

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15542: Assignee: Apache Spark > Make error message clear for script './R/install-dev.sh' when R

[jira] [Commented] (SPARK-15542) Make error message clear for script './R/install-dev.sh' when R is missing on Mac

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301029#comment-15301029 ] Apache Spark commented on SPARK-15542: -- User 'keypointt' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15542) Make error message clear for script './R/install-dev.sh' when R is missing on Mac

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15542: Assignee: (was: Apache Spark) > Make error message clear for script

[jira] [Updated] (SPARK-15542) Make error message clear for script './R/install-dev.sh' when R is missing on Mac

2016-05-25 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-15542: Description: I followed instructions here https://github.com/apache/spark/tree/master/R to build sparkR

[jira] [Resolved] (SPARK-15525) Clean sbt build fails to resolve sbt-antlr4 plugin

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15525. - Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.0.0 > Clean sbt

[jira] [Commented] (SPARK-13638) Support for saving with a quote mode

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301017#comment-15301017 ] Reynold Xin commented on SPARK-13638: - [~jurriaanpruis] would you be able to make a pr? > Support

[jira] [Created] (SPARK-15542) Make error message clear for script './R/install-dev.sh' when R is missing on Mac

2016-05-25 Thread Xin Ren (JIRA)
Xin Ren created SPARK-15542: --- Summary: Make error message clear for script './R/install-dev.sh' when R is missing on Mac Key: SPARK-15542 URL: https://issues.apache.org/jira/browse/SPARK-15542 Project:

  1   2   3   >