[jira] [Updated] (SPARK-15803) Support with statement syntax for SparkSession

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15803: --- Assignee: Jeff Zhang > Support with statement syntax for SparkSession >

[jira] [Resolved] (SPARK-15803) Support with statement syntax for SparkSession

2016-06-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15803. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13541

[jira] [Updated] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16035: -- Assignee: Andrea Pasqua > The SparseVector parser fails checking for valid end parenthesis >

[jira] [Resolved] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16035. --- Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by

[jira] [Assigned] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16041: Assignee: Apache Spark > Disallow Duplicate Columns in `partitionBy`, `blockBy` and

[jira] [Assigned] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16041: Assignee: (was: Apache Spark) > Disallow Duplicate Columns in `partitionBy`,

[jira] [Commented] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337552#comment-15337552 ] Apache Spark commented on SPARK-16041: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Updated] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter

2016-06-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16041: Description: Duplicate columns are not allowed in `partitionBy`, `blockBy`, `sortBy` in . The duplicate

[jira] [Updated] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter

2016-06-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16041: Description: Duplicate columns are not allowed in `partitionBy`, `blockBy`, `sortBy` in DataFrameWriter.

[jira] [Created] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy`

2016-06-17 Thread Xiao Li (JIRA)
Xiao Li created SPARK-16041: --- Summary: Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` Key: SPARK-16041 URL: https://issues.apache.org/jira/browse/SPARK-16041 Project: Spark

[jira] [Updated] (SPARK-16041) Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter

2016-06-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-16041: Summary: Disallow Duplicate Columns in `partitionBy`, `blockBy` and `sortBy` in DataFrameWriter (was:

[jira] [Assigned] (SPARK-16040) spark.mllib PIC document extra line of refernece

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16040: Assignee: (was: Apache Spark) > spark.mllib PIC document extra line of refernece >

[jira] [Assigned] (SPARK-16040) spark.mllib PIC document extra line of refernece

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16040: Assignee: Apache Spark > spark.mllib PIC document extra line of refernece >

[jira] [Commented] (SPARK-16040) spark.mllib PIC document extra line of refernece

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337550#comment-15337550 ] Apache Spark commented on SPARK-16040: -- User 'wangmiao1981' has created a pull request for this

[jira] [Created] (SPARK-16040) spark.mllib PIC document extra line of refernece

2016-06-17 Thread Miao Wang (JIRA)
Miao Wang created SPARK-16040: - Summary: spark.mllib PIC document extra line of refernece Key: SPARK-16040 URL: https://issues.apache.org/jira/browse/SPARK-16040 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-16020) Fix complete mode aggregation with console sink

2016-06-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-16020. -- Resolution: Fixed Fix Version/s: 2.0.0 > Fix complete mode aggregation with console

[jira] [Assigned] (SPARK-16037) use by-position resolution when insert into hive table

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16037: Assignee: Apache Spark (was: Wenchen Fan) > use by-position resolution when insert into

[jira] [Assigned] (SPARK-16036) better error message if the number of columns in SELECT clause doesn't match the table schema

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16036: Assignee: Wenchen Fan (was: Apache Spark) > better error message if the number of

[jira] [Assigned] (SPARK-16036) better error message if the number of columns in SELECT clause doesn't match the table schema

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16036: Assignee: Apache Spark (was: Wenchen Fan) > better error message if the number of

[jira] [Commented] (SPARK-16037) use by-position resolution when insert into hive table

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337543#comment-15337543 ] Apache Spark commented on SPARK-16037: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16037) use by-position resolution when insert into hive table

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16037: Assignee: Wenchen Fan (was: Apache Spark) > use by-position resolution when insert into

[jira] [Commented] (SPARK-16036) better error message if the number of columns in SELECT clause doesn't match the table schema

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337542#comment-15337542 ] Apache Spark commented on SPARK-16036: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16029) Deprecate dropTempTable in SparkR

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16029: Assignee: Apache Spark > Deprecate dropTempTable in SparkR >

[jira] [Assigned] (SPARK-16029) Deprecate dropTempTable in SparkR

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16029: Assignee: (was: Apache Spark) > Deprecate dropTempTable in SparkR >

[jira] [Commented] (SPARK-16029) Deprecate dropTempTable in SparkR

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337536#comment-15337536 ] Apache Spark commented on SPARK-16029: -- User 'felixcheung' has created a pull request for this

[jira] [Deleted] (SPARK-16038) we can omit partition list when insert into hive table

2016-06-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan deleted SPARK-16038: > we can omit partition list when insert into hive table >

[jira] [Assigned] (SPARK-16028) Remove the need to pass in a SparkContext for spark.lapply

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16028: Assignee: Apache Spark > Remove the need to pass in a SparkContext for spark.lapply >

[jira] [Assigned] (SPARK-16028) Remove the need to pass in a SparkContext for spark.lapply

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16028: Assignee: (was: Apache Spark) > Remove the need to pass in a SparkContext for

[jira] [Commented] (SPARK-16028) Remove the need to pass in a SparkContext for spark.lapply

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337534#comment-15337534 ] Apache Spark commented on SPARK-16028: -- User 'felixcheung' has created a pull request for this

[jira] [Commented] (SPARK-15159) SparkSession R API

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337532#comment-15337532 ] Apache Spark commented on SPARK-15159: -- User 'felixcheung' has created a pull request for this

[jira] [Commented] (SPARK-9857) Add expression functions into SparkR which conflict with the existing R's generic

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337526#comment-15337526 ] Shivaram Venkataraman commented on SPARK-9857: -- [~yuu.ishik...@gmail.com] [~sunrui] Do we

[jira] [Commented] (SPARK-15124) R 2.0 QA: New R APIs and API docs

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337523#comment-15337523 ] Shivaram Venkataraman commented on SPARK-15124: --- One more item on this list is the

[jira] [Commented] (SPARK-6817) DataFrame UDFs in R

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337525#comment-15337525 ] Shivaram Venkataraman commented on SPARK-6817: -- I think all the ones we need for 2.0 are

[jira] [Resolved] (SPARK-15159) SparkSession R API

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-15159. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Updated] (SPARK-15159) SparkSession R API

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-15159: -- Assignee: Felix Cheung > SparkSession R API > -- > >

[jira] [Resolved] (SPARK-15946) Wrap the conversion utils in Python

2016-06-17 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-15946. - Resolution: Fixed Fix Version/s: 2.0.0 > Wrap the conversion utils in Python >

[jira] [Resolved] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15129. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13285

[jira] [Updated] (SPARK-15892) Incorrectly merged AFTAggregator with zero total count

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15892: -- Fix Version/s: 2.0.0 > Incorrectly merged AFTAggregator with zero total count >

[jira] [Resolved] (SPARK-15892) Incorrectly merged AFTAggregator with zero total count

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15892. --- Resolution: Fixed Fix Version/s: (was: 2.0.0) 1.6.2 Issue

[jira] [Resolved] (SPARK-15603) Replace SQLContext with SparkSession in ML/MLLib

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15603. --- Resolution: Fixed Fix Version/s: 2.0.0 > Replace SQLContext with SparkSession in

[jira] [Resolved] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16033. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13747

[jira] [Commented] (SPARK-16028) Remove the need to pass in a SparkContext for spark.lapply

2016-06-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337475#comment-15337475 ] Felix Cheung commented on SPARK-16028: -- Fix ready as soon as the parent PR is merged. > Remove the

[jira] [Commented] (SPARK-16027) Fix SparkR session unit test

2016-06-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337476#comment-15337476 ] Felix Cheung commented on SPARK-16027: -- Fix ready as soon as parent PR is merged. > Fix SparkR

[jira] [Created] (SPARK-16039) Spark SQL - Number of rows inserted by Insert Sql

2016-06-17 Thread Prabhu Kasinathan (JIRA)
Prabhu Kasinathan created SPARK-16039: - Summary: Spark SQL - Number of rows inserted by Insert Sql Key: SPARK-16039 URL: https://issues.apache.org/jira/browse/SPARK-16039 Project: Spark

[jira] [Comment Edited] (SPARK-15340) Limit the size of the map used to cache JobConfs to void OOM

2016-06-17 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337421#comment-15337421 ] Zhongshuai Pei edited comment on SPARK-15340 at 6/18/16 1:47 AM: -

[jira] [Commented] (SPARK-16016) where i can find the code of Extreme Learning Machine(elm) on spark

2016-06-17 Thread yueyou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337423#comment-15337423 ] yueyou commented on SPARK-16016: you say nothing > where i can find the code of Extreme Learning

[jira] [Commented] (SPARK-15340) Limit the size of the map used to cache JobConfs to void OOM

2016-06-17 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337421#comment-15337421 ] Zhongshuai Pei commented on SPARK-15340: [~clockfly] 1. I run in the cluster mode on YARN and

[jira] [Commented] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Andrea Pasqua (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337389#comment-15337389 ] Andrea Pasqua commented on SPARK-16035: --- https://github.com/apache/spark/pull/13750 > The

[jira] [Issue Comment Deleted] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Andrea Pasqua (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Pasqua updated SPARK-16035: -- Comment: was deleted (was: https://github.com/apache/spark/pull/13750) > The SparseVector

[jira] [Assigned] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16035: Assignee: (was: Apache Spark) > The SparseVector parser fails checking for valid end

[jira] [Commented] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337388#comment-15337388 ] Apache Spark commented on SPARK-16035: -- User 'andreapasqua' has created a pull request for this

[jira] [Assigned] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16035: Assignee: Apache Spark > The SparseVector parser fails checking for valid end parenthesis

[jira] [Assigned] (SPARK-16034) Checks the partition columns when calling dataFrame.write.mode("append").saveAsTable

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16034: Assignee: (was: Apache Spark) > Checks the partition columns when calling >

[jira] [Assigned] (SPARK-16034) Checks the partition columns when calling dataFrame.write.mode("append").saveAsTable

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16034: Assignee: Apache Spark > Checks the partition columns when calling >

[jira] [Commented] (SPARK-16034) Checks the partition columns when calling dataFrame.write.mode("append").saveAsTable

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337385#comment-15337385 ] Apache Spark commented on SPARK-16034: -- User 'clockfly' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16031) Add debug-only socket source in Structured Streaming

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16031: Assignee: Matei Zaharia (was: Apache Spark) > Add debug-only socket source in Structured

[jira] [Commented] (SPARK-16031) Add debug-only socket source in Structured Streaming

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337373#comment-15337373 ] Apache Spark commented on SPARK-16031: -- User 'mateiz' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16031) Add debug-only socket source in Structured Streaming

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16031: Assignee: Apache Spark (was: Matei Zaharia) > Add debug-only socket source in Structured

[jira] [Created] (SPARK-16038) we can omit partition list when insert into hive table

2016-06-17 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-16038: --- Summary: we can omit partition list when insert into hive table Key: SPARK-16038 URL: https://issues.apache.org/jira/browse/SPARK-16038 Project: Spark Issue

[jira] [Created] (SPARK-16037) use by-position resolution when insert into hive table

2016-06-17 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-16037: --- Summary: use by-position resolution when insert into hive table Key: SPARK-16037 URL: https://issues.apache.org/jira/browse/SPARK-16037 Project: Spark Issue

[jira] [Updated] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Andrea Pasqua (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Pasqua updated SPARK-16035: -- Description: Running SparseVector.parse(' (4, [0,1 ],[ 4.0,5.0] ') will

[jira] [Updated] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Andrea Pasqua (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Pasqua updated SPARK-16035: -- Component/s: PySpark > The SparseVector parser fails checking for valid end parenthesis >

[jira] [Updated] (SPARK-16034) Checks the partition columns when calling dataFrame.write.mode("append").saveAsTable

2016-06-17 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Zhong updated SPARK-16034: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > Checks the partition columns when

[jira] [Created] (SPARK-16036) better error message if the number of columns in SELECT clause doesn't match the table schema

2016-06-17 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-16036: --- Summary: better error message if the number of columns in SELECT clause doesn't match the table schema Key: SPARK-16036 URL: https://issues.apache.org/jira/browse/SPARK-16036

[jira] [Updated] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Andrea Pasqua (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Pasqua updated SPARK-16035: -- Description: Running ``` SparseVector.parse(' (4, [0,1 ],[ 4.0,5.0] ') ``` > The SparseVector

[jira] [Created] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Andrea Pasqua (JIRA)
Andrea Pasqua created SPARK-16035: - Summary: The SparseVector parser fails checking for valid end parenthesis Key: SPARK-16035 URL: https://issues.apache.org/jira/browse/SPARK-16035 Project: Spark

[jira] [Created] (SPARK-16034) Checks the partition columns when calling dataFrame.write.mode("append").saveAsTable

2016-06-17 Thread Sean Zhong (JIRA)
Sean Zhong created SPARK-16034: -- Summary: Checks the partition columns when calling dataFrame.write.mode("append").saveAsTable Key: SPARK-16034 URL: https://issues.apache.org/jira/browse/SPARK-16034

[jira] [Assigned] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16033: Assignee: Cheng Lian (was: Apache Spark) > DataFrameWriter.partitionBy() can't be used

[jira] [Assigned] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16033: Assignee: Apache Spark (was: Cheng Lian) > DataFrameWriter.partitionBy() can't be used

[jira] [Assigned] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16030: Assignee: Apache Spark (was: Yin Huai) > Allow specifying static partitions in an INSERT

[jira] [Commented] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337341#comment-15337341 ] Apache Spark commented on SPARK-16030: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16030: Assignee: Yin Huai (was: Apache Spark) > Allow specifying static partitions in an INSERT

[jira] [Commented] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337343#comment-15337343 ] Apache Spark commented on SPARK-16033: -- User 'liancheng' has created a pull request for this issue:

[jira] [Commented] (SPARK-15997) Audit ml.feature Update documentation for ml feature transformers

2016-06-17 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337335#comment-15337335 ] Gayathri Murali commented on SPARK-15997: - https://github.com/apache/spark/pull/13745 - This is

[jira] [Updated] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16033: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > DataFrameWriter.partitionBy() can't

[jira] [Created] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16033: -- Summary: DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto() Key: SPARK-16033 URL: https://issues.apache.org/jira/browse/SPARK-16033

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16030: --- Assignee: Yin Huai > Allow specifying static partitions in an INSERT statement for data source >

[jira] [Created] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16032: -- Summary: Audit semantics of various insertion operations related to partitioned tables Key: SPARK-16032 URL: https://issues.apache.org/jira/browse/SPARK-16032 Project:

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16030: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > Allow specifying static partitions in

[jira] [Assigned] (SPARK-15997) Audit ml.feature Update documentation for ml feature transformers

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15997: Assignee: Gayathri Murali (was: Apache Spark) > Audit ml.feature Update documentation

[jira] [Commented] (SPARK-15997) Audit ml.feature Update documentation for ml feature transformers

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337285#comment-15337285 ] Apache Spark commented on SPARK-15997: -- User 'GayathriMurali' has created a pull request for this

[jira] [Assigned] (SPARK-15997) Audit ml.feature Update documentation for ml feature transformers

2016-06-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15997: Assignee: Apache Spark (was: Gayathri Murali) > Audit ml.feature Update documentation

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16030: - Priority: Critical (was: Major) > Allow specifying static partitions in an INSERT statement for data

[jira] [Updated] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15916: --- Description: A table from SQL server Northwind database was registered as a JDBC dataframe. A query

[jira] [Commented] (SPARK-15984) WARN message "o.a.h.y.s.resourcemanager.rmapp.RMAppImpl: The specific max attempts: 0 for application: 8 is invalid" when starting application on YARN

2016-06-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337278#comment-15337278 ] Saisai Shao commented on SPARK-15984: - Is there any problem? I guess you might set max app attempt to

[jira] [Updated] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15916: --- Assignee: Hyukjin Kwon > JDBC AND/OR operator push down does not respect lower OR operator

[jira] [Resolved] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15916. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13743

[jira] [Resolved] (SPARK-16005) Add `randomSplit` to SparkR

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-16005. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s:

[jira] [Commented] (SPARK-16031) Add debug-only socket source in Structured Streaming

2016-06-17 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337182#comment-15337182 ] Matei Zaharia commented on SPARK-16031: --- FYI I'll post a PR for this soon. > Add debug-only socket

[jira] [Created] (SPARK-16031) Add debug-only socket source in Structured Streaming

2016-06-17 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-16031: - Summary: Add debug-only socket source in Structured Streaming Key: SPARK-16031 URL: https://issues.apache.org/jira/browse/SPARK-16031 Project: Spark Issue

[jira] [Created] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Yin Huai (JIRA)
Yin Huai created SPARK-16030: Summary: Allow specifying static partitions in an INSERT statement for data source tables Key: SPARK-16030 URL: https://issues.apache.org/jira/browse/SPARK-16030 Project:

[jira] [Commented] (SPARK-16022) Input size is different when I use 1 or 3 nodes but the shufle size remains +- icual, do you know why?

2016-06-17 Thread jon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337171#comment-15337171 ] jon commented on SPARK-16022: - Hi, thanks for the correction. Where is that users first? > Input size is

[jira] [Resolved] (SPARK-16014) Rename optimizer rules to be more consistent

2016-06-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16014. - Resolution: Fixed > Rename optimizer rules to be more consistent >

[jira] [Resolved] (SPARK-16017) YarnClientSchedulerBackend now registers backends as IPs instead of Hostnames which causes all tasks to run with RACK_LOCAL locality.

2016-06-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-16017. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-16017) YarnClientSchedulerBackend now registers backends as IPs instead of Hostnames which causes all tasks to run with RACK_LOCAL locality.

2016-06-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16017: - Fix Version/s: 1.6.2 > YarnClientSchedulerBackend now registers backends as IPs instead of

[jira] [Updated] (SPARK-16017) YarnClientSchedulerBackend now registers backends as IPs instead of Hostnames which causes all tasks to run with RACK_LOCAL locality.

2016-06-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16017: - Target Version/s: 1.6.2, 2.0.0 (was: 2.0.0) > YarnClientSchedulerBackend now registers backends

[jira] [Created] (SPARK-16029) Deprecate dropTempTable in SparkR

2016-06-17 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-16029: - Summary: Deprecate dropTempTable in SparkR Key: SPARK-16029 URL: https://issues.apache.org/jira/browse/SPARK-16029 Project: Spark Issue

[jira] [Commented] (SPARK-16029) Deprecate dropTempTable in SparkR

2016-06-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337159#comment-15337159 ] Shivaram Venkataraman commented on SPARK-16029: --- cc [~liancheng] [~felixcheung] >

[jira] [Commented] (SPARK-16017) YarnClientSchedulerBackend now registers backends as IPs instead of Hostnames which causes all tasks to run with RACK_LOCAL locality.

2016-06-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337155#comment-15337155 ] Shixiong Zhu commented on SPARK-16017: -- [~tleftwich] Thanks! I'm merging it into 2.0 now! >

[jira] [Commented] (SPARK-16017) YarnClientSchedulerBackend now registers backends as IPs instead of Hostnames which causes all tasks to run with RACK_LOCAL locality.

2016-06-17 Thread Trystan Leftwich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337154#comment-15337154 ] Trystan Leftwich commented on SPARK-16017: -- [~zsxwing] I've tested your fix locally and it all

  1   2   3   >