[jira] [Created] (SPARK-18291) SparkR glm predict should output original label when family = "binomial"

Yanbo Liang created SPARK-18291: --- Summary: SparkR glm predict should output original label when family = "binomial" Key: SPARK-18291 URL: https://issues.apache.org/jira/browse/SPARK-18291 Project:

[jira] [Resolved] (SPARK-18276) Some ML training summaries are not copied when {{copy()}} is called.

[ https://issues.apache.org/jira/browse/SPARK-18276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-18276. - Resolution: Fixed Assignee: Seth Hendrickson Fix Version/s: 2.1.0 > Some ML

[jira] [Assigned] (SPARK-18286) Add Scala/Java/Python examples for MinHash and RandomProjection

[ https://issues.apache.org/jira/browse/SPARK-18286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18286: Assignee: (was: Apache Spark) > Add Scala/Java/Python examples for MinHash and

[jira] [Commented] (SPARK-18286) Add Scala/Java/Python examples for MinHash and RandomProjection

[ https://issues.apache.org/jira/browse/SPARK-18286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15641208#comment-15641208 ] Apache Spark commented on SPARK-18286: -- User 'jiayuezhangsc' has created a pull request for this

[jira] [Assigned] (SPARK-18286) Add Scala/Java/Python examples for MinHash and RandomProjection

[ https://issues.apache.org/jira/browse/SPARK-18286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18286: Assignee: Apache Spark > Add Scala/Java/Python examples for MinHash and RandomProjection

[jira] [Commented] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15640994#comment-15640994 ] KaiXu commented on SPARK-18289: --- this issue can be reproduced with the above parameters. >

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Created] (SPARK-18290) ucnjijetul

Emmett Keyser created SPARK-18290: - Summary: ucnjijetul Key: SPARK-18290 URL: https://issues.apache.org/jira/browse/SPARK-18290 Project: Spark Issue Type: Bug Reporter: Emmett

[jira] [Issue Comment Deleted] (SPARK-18279) ML programming guide should have R examples

[ https://issues.apache.org/jira/browse/SPARK-18279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-18279: -- Comment: was deleted (was: I can work on this) > ML programming guide should have R examples

[jira] [Commented] (SPARK-18279) ML programming guide should have R examples

[ https://issues.apache.org/jira/browse/SPARK-18279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15640669#comment-15640669 ] Sandeep Singh commented on SPARK-18279: --- I can work on this > ML programming guide should have R

[jira] [Commented] (SPARK-18268) ALS.run fail with UnsupportedOperationException if run on an empty ratings RDD

[ https://issues.apache.org/jira/browse/SPARK-18268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15640202#comment-15640202 ] Sean Owen commented on SPARK-18268: --- We don't generally assign issues. You can just work on it. >

[jira] [Updated] (SPARK-17964) Enable SparkR with Mesos client mode

[ https://issues.apache.org/jira/browse/SPARK-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17964: -- Assignee: Susan X. Huynh Priority: Minor (was: Major) > Enable SparkR with Mesos client mode >

[jira] [Resolved] (SPARK-17964) Enable SparkR with Mesos client mode

[ https://issues.apache.org/jira/browse/SPARK-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17964. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15700

[jira] [Comment Edited] (SPARK-18268) ALS.run fail with UnsupportedOperationException if run on an empty ratings RDD

[ https://issues.apache.org/jira/browse/SPARK-18268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639931#comment-15639931 ] Aditya edited comment on SPARK-18268 at 11/5/16 4:27 PM: - how do you assign an

[jira] [Commented] (SPARK-18268) ALS.run fail with UnsupportedOperationException if run on an empty ratings RDD

[ https://issues.apache.org/jira/browse/SPARK-18268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639931#comment-15639931 ] Aditya commented on SPARK-18268: how do you assign an issue? > ALS.run fail with

[jira] [Resolved] (SPARK-17849) grouping set throws NPE

[ https://issues.apache.org/jira/browse/SPARK-17849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17849. --- Resolution: Fixed Assignee: Yang Wang Fix Version/s: 2.1.0

[jira] [Assigned] (SPARK-18261) Add statistics to MemorySink for joining

[ https://issues.apache.org/jira/browse/SPARK-18261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18261: Assignee: (was: Apache Spark) > Add statistics to MemorySink for joining >

[jira] [Commented] (SPARK-18261) Add statistics to MemorySink for joining

[ https://issues.apache.org/jira/browse/SPARK-18261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639642#comment-15639642 ] Apache Spark commented on SPARK-18261: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18261) Add statistics to MemorySink for joining

[ https://issues.apache.org/jira/browse/SPARK-18261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18261: Assignee: Apache Spark > Add statistics to MemorySink for joining >

[jira] [Commented] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639638#comment-15639638 ] KaiXu commented on SPARK-18289: --- find a similar issue:https://issues.apache.org/jira/browse/SPARK-11293 >

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Description: We use BigBench to test the performance of Hive on Spark2.0 on Intel(R) Xeon(R) CPU E5-2699 v4(1

[jira] [Created] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

KaiXu created SPARK-18289: - Summary: spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk Key: SPARK-18289 URL: https://issues.apache.org/jira/browse/SPARK-18289

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Environment: CentOS7.2 kernel: 3.10.0-327.el7.x86_64 Hadoop2.7.1 Spark2.0.0 release version Hive2.1 with patch

[jira] [Updated] (SPARK-18289) spark.util.collection.ExternalSorter leak memory when task force spilling in-memory map to disk

[ https://issues.apache.org/jira/browse/SPARK-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXu updated SPARK-18289: -- Environment: CentOS7.2 kernel: 3.10.0-327.el7.x86_64 Hadoop2.7.1 Spark2.0.0 release version Hive2.1 with

[jira] [Updated] (SPARK-17693) Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field

[ https://issues.apache.org/jira/browse/SPARK-17693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17693: -- Fix Version/s: 2.0.3 > Fixed Insert Failure To Data Source Tables when the Schema has

[jira] [Updated] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17981: -- Fix Version/s: (was: 2.0.2) 2.0.3 > Incorrectly Set Nullability

[jira] [Updated] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17981: -- Fix Version/s: 2.0.2 > Incorrectly Set Nullability to False in FilterExec >

[jira] [Resolved] (SPARK-18287) Move hash expressions from misc.scala into hash.scala

[ https://issues.apache.org/jira/browse/SPARK-18287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18287. --- Resolution: Fixed Fix Version/s: 2.1.0 > Move hash expressions from

[jira] [Commented] (SPARK-18288) SparkLauncer 2.0.1 version working incosistently in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-18288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639182#comment-15639182 ] Sean Owen commented on SPARK-18288: --- (Please format the code; it's not clear what line you are talking

[jira] [Created] (SPARK-18288) SparkLauncer 2.0.1 version working incosistently in yarn-client mode

Elkhan Dadashov created SPARK-18288: --- Summary: SparkLauncer 2.0.1 version working incosistently in yarn-client mode Key: SPARK-18288 URL: https://issues.apache.org/jira/browse/SPARK-18288 Project:

[jira] [Commented] (SPARK-18192) Support all file formats in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639102#comment-15639102 ] Apache Spark commented on SPARK-18192: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-18258) Sinks need access to offset representation

[ https://issues.apache.org/jira/browse/SPARK-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15639002#comment-15639002 ] Liwei Lin commented on SPARK-18258: --- I think Cody refers to the {{OffsetSeq}} from

[jira] [Updated] (SPARK-17990) ALTER TABLE ... ADD PARTITION does not play nice with mixed-case partition column names

[ https://issues.apache.org/jira/browse/SPARK-17990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17990: Issue Type: Sub-task (was: Bug) Parent: SPARK-17861 > ALTER TABLE ... ADD PARTITION does

[jira] [Resolved] (SPARK-17183) put hive serde table schema to table properties like data source table

[ https://issues.apache.org/jira/browse/SPARK-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17183. - Resolution: Fixed Fix Version/s: 2.1.0 > put hive serde table schema to table properties

[jira] [Resolved] (SPARK-17983) Can't filter over mixed case parquet columns of converted Hive tables

[ https://issues.apache.org/jira/browse/SPARK-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17983. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.1.0 > Can't filter over

[jira] [Resolved] (SPARK-18101) ExternalCatalogSuite should test with mixed case fields

[ https://issues.apache.org/jira/browse/SPARK-18101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18101. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-18258) Sinks need access to offset representation

[ https://issues.apache.org/jira/browse/SPARK-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638887#comment-15638887 ] Reynold Xin commented on SPARK-18258: - What is OffsetSeq? > Sinks need access to offset

[jira] [Updated] (SPARK-18269) NumberFormatException when reading csv for a nullable column

[ https://issues.apache.org/jira/browse/SPARK-18269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18269: Description: Having a schema with a nullable column thrown an java.lang.NumberFormatException:

[jira] [Comment Edited] (SPARK-10912) Improve Spark metrics executor.filesystem

[ https://issues.apache.org/jira/browse/SPARK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638836#comment-15638836 ] Yongjia Wang edited comment on SPARK-10912 at 11/5/16 7:09 AM: --- s3a and

[jira] [Resolved] (SPARK-18260) from_json can throw a better exception when it can't find the column or be nullSafe

[ https://issues.apache.org/jira/browse/SPARK-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18260. - Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 2.1.0 > from_json can

[jira] [Reopened] (SPARK-10912) Improve Spark metrics executor.filesystem

[ https://issues.apache.org/jira/browse/SPARK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjia Wang reopened SPARK-10912: -- > Improve Spark metrics executor.filesystem > - > >

[jira] [Commented] (SPARK-10912) Improve Spark metrics executor.filesystem

[ https://issues.apache.org/jira/browse/SPARK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638836#comment-15638836 ] Yongjia Wang commented on SPARK-10912: -- s3a and hdfs are different "schemes" in Spark's

[jira] [Assigned] (SPARK-18287) Move hash expressions from misc.scala into hash.scala

[ https://issues.apache.org/jira/browse/SPARK-18287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18287: Assignee: Apache Spark (was: Reynold Xin) > Move hash expressions from misc.scala into

[jira] [Commented] (SPARK-18287) Move hash expressions from misc.scala into hash.scala

[ https://issues.apache.org/jira/browse/SPARK-18287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638828#comment-15638828 ] Apache Spark commented on SPARK-18287: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18287) Move hash expressions from misc.scala into hash.scala

[ https://issues.apache.org/jira/browse/SPARK-18287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18287: Assignee: Reynold Xin (was: Apache Spark) > Move hash expressions from misc.scala into

[jira] [Created] (SPARK-18287) Move hash expressions from misc.scala into hash.scala

Reynold Xin created SPARK-18287: --- Summary: Move hash expressions from misc.scala into hash.scala Key: SPARK-18287 URL: https://issues.apache.org/jira/browse/SPARK-18287 Project: Spark Issue

[jira] [Commented] (SPARK-18286) Add Scala/Java/Python examples for MinHash and RandomProjection

[ https://issues.apache.org/jira/browse/SPARK-18286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638806#comment-15638806 ] Weichen Xu commented on SPARK-18286: I will work on it, thanks~ > Add Scala/Java/Python examples for

[jira] [Created] (SPARK-18286) Add Scala/Java/Python examples for MinHash and RandomProjection

Yanbo Liang created SPARK-18286: --- Summary: Add Scala/Java/Python examples for MinHash and RandomProjection Key: SPARK-18286 URL: https://issues.apache.org/jira/browse/SPARK-18286 Project: Spark

[jira] [Commented] (SPARK-17693) Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field

[ https://issues.apache.org/jira/browse/SPARK-17693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638786#comment-15638786 ] Apache Spark commented on SPARK-17693: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Commented] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638756#comment-15638756 ] Apache Spark commented on SPARK-17981: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18284) Scheme of DataFrame generated from RDD is diffrent between master and 2.0

[ https://issues.apache.org/jira/browse/SPARK-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18284: Assignee: (was: Apache Spark) > Scheme of DataFrame generated from RDD is diffrent

[jira] [Commented] (SPARK-17957) Calling outer join and na.fill(0) and then inner join will miss rows

[ https://issues.apache.org/jira/browse/SPARK-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638757#comment-15638757 ] Apache Spark commented on SPARK-17957: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18284) Scheme of DataFrame generated from RDD is diffrent between master and 2.0

[ https://issues.apache.org/jira/browse/SPARK-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18284: Assignee: Apache Spark > Scheme of DataFrame generated from RDD is diffrent between

[jira] [Commented] (SPARK-18284) Scheme of DataFrame generated from RDD is diffrent between master and 2.0

[ https://issues.apache.org/jira/browse/SPARK-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15638754#comment-15638754 ] Apache Spark commented on SPARK-18284: -- User 'kiszk' has created a pull request for this issue:

[jira] [Updated] (SPARK-18284) Scheme of DataFrame generated from RDD is diffrent between master and 2.0

[ https://issues.apache.org/jira/browse/SPARK-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-18284: - Description: When the following program is executed, a schema of dataframe is different

[jira] [Updated] (SPARK-18284) Scheme of DataFrame generated from RDD is diffrent between master and 2.0

[ https://issues.apache.org/jira/browse/SPARK-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-18284: - Affects Version/s: 2.1.0 > Scheme of DataFrame generated from RDD is diffrent between