[jira] [Commented] (SPARK-2875) SchemaRDD#javaToPython failed on come cases

[ https://issues.apache.org/jira/browse/SPARK-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087338#comment-14087338 ] Apache Spark commented on SPARK-2875: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-1022) Add unit tests for kafka streaming

[ https://issues.apache.org/jira/browse/SPARK-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087333#comment-14087333 ] Patrick Wendell commented on SPARK-1022: There was a follow up to this issue: http

[jira] [Resolved] (SPARK-1022) Add unit tests for kafka streaming

[ https://issues.apache.org/jira/browse/SPARK-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1022. Resolution: Fixed > Add unit tests for kafka streaming > --

[jira] [Resolved] (SPARK-2294) TaskSchedulerImpl and TaskSetManager do not properly prioritize which tasks get assigned to an executor

[ https://issues.apache.org/jira/browse/SPARK-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2294. -- Resolution: Fixed Fix Version/s: 1.1.0 > TaskSchedulerImpl and TaskSetManager do not pro

[jira] [Created] (SPARK-2875) SchemaRDD#javaToPython failed on come cases

Yin Huai created SPARK-2875: --- Summary: SchemaRDD#javaToPython failed on come cases Key: SPARK-2875 URL: https://issues.apache.org/jira/browse/SPARK-2875 Project: Spark Issue Type: Bug Com

[jira] [Updated] (SPARK-2806) update json4s-jackson to version 3.2.10

[ https://issues.apache.org/jira/browse/SPARK-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2806: --- Assignee: Anand Avati > update json4s-jackson to version 3.2.10 > ---

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087235#comment-14087235 ] Carlos Fuertes commented on SPARK-2016: --- I have done some very simple benchmarks com

[jira] [Commented] (SPARK-2874) Spark SQL related scripts don't show complete usage message

[ https://issues.apache.org/jira/browse/SPARK-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087229#comment-14087229 ] Apache Spark commented on SPARK-2874: - User 'liancheng' has created a pull request for

[jira] [Created] (SPARK-2874) Spark SQL related scripts don't show complete usage message

Cheng Lian created SPARK-2874: - Summary: Spark SQL related scripts don't show complete usage message Key: SPARK-2874 URL: https://issues.apache.org/jira/browse/SPARK-2874 Project: Spark Issue Ty

[jira] [Updated] (SPARK-2678) `Spark-submit` overrides user application options

[ https://issues.apache.org/jira/browse/SPARK-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2678: -- Target Version/s: 1.2.0 (was: 1.1.0) > `Spark-submit` overrides user application options > ---

[jira] [Created] (SPARK-2873) OOM happens when group by and join operation with big data

guowei created SPARK-2873: - Summary: OOM happens when group by and join operation with big data Key: SPARK-2873 URL: https://issues.apache.org/jira/browse/SPARK-2873 Project: Spark Issue Type: Impr

[jira] [Resolved] (SPARK-2866) ORDER BY attributes must appear in SELECT clause

[ https://issues.apache.org/jira/browse/SPARK-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2866. - Resolution: Fixed Fix Version/s: 1.2.0 > ORDER BY attributes must appear in SELECT

[jira] [Commented] (SPARK-2787) Make sort-based shuffle write files directly when there is no sorting / aggregation and # of partitions is small

[ https://issues.apache.org/jira/browse/SPARK-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087197#comment-14087197 ] Apache Spark commented on SPARK-2787: - User 'mateiz' has created a pull request for th

[jira] [Commented] (SPARK-2848) Shade Guava in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087179#comment-14087179 ] Marcelo Vanzin commented on SPARK-2848: --- Nevermind the question, I got code mostly w

[jira] [Commented] (SPARK-2828) Check MLlib v1.1 API consistency

[ https://issues.apache.org/jira/browse/SPARK-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087160#comment-14087160 ] Apache Spark commented on SPARK-2828: - User 'jkbradley' has created a pull request for

[jira] [Updated] (SPARK-1312) Batch should read based on the batch interval provided in the StreamingContext

[ https://issues.apache.org/jira/browse/SPARK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-1312: - Priority: Critical (was: Minor) > Batch should read based on the batch interval provided in the

[jira] [Commented] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087120#comment-14087120 ] Zhihui commented on SPARK-2872: --- PR https://github.com/apache/spark/pull/1684 > Fix conflic

[jira] [Created] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

Zhihui created SPARK-2872: - Summary: Fix conflict between code and doc in YarnClientSchedulerBackend Key: SPARK-2872 URL: https://issues.apache.org/jira/browse/SPARK-2872 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2636) no where to get job identifier while submit spark job through spark API

[ https://issues.apache.org/jira/browse/SPARK-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087102#comment-14087102 ] Chengxiang Li commented on SPARK-2636: -- {quote} There are two ways I think. One is fo

[jira] [Commented] (SPARK-1022) Add unit tests for kafka streaming

[ https://issues.apache.org/jira/browse/SPARK-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087092#comment-14087092 ] Apache Spark commented on SPARK-1022: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-2419) Misc updates to streaming programming guide

[ https://issues.apache.org/jira/browse/SPARK-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087084#comment-14087084 ] Nicholas Chammas commented on SPARK-2419: - {quote} Twitter4j version {quote} Yes!

[jira] [Commented] (SPARK-2866) ORDER BY attributes must appear in SELECT clause

[ https://issues.apache.org/jira/browse/SPARK-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087077#comment-14087077 ] Apache Spark commented on SPARK-2866: - User 'marmbrus' has created a pull request for

[jira] [Updated] (SPARK-2419) Misc updates to streaming programming guide

[ https://issues.apache.org/jira/browse/SPARK-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2419: - Description: This JIRA collects together a number of small issues that should be added to the st

[jira] [Resolved] (SPARK-2869) Potential leak of Jdbc Connection and PreparedStatement in case of error in JdbcRDD

[ https://issues.apache.org/jira/browse/SPARK-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2869. Resolution: Fixed Fix Version/s: 1.0.3 1.1.0 Assignee: Stephen Bo

[jira] [Updated] (SPARK-2650) Wrong initial sizes for in-memory column buffers

[ https://issues.apache.org/jira/browse/SPARK-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2650: -- Target Version/s: 1.2.0 (was: 1.1.0) > Wrong initial sizes for in-memory column buffers >

[jira] [Created] (SPARK-2871) Missing API in PySpark

Davies Liu created SPARK-2871: - Summary: Missing API in PySpark Key: SPARK-2871 URL: https://issues.apache.org/jira/browse/SPARK-2871 Project: Spark Issue Type: Improvement Reporter:

[jira] [Resolved] (SPARK-2550) Support regularization and intercept in pyspark's linear methods

[ https://issues.apache.org/jira/browse/SPARK-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2550. -- Resolution: Fixed Issue resolved by pull request 1775 [https://github.com/apache/spark/pull/177

[jira] [Resolved] (SPARK-2864) fix random seed in Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-2864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2864. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1790 [https://gith

[jira] [Created] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

Nicholas Chammas created SPARK-2870: --- Summary: Thorough schema inference directly on RDDs of Python dictionaries Key: SPARK-2870 URL: https://issues.apache.org/jira/browse/SPARK-2870 Project: Spark

[jira] [Commented] (SPARK-2854) Finalize _acceptable_types in pyspark.sql

[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086877#comment-14086877 ] Apache Spark commented on SPARK-2854: - User 'yhuai' has created a pull request for thi

[jira] [Commented] (SPARK-2869) Potential leak of Jdbc Connection and PreparedStatement in case of error in JdbcRDD

[ https://issues.apache.org/jira/browse/SPARK-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086876#comment-14086876 ] Apache Spark commented on SPARK-2869: - User 'javadba' has created a pull request for t

[jira] [Commented] (SPARK-2864) fix random seed in Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-2864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086874#comment-14086874 ] Apache Spark commented on SPARK-2864: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086830#comment-14086830 ] Andrew Ash commented on SPARK-2546: --- I don't have a reliable repro that's in a unit test

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086817#comment-14086817 ] Marcelo Vanzin commented on SPARK-1537: --- Currently busy with other more urgent tasks

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086808#comment-14086808 ] Zhan Zhang commented on SPARK-1537: --- Do you mind sharing your thoughts, design document

[jira] [Updated] (SPARK-2844) Existing JVM Hive Context not correctly used in Python Hive Context

[ https://issues.apache.org/jira/browse/SPARK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2844: Priority: Major (was: Minor) Target Version/s: 1.1.0 > Existing JVM Hive Conte

[jira] [Updated] (SPARK-2844) Existing JVM Hive Context not correctly used in Python Hive Context

[ https://issues.apache.org/jira/browse/SPARK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2844: Assignee: Ahir Reddy > Existing JVM Hive Context not correctly used in Python Hive Context

[jira] [Updated] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive functions

[ https://issues.apache.org/jira/browse/SPARK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2863: Target Version/s: 1.2.0 > Emulate Hive type coercion in native reimplementations of Hive fu

[jira] [Updated] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive functions

[ https://issues.apache.org/jira/browse/SPARK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2863: Assignee: William Benton > Emulate Hive type coercion in native reimplementations of Hive f

[jira] [Resolved] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1680. -- Resolution: Fixed > Clean up use of setExecutorEnvs in SparkConf > ---

[jira] [Comment Edited] (SPARK-2854) Finalize _acceptable_types in pyspark.sql

[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086560#comment-14086560 ] Yin Huai edited comment on SPARK-2854 at 8/5/14 8:31 PM: - Since we

[jira] [Assigned] (SPARK-2717) BasicBlockFetchIterator#next should log when it gets stuck

[ https://issues.apache.org/jira/browse/SPARK-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2717: - Assignee: Josh Rosen > BasicBlockFetchIterator#next should log when it gets stuck > -

[jira] [Assigned] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2677: - Assignee: Josh Rosen > BasicBlockFetchIterator#next can wait forever > --

[jira] [Created] (SPARK-2869) Potential leak of Jdbc Connection and PreparedStatement in case of error in JdbcRDD

Stephen Boesch created SPARK-2869: - Summary: Potential leak of Jdbc Connection and PreparedStatement in case of error in JdbcRDD Key: SPARK-2869 URL: https://issues.apache.org/jira/browse/SPARK-2869 P

[jira] [Commented] (SPARK-1856) Standardize MLlib interfaces

[ https://issues.apache.org/jira/browse/SPARK-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086697#comment-14086697 ] Xiangrui Meng commented on SPARK-1856: -- Yes, MLI and MLbase are research projects at

[jira] [Assigned] (SPARK-2583) ConnectionManager cannot distinguish whether error occurred or not

[ https://issues.apache.org/jira/browse/SPARK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2583: - Assignee: Josh Rosen (was: Kousuke Saruta) > ConnectionManager cannot distinguish whether error

[jira] [Created] (SPARK-2868) Support named accumulators in Python

Patrick Wendell created SPARK-2868: -- Summary: Support named accumulators in Python Key: SPARK-2868 URL: https://issues.apache.org/jira/browse/SPARK-2868 Project: Spark Issue Type: New Featur

[jira] [Resolved] (SPARK-2380) Support displaying accumulator contents in the web UI

[ https://issues.apache.org/jira/browse/SPARK-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2380. Resolution: Fixed Resolved by: https://github.com/apache/spark/pull/1309 > Support display

[jira] [Updated] (SPARK-2380) Support displaying accumulator contents in the web UI

[ https://issues.apache.org/jira/browse/SPARK-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2380: --- Fix Version/s: 1.1.0 > Support displaying accumulator contents in the web UI > --

[jira] [Updated] (SPARK-2699) Improve compatibility with parquet file/table

[ https://issues.apache.org/jira/browse/SPARK-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2699: Target Version/s: 1.1.0 > Improve compatibility with parquet file/table > -

[jira] [Updated] (SPARK-2699) Improve compatibility with parquet file/table

[ https://issues.apache.org/jira/browse/SPARK-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2699: Target Version/s: 1.2.0 (was: 1.1.0) > Improve compatibility with parquet file/table > ---

[jira] [Commented] (SPARK-2848) Shade Guava in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086670#comment-14086670 ] Marcelo Vanzin commented on SPARK-2848: --- Question for others ([~pwendell], [~sowen],

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086663#comment-14086663 ] Marcelo Vanzin commented on SPARK-1537: --- I have a prototype ready. But I'm still inv

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086660#comment-14086660 ] Josh Rosen commented on SPARK-2546: --- Hi Andrew, Do you have any way to reliably reprodu

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086657#comment-14086657 ] Zhan Zhang commented on SPARK-1537: --- I am also interested in it and trying to integrate

[jira] [Commented] (SPARK-1834) NoSuchMethodError when invoking JavaPairRDD.reduce() in Java

[ https://issues.apache.org/jira/browse/SPARK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086649#comment-14086649 ] Sean Owen commented on SPARK-1834: -- Ah, you're right: https://github.com/apache/spark/co

[jira] [Commented] (SPARK-1834) NoSuchMethodError when invoking JavaPairRDD.reduce() in Java

[ https://issues.apache.org/jira/browse/SPARK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086643#comment-14086643 ] John Snodgrass commented on SPARK-1834: --- [~srowen] In version 0.9.1, the map method

[jira] [Updated] (SPARK-1977) mutable.BitSet in ALS not serializable with KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-1977: - Fix Version/s: (was: 1.0.1) 1.0.2 > mutable.BitSet in ALS not serializable

[jira] [Updated] (SPARK-2152) the error of comput rightNodeAgg about Decision tree algorithm in Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-2152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2152: - Fix Version/s: (was: 1.0.1) 1.0.2 > the error of comput rightNodeAgg about

[jira] [Created] (SPARK-2867) saveAsHadoopFile() in PairRDDFunction.scala should allow use other OutputCommiter class

Joseph Su created SPARK-2867: Summary: saveAsHadoopFile() in PairRDDFunction.scala should allow use other OutputCommiter class Key: SPARK-2867 URL: https://issues.apache.org/jira/browse/SPARK-2867 Project

[jira] [Updated] (SPARK-2534) Avoid pulling in the entire RDD or PairRDDFunctions in various operators

[ https://issues.apache.org/jira/browse/SPARK-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2534: - Component/s: Spark Core > Avoid pulling in the entire RDD or PairRDDFunctions in various operator

[jira] [Resolved] (SPARK-2859) Update url of Kryo project in related docs

[ https://issues.apache.org/jira/browse/SPARK-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2859. Resolution: Fixed Fix Version/s: 1.1.0 1.0.3 Issue resolved by pu

[jira] [Updated] (SPARK-2859) Update url of Kryo project in related docs

[ https://issues.apache.org/jira/browse/SPARK-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2859: --- Assignee: Guancheng Chen > Update url of Kryo project in related docs > -

[jira] [Created] (SPARK-2866) ORDER BY attributes must appear in SELECT clause

Michael Armbrust created SPARK-2866: --- Summary: ORDER BY attributes must appear in SELECT clause Key: SPARK-2866 URL: https://issues.apache.org/jira/browse/SPARK-2866 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-2860) Resolving CASE WHEN throws None.get exception

[ https://issues.apache.org/jira/browse/SPARK-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2860. - Resolution: Fixed Fix Version/s: 1.1.0 > Resolving CASE WHEN throws None.get excep

[jira] [Updated] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

[ https://issues.apache.org/jira/browse/SPARK-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zongheng Yang updated SPARK-2865: - Description: In the application I tested, most of the tasks out of 128 tasks could finish, but s

[jira] [Updated] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

[ https://issues.apache.org/jira/browse/SPARK-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zongheng Yang updated SPARK-2865: - Description: In the application I tested, most of the tasks out of 128 tasks could finish, but s

[jira] [Created] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

Zongheng Yang created SPARK-2865: Summary: Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish Key: SPARK-2865 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Commented] (SPARK-2854) Finalize _acceptable_types in pyspark.sql

[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086560#comment-14086560 ] Yin Huai commented on SPARK-2854: - Since we have already do conversions for ByteType and S

[jira] [Resolved] (SPARK-1890) add modify acls to the web UI for the "kill" button

[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1890. -- Resolution: Fixed > add modify acls to the web UI for the "kill" button > -

[jira] [Resolved] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1528. -- Resolution: Fixed Fix Version/s: 1.1.0 > Spark on Yarn: Add option for user to specify a

[jira] [Closed] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-2622. Resolution: Fixed > Add Jenkins build numbers to SparkQA messages > ---

[jira] [Commented] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086513#comment-14086513 ] Xiangrui Meng commented on SPARK-2622: -- The build number is included in the SparkQA m

[jira] [Created] (SPARK-2864) fix random seed in Word2Vec

Xiangrui Meng created SPARK-2864: Summary: fix random seed in Word2Vec Key: SPARK-2864 URL: https://issues.apache.org/jira/browse/SPARK-2864 Project: Spark Issue Type: Bug Component

[jira] [Commented] (SPARK-2636) no where to get job identifier while submit spark job through spark API

[ https://issues.apache.org/jira/browse/SPARK-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086501#comment-14086501 ] Reynold Xin commented on SPARK-2636: Yes - I meant job id. > no where to get job iden

[jira] [Commented] (SPARK-2636) no where to get job identifier while submit spark job through spark API

[ https://issues.apache.org/jira/browse/SPARK-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086494#comment-14086494 ] Marcelo Vanzin commented on SPARK-2636: --- (BTW, just checked SPARK-2321, so if you re

[jira] [Commented] (SPARK-2636) no where to get job identifier while submit spark job through spark API

[ https://issues.apache.org/jira/browse/SPARK-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086475#comment-14086475 ] Marcelo Vanzin commented on SPARK-2636: --- The PR I posted exposes the application ID

[jira] [Closed] (SPARK-2454) Separate driver spark home from executor spark home

[ https://issues.apache.org/jira/browse/SPARK-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2454. Resolution: Fixed > Separate driver spark home from executor spark home > -

[jira] [Commented] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086342#comment-14086342 ] Apache Spark commented on SPARK-2167: - User 'witgo' has created a pull request for thi

[jira] [Comment Edited] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049033#comment-14049033 ] Guoqiang Li edited comment on SPARK-2167 at 8/5/14 3:00 PM: PR

[jira] [Commented] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086338#comment-14086338 ] Chandan Kumar commented on SPARK-2862: -- [~srowen] My colleague [~Shiti] is working on

[jira] [Updated] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive functions

[ https://issues.apache.org/jira/browse/SPARK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Benton updated SPARK-2863: -- Summary: Emulate Hive type coercion in native reimplementations of Hive functions (was: Emulat

[jira] [Created] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive UDFs

William Benton created SPARK-2863: - Summary: Emulate Hive type coercion in native reimplementations of Hive UDFs Key: SPARK-2863 URL: https://issues.apache.org/jira/browse/SPARK-2863 Project: Spark

[jira] [Commented] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive functions

[ https://issues.apache.org/jira/browse/SPARK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086298#comment-14086298 ] William Benton commented on SPARK-2863: --- I'm working on this; can someone assign it

[jira] [Updated] (SPARK-2863) Emulate Hive type coercion in native reimplementations of Hive functions

[ https://issues.apache.org/jira/browse/SPARK-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Benton updated SPARK-2863: -- Description: Native reimplementations of Hive functions no longer have the same type-coercion

[jira] [Updated] (SPARK-2861) Doc comment of DoubleRDDFunctions.histogram is incorrect

[ https://issues.apache.org/jira/browse/SPARK-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan Kumar updated SPARK-2861: - Description: The documentation comment of histogram method of DoubleRDDFunctions class in source

[jira] [Updated] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-2862: --- Affects Version/s: 1.0.1 > DoubleRDDFunctions.histogram() throws exception for some inputs >

[jira] [Commented] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086182#comment-14086182 ] Apache Spark commented on SPARK-2862: - User 'nrchandan' has created a pull request for

[jira] [Commented] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086181#comment-14086181 ] Sean Owen commented on SPARK-2862: -- It looks like a Scala bug, which I see you've already

[jira] [Reopened] (SPARK-2750) Add Https support for Web UI

[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic reopened SPARK-2750: > Add Https support for Web UI > > > Key: SPARK-27

[jira] [Commented] (SPARK-2408) RDD.map(func) dependencies issue after checkpoint & count

[ https://issues.apache.org/jira/browse/SPARK-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086150#comment-14086150 ] Ravindra Pesala commented on SPARK-2408: Yes, Daniel is right, it is the problem o

[jira] [Updated] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

[ https://issues.apache.org/jira/browse/SPARK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan Kumar updated SPARK-2862: - Description: histogram method call throws the below stack trace when the choice of bucketCount p

[jira] [Created] (SPARK-2862) DoubleRDDFunctions.histogram() throws exception for some inputs

Chandan Kumar created SPARK-2862: Summary: DoubleRDDFunctions.histogram() throws exception for some inputs Key: SPARK-2862 URL: https://issues.apache.org/jira/browse/SPARK-2862 Project: Spark

[jira] [Commented] (SPARK-2861) Doc comment of DoubleRDDFunctions.histogram is incorrect

[ https://issues.apache.org/jira/browse/SPARK-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086103#comment-14086103 ] Apache Spark commented on SPARK-2861: - User 'nrchandan' has created a pull request for

[jira] [Commented] (SPARK-1834) NoSuchMethodError when invoking JavaPairRDD.reduce() in Java

[ https://issues.apache.org/jira/browse/SPARK-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086085#comment-14086085 ] Sean Owen commented on SPARK-1834: -- [~franklynDsouza] There is a reduce method available

[jira] [Created] (SPARK-2861) Doc comment of DoubleRDDFunctions.histogram is incorrect

Chandan Kumar created SPARK-2861: Summary: Doc comment of DoubleRDDFunctions.histogram is incorrect Key: SPARK-2861 URL: https://issues.apache.org/jira/browse/SPARK-2861 Project: Spark Issue

[jira] [Commented] (SPARK-2858) Default log4j file no longer seems to work

[ https://issues.apache.org/jira/browse/SPARK-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086066#comment-14086066 ] Sean Owen commented on SPARK-2858: -- Where is this file and how do you run things? my read

[jira] [Resolved] (SPARK-2750) Add Https support for Web UI

[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic resolved SPARK-2750. Resolution: Implemented Fix Version/s: 1.0.2 Target Version/s: 1.0.2 Alrea

[jira] [Commented] (SPARK-2860) Resolving CASE WHEN throws None.get exception

[ https://issues.apache.org/jira/browse/SPARK-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086036#comment-14086036 ] Apache Spark commented on SPARK-2860: - User 'marmbrus' has created a pull request for

[jira] [Created] (SPARK-2860) Resolving CASE WHEN throws None.get exception

Michael Armbrust created SPARK-2860: --- Summary: Resolving CASE WHEN throws None.get exception Key: SPARK-2860 URL: https://issues.apache.org/jira/browse/SPARK-2860 Project: Spark Issue Type:

  1   2   >