[jira] [Updated] (SPARK-2953) Allow using short names for io compression codecs

[ https://issues.apache.org/jira/browse/SPARK-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2953: --- Issue Type: Improvement (was: Bug) Allow using short names for io compression codecs

[jira] [Updated] (SPARK-2953) Allow using short names for io compression codecs

[ https://issues.apache.org/jira/browse/SPARK-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2953: --- Component/s: Spark Core Allow using short names for io compression codecs

[jira] [Updated] (SPARK-2953) Allow using short names for io compression codecs

[ https://issues.apache.org/jira/browse/SPARK-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2953: --- Description: Instead of requiring org.apache.spark.io.LZ4CompressionCodec, it is easier for users if

[jira] [Commented] (SPARK-2953) Allow using short names for io compression codecs

[ https://issues.apache.org/jira/browse/SPARK-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092034#comment-14092034 ] Apache Spark commented on SPARK-2953: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-2947) DAGScheduler scheduling infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2947: --- Summary: DAGScheduler scheduling infinite loop (was: DAGScheduler scheduling dead cycle)

[jira] [Created] (SPARK-2954) PySpark MLlib serialization tests fail on Python 2.6

Josh Rosen created SPARK-2954: - Summary: PySpark MLlib serialization tests fail on Python 2.6 Key: SPARK-2954 URL: https://issues.apache.org/jira/browse/SPARK-2954 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-2954) PySpark MLlib serialization tests fail on Python 2.6

[ https://issues.apache.org/jira/browse/SPARK-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2954: - Assignee: Josh Rosen PySpark MLlib serialization tests fail on Python 2.6

[jira] [Updated] (SPARK-2954) PySpark MLlib serialization tests fail on Python 2.6

[ https://issues.apache.org/jira/browse/SPARK-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2954: -- Component/s: PySpark PySpark MLlib serialization tests fail on Python 2.6

[jira] [Commented] (SPARK-2954) PySpark MLlib serialization tests fail on Python 2.6

[ https://issues.apache.org/jira/browse/SPARK-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092044#comment-14092044 ] Apache Spark commented on SPARK-2954: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-2948) PySpark doesn't work on Python 2.6

[ https://issues.apache.org/jira/browse/SPARK-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092045#comment-14092045 ] Apache Spark commented on SPARK-2948: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-2948) PySpark doesn't work on Python 2.6

[ https://issues.apache.org/jira/browse/SPARK-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2948: - Assignee: Josh Rosen PySpark doesn't work on Python 2.6 --

[jira] [Commented] (SPARK-2101) Python unit tests fail on Python 2.6 because of lack of unittest.skipIf()

[ https://issues.apache.org/jira/browse/SPARK-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092048#comment-14092048 ] Apache Spark commented on SPARK-2101: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-2910) Test with Python 2.6 on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092046#comment-14092046 ] Apache Spark commented on SPARK-2910: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-2910) Test with Python 2.6 on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-2910: - Assignee: Josh Rosen Test with Python 2.6 on Jenkins ---

[jira] [Commented] (SPARK-2945) Allow specifying num of executors in the context configuration

[ https://issues.apache.org/jira/browse/SPARK-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092058#comment-14092058 ] Shay Rojansky commented on SPARK-2945: -- I just did a quick test on Spark 1.0.2, and

[jira] [Updated] (SPARK-2947) DAGScheduler resubmit the stage into an infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2947: --- Summary: DAGScheduler resubmit the stage into an infinite loop (was: DAGScheduler resubmit the task

[jira] [Updated] (SPARK-2947) DAGScheduler resubmit the task into an infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2947: --- Summary: DAGScheduler resubmit the task into an infinite loop (was: DAGScheduler scheduling

[jira] [Commented] (SPARK-2947) DAGScheduler resubmit the stage into an infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092100#comment-14092100 ] Apache Spark commented on SPARK-2947: - User 'witgo' has created a pull request for

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-1297: -- Attachment: spark-1297-v2.txt Tentative patch adds hbase-hadoop2 profile. Upgrade HBase dependency to 0.98.0

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-1297: -- Attachment: (was: spark-1297-v2.txt) Upgrade HBase dependency to 0.98.0

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-1297: -- Attachment: spark-1297-v2.txt Upgrade HBase dependency to 0.98.0 --

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092115#comment-14092115 ] Sean Owen commented on SPARK-1297: -- This doesn't work with Hadoop 1 though. It also

[jira] [Commented] (SPARK-2944) sc.makeRDD doesn't distribute partitions evenly

[ https://issues.apache.org/jira/browse/SPARK-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092135#comment-14092135 ] Xiangrui Meng commented on SPARK-2944: -- Found that this behavior is not

[jira] [Updated] (SPARK-2950) Add GC time and Shuffle Write time to JobLogger output

[ https://issues.apache.org/jira/browse/SPARK-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-2950: - Fix Version/s: 1.2.0 Add GC time and Shuffle Write time to JobLogger output

[jira] [Resolved] (SPARK-2950) Add GC time and Shuffle Write time to JobLogger output

[ https://issues.apache.org/jira/browse/SPARK-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-2950. -- Resolution: Fixed Add GC time and Shuffle Write time to JobLogger output

[jira] [Resolved] (SPARK-2898) Failed to connect to daemon

[ https://issues.apache.org/jira/browse/SPARK-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2898. --- Resolution: Fixed Fix Version/s: 1.1.0 Failed to connect to daemon

[jira] [Created] (SPARK-2955) Test code fails to compile with mvn compile without install

Sean Owen created SPARK-2955: Summary: Test code fails to compile with mvn compile without install Key: SPARK-2955 URL: https://issues.apache.org/jira/browse/SPARK-2955 Project: Spark Issue

[jira] [Commented] (SPARK-2955) Test code fails to compile with mvn compile without install

[ https://issues.apache.org/jira/browse/SPARK-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092180#comment-14092180 ] Apache Spark commented on SPARK-2955: - User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-2650) Caching tables larger than memory causes OOMs

[ https://issues.apache.org/jira/browse/SPARK-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2650: Summary: Caching tables larger than memory causes OOMs (was: Wrong initial sizes for

[jira] [Commented] (SPARK-2650) Caching tables larger than memory causes OOMs

[ https://issues.apache.org/jira/browse/SPARK-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092209#comment-14092209 ] Apache Spark commented on SPARK-2650: - User 'marmbrus' has created a pull request for

[jira] [Resolved] (SPARK-2937) Separate out sampleByKeyExact in PairRDDFunctions as its own API

[ https://issues.apache.org/jira/browse/SPARK-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2937. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1866

[jira] [Updated] (SPARK-2956) Support transferring large blocks in Netty network module

[ https://issues.apache.org/jira/browse/SPARK-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2956: --- Summary: Support transferring large blocks in Netty network module (was: Support transferring

[jira] [Created] (SPARK-2956) Support transferring blocks larger than MTU size

Reynold Xin created SPARK-2956: -- Summary: Support transferring blocks larger than MTU size Key: SPARK-2956 URL: https://issues.apache.org/jira/browse/SPARK-2956 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2957) Leverage Hadoop native io's fadvise and read-ahead in Netty transferTo

[ https://issues.apache.org/jira/browse/SPARK-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092290#comment-14092290 ] Reynold Xin commented on SPARK-2957: cc [~tlipcon] [~t...@lipcon.org] will probably

[jira] [Updated] (SPARK-2468) Netty-based shuffle network module

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2468: --- Summary: Netty-based shuffle network module (was: Netty based network communication) Netty-based

[jira] [Updated] (SPARK-2468) Netty based network communication

[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2468: --- Summary: Netty based network communication (was: zero-copy shuffle network communication) Netty

[jira] [Created] (SPARK-2957) Leverage Hadoop native io's fadvise and read-ahead in Netty transferTo

Reynold Xin created SPARK-2957: -- Summary: Leverage Hadoop native io's fadvise and read-ahead in Netty transferTo Key: SPARK-2957 URL: https://issues.apache.org/jira/browse/SPARK-2957 Project: Spark

[jira] [Updated] (SPARK-2956) Support transferring large blocks in Netty network module

[ https://issues.apache.org/jira/browse/SPARK-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2956: --- Description: The existing Netty shuffle implementation does not support large blocks. The culprit

[jira] [Commented] (SPARK-2957) Leverage Hadoop native io's fadvise and read-ahead in Netty transferTo

[ https://issues.apache.org/jira/browse/SPARK-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092294#comment-14092294 ] Todd Lipcon commented on SPARK-2957: Sure, happy to help Leverage Hadoop native io's

[jira] [Created] (SPARK-2958) FileClientHandler should not be shared in the pipeline

Reynold Xin created SPARK-2958: -- Summary: FileClientHandler should not be shared in the pipeline Key: SPARK-2958 URL: https://issues.apache.org/jira/browse/SPARK-2958 Project: Spark Issue Type:

[jira] [Created] (SPARK-2959) Use a single FileClient and Netty client thread pool

Reynold Xin created SPARK-2959: -- Summary: Use a single FileClient and Netty client thread pool Key: SPARK-2959 URL: https://issues.apache.org/jira/browse/SPARK-2959 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092355#comment-14092355 ] Kousuke Saruta commented on SPARK-2677: --- SPARK-2538 was resolved but there is still

[jira] [Updated] (SPARK-2960) Spark executables fail to start via symlinks

[ https://issues.apache.org/jira/browse/SPARK-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Rojansky updated SPARK-2960: - Priority: Minor (was: Major) Spark executables fail to start via symlinks

[jira] [Updated] (SPARK-2960) Spark executables fail to start via symlinks

[ https://issues.apache.org/jira/browse/SPARK-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Rojansky updated SPARK-2960: - Summary: Spark executables fail to start via symlinks (was: Spark executables failed to start

[jira] [Created] (SPARK-2960) Spark executables failed to start via symlinks

Shay Rojansky created SPARK-2960: Summary: Spark executables failed to start via symlinks Key: SPARK-2960 URL: https://issues.apache.org/jira/browse/SPARK-2960 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2961) Use statistics to skip partitions when reading from in-memory columnar data

Michael Armbrust created SPARK-2961: --- Summary: Use statistics to skip partitions when reading from in-memory columnar data Key: SPARK-2961 URL: https://issues.apache.org/jira/browse/SPARK-2961

[jira] [Updated] (SPARK-2961) Use statistics to skip partitions when reading from in-memory columnar data

[ https://issues.apache.org/jira/browse/SPARK-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2961: Target Version/s: 1.1.0 Use statistics to skip partitions when reading from in-memory

[jira] [Commented] (SPARK-2961) Use statistics to skip partitions when reading from in-memory columnar data

[ https://issues.apache.org/jira/browse/SPARK-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092373#comment-14092373 ] Apache Spark commented on SPARK-2961: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-2062) VertexRDD.apply does not use the mergeFunc

[ https://issues.apache.org/jira/browse/SPARK-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092381#comment-14092381 ] Larry Xiao commented on SPARK-2062: --- Is anyone working on it? I want to take it. My plan

[jira] [Resolved] (SPARK-2936) Migrate Netty network module from Java to Scala

[ https://issues.apache.org/jira/browse/SPARK-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-2936. --- Resolution: Fixed Migrate Netty network module from Java to Scala

[jira] [Created] (SPARK-2962) Suboptimal scheduling in spark

Mridul Muralidharan created SPARK-2962: -- Summary: Suboptimal scheduling in spark Key: SPARK-2962 URL: https://issues.apache.org/jira/browse/SPARK-2962 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092418#comment-14092418 ] Matei Zaharia commented on SPARK-2962: -- I thought this was fixed in

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092427#comment-14092427 ] Mridul Muralidharan commented on SPARK-2962: To give more context; a) Our

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092430#comment-14092430 ] Mridul Muralidharan commented on SPARK-2962: Hi [~matei], I am referencing

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092431#comment-14092431 ] Mridul Muralidharan commented on SPARK-2962: Note, I dont think this is a

[jira] [Comment Edited] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092427#comment-14092427 ] Mridul Muralidharan edited comment on SPARK-2962 at 8/11/14 4:35 AM:

[jira] [Commented] (SPARK-2912) Jenkins should include the commit hash in his messages

[ https://issues.apache.org/jira/browse/SPARK-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092434#comment-14092434 ] Michael Yannakopoulos commented on SPARK-2912: -- Hi Nicholas, I can work on

[jira] [Commented] (SPARK-2912) Jenkins should include the commit hash in his messages

[ https://issues.apache.org/jira/browse/SPARK-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092435#comment-14092435 ] Patrick Wendell commented on SPARK-2912: Hey Michael - I believe [~nchammas] is

[jira] [Commented] (SPARK-2912) Jenkins should include the commit hash in his messages

[ https://issues.apache.org/jira/browse/SPARK-2912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092437#comment-14092437 ] Michael Yannakopoulos commented on SPARK-2912: -- Thanks for the quick reply

[jira] [Created] (SPARK-2963) There no documentation for building about SparkSQL

Kousuke Saruta created SPARK-2963: - Summary: There no documentation for building about SparkSQL Key: SPARK-2963 URL: https://issues.apache.org/jira/browse/SPARK-2963 Project: Spark Issue