[jira] [Created] (SPARK-2714) DAGScheduler logs jobid when runJob finishes

2014-07-28 Thread YanTang Zhai (JIRA)
YanTang Zhai created SPARK-2714: --- Summary: DAGScheduler logs jobid when runJob finishes Key: SPARK-2714 URL: https://issues.apache.org/jira/browse/SPARK-2714 Project: Spark Issue Type:

[jira] [Closed] (SPARK-2613) CLONE - word2vec: Distributed Representation of Words

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-2613. Assignee: Xiangrui Meng (was: Liquan Pei) CLONE - word2vec: Distributed Representation of Words

[jira] [Commented] (SPARK-2510) word2vec: Distributed Representation of Words

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075951#comment-14075951 ] Xiangrui Meng commented on SPARK-2510: -- Had an offline discussion with [~liquanpei]

[jira] [Updated] (SPARK-2692) Decision Tree API update

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2692: - Assignee: Joseph K. Bradley Decision Tree API update

[jira] [Updated] (SPARK-2692) Decision Tree API update

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2692: - Target Version/s: 1.1.0 Affects Version/s: 1.0.0 Decision Tree API update

[jira] [Created] (SPARK-2715) ExternalAppendOnlyMap adds max limit of times and max limit of disk bytes written for spilling

2014-07-28 Thread YanTang Zhai (JIRA)
YanTang Zhai created SPARK-2715: --- Summary: ExternalAppendOnlyMap adds max limit of times and max limit of disk bytes written for spilling Key: SPARK-2715 URL: https://issues.apache.org/jira/browse/SPARK-2715

[jira] [Updated] (SPARK-2702) Upgrade Tachyon dependency to 0.5.0

2014-07-28 Thread Haoyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-2702: -- Assignee: Rong Gu Upgrade Tachyon dependency to 0.5.0 ---

[jira] [Updated] (SPARK-2703) Make Tachyon related unit tests execute without deploying a Tachyon system locally.

2014-07-28 Thread Haoyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-2703: -- Assignee: Rong Gu Make Tachyon related unit tests execute without deploying a Tachyon system

[jira] [Commented] (SPARK-2614) Add the spark-examples-xxx-.jar to the Debian packages created with mvn ... -Pdeb (using assembly/pom.xml)

2014-07-28 Thread Christian Tzolov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075998#comment-14075998 ] Christian Tzolov commented on SPARK-2614: - The #1611 pull request addresses some

[jira] [Commented] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

2014-07-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076033#comment-14076033 ] Guoqiang Li commented on SPARK-2677: [~pwendell] , [~sarutak] How about the following

[jira] [Comment Edited] (SPARK-2511) Add TF-IDF featurizer

2014-07-28 Thread duanfa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076035#comment-14076035 ] duanfa edited comment on SPARK-2511 at 7/28/14 9:05 AM: i need it

[jira] [Commented] (SPARK-2511) Add TF-IDF featurizer

2014-07-28 Thread duanfa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076035#comment-14076035 ] duanfa commented on SPARK-2511: --- i need it alse Add TF-IDF featurizer

[jira] [Comment Edited] (SPARK-2511) Add TF-IDF featurizer

2014-07-28 Thread duanfa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076035#comment-14076035 ] duanfa edited comment on SPARK-2511 at 7/28/14 9:12 AM: i need it

[jira] [Commented] (SPARK-2576) slave node throws NoClassDefFoundError $line11.$read$ when executing a Spark QL query on HDFS CSV file

2014-07-28 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076086#comment-14076086 ] Teng Qiu commented on SPARK-2576: - i get same problem, 1.0.1, standalone cluster slave

[jira] [Commented] (SPARK-2417) Decision tree tests are failing

2014-07-28 Thread Patrick Morton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076101#comment-14076101 ] Patrick Morton commented on SPARK-2417: --- Hallucinogenic stroke of important

[jira] [Commented] (SPARK-2415) RowWriteSupport should handle empty ArrayType correctly.

2014-07-28 Thread Patrick Morton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076104#comment-14076104 ] Patrick Morton commented on SPARK-2415: --- In the ethical, three endings of core

[jira] [Commented] (SPARK-2714) DAGScheduler logs jobid when runJob finishes

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076232#comment-14076232 ] Apache Spark commented on SPARK-2714: - User 'YanTangZhai' has created a pull request

[jira] [Issue Comment Deleted] (SPARK-2415) RowWriteSupport should handle empty ArrayType correctly.

2014-07-28 Thread Jake Farrell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jake Farrell updated SPARK-2415: Comment: was deleted (was: In the ethical, three endings of core symptoms have been not linked

[jira] [Issue Comment Deleted] (SPARK-2417) Decision tree tests are failing

2014-07-28 Thread Jake Farrell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jake Farrell updated SPARK-2417: Comment: was deleted (was: Hallucinogenic stroke of important metabolites during black father may

[jira] [Commented] (SPARK-2715) ExternalAppendOnlyMap adds max limit of times and max limit of disk bytes written for spilling

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076257#comment-14076257 ] Apache Spark commented on SPARK-2715: - User 'YanTangZhai' has created a pull request

[jira] [Commented] (SPARK-2141) Add sc.getPersistentRDDs() to PySpark

2014-07-28 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076262#comment-14076262 ] Kan Zhang commented on SPARK-2141: -- Hi [~nchammas], we are debating potential use cases

[jira] [Commented] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076278#comment-14076278 ] Apache Spark commented on SPARK-2677: - User 'witgo' has created a pull request for

[jira] [Comment Edited] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

2014-07-28 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076033#comment-14076033 ] Guoqiang Li edited comment on SPARK-2677 at 7/28/14 3:00 PM: -

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running applications

2014-07-28 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076403#comment-14076403 ] Aaron Davidson commented on SPARK-1860: --- There's not an easy way to tell if an

[jira] [Updated] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-07-28 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-1860: -- Description: The default values of the standalone worker cleanup code cleanup all application

[jira] [Created] (SPARK-2716) Having clause with no references fails to resolve

2014-07-28 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2716: --- Summary: Having clause with no references fails to resolve Key: SPARK-2716 URL: https://issues.apache.org/jira/browse/SPARK-2716 Project: Spark Issue

[jira] [Updated] (SPARK-2563) Re-open sockets to handle connect timeouts

2014-07-28 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-2563: - Description: In a large EC2 cluster, I often see the first shuffle stage in a

[jira] [Comment Edited] (SPARK-2563) Re-open sockets to handle connect timeouts

2014-07-28 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065735#comment-14065735 ] Shivaram Venkataraman edited comment on SPARK-2563 at 7/28/14 5:43 PM:

[jira] [Commented] (SPARK-2410) Thrift/JDBC Server

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076453#comment-14076453 ] Apache Spark commented on SPARK-2410: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-2420) Change Spark build to minimize library conflicts

2014-07-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076468#comment-14076468 ] Sean Owen commented on SPARK-2420: -- I'm sure shading just means moving the packages, and

[jira] [Commented] (SPARK-2420) Change Spark build to minimize library conflicts

2014-07-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076456#comment-14076456 ] Marcelo Vanzin commented on SPARK-2420: --- So let me see if I'm following things so

[jira] [Commented] (SPARK-2420) Change Spark build to minimize library conflicts

2014-07-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076488#comment-14076488 ] Marcelo Vanzin commented on SPARK-2420: --- Forking {{Optional}} would make Option 2

[jira] [Resolved] (SPARK-2523) For partitioned Hive tables, partition-specific ObjectInspectors should be used.

2014-07-28 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2523. - Resolution: Fixed Fix Version/s: 1.1.0 For partitioned Hive tables,

[jira] [Resolved] (SPARK-2479) Comparing floating-point numbers using relative error in UnitTests

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2479. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1425

[jira] [Updated] (SPARK-2544) Improve ALS algorithm resource usage

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2544: - Target Version/s: 1.1.0 Improve ALS algorithm resource usage

[jira] [Updated] (SPARK-2544) Improve ALS algorithm resource usage

2014-07-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2544: - Assignee: Guoqiang Li Improve ALS algorithm resource usage

[jira] [Resolved] (SPARK-2410) Thrift/JDBC Server

2014-07-28 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2410. - Resolution: Fixed Thrift/JDBC Server -- Key:

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-07-28 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076778#comment-14076778 ] Mark Hamstra commented on SPARK-1860: - I don't think that there is much in the way of

[jira] [Updated] (SPARK-2305) pyspark - depend on py4j 0.8.1

2014-07-28 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2305: -- Target Version/s: 1.1.0 Assignee: Josh Rosen Py4J 0.8.2.1 was just released; I'll look

[jira] [Resolved] (SPARK-2411) Standalone Master - direct users to turn on event logs

2014-07-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-2411. -- Resolution: Fixed Standalone Master - direct users to turn on event logs

[jira] [Comment Edited] (SPARK-1649) Figure out Nullability semantics for Array elements and Map values

2014-07-28 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076739#comment-14076739 ] Yin Huai edited comment on SPARK-1649 at 7/28/14 8:42 PM: -- Seems

[jira] [Commented] (SPARK-1687) Support NamedTuples in RDDs

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076849#comment-14076849 ] Davies Liu commented on SPARK-1687: --- Dill is implemented in pure Python, so it will have

[jira] [Commented] (SPARK-2655) Change the default logging level to WARN

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076863#comment-14076863 ] Davies Liu commented on SPARK-2655: --- [~pwendell] [~matei], how do you think about this?

[jira] [Updated] (SPARK-2717) BasicBlockFetchIterator#next should log when it gets stuck

2014-07-28 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2717: --- Component/s: Spark Core BasicBlockFetchIterator#next should log when it gets stuck

[jira] [Created] (SPARK-2717) BasicBlockFetchIterator#next should log when it gets stuck

2014-07-28 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2717: -- Summary: BasicBlockFetchIterator#next should log when it gets stuck Key: SPARK-2717 URL: https://issues.apache.org/jira/browse/SPARK-2717 Project: Spark

[jira] [Updated] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

2014-07-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2718: - Affects Version/s: (was: 1.0.1) 1.0.2 YARN does not handle spark configs

[jira] [Created] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

2014-07-28 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2718: Summary: YARN does not handle spark configs with quotes or backslashes Key: SPARK-2718 URL: https://issues.apache.org/jira/browse/SPARK-2718 Project: Spark Issue

[jira] [Updated] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

2014-07-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2718: - Description: Say we have the following config: {code} spark.app.name spark shell with spaces and quotes

[jira] [Updated] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

2014-07-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2718: - Description: Say we have the following config: {code} spark.app.name spark shell with spaces and quotes

[jira] [Commented] (SPARK-1343) PySpark OOMs without caching

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077011#comment-14077011 ] Davies Liu commented on SPARK-1343: --- Maybe it's related to partitionBy() with small

[jira] [Updated] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

2014-07-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2718: - Description: Say we have the following config: {code} spark.app.name spark shell with spaces and quotes

[jira] [Resolved] (SPARK-1343) PySpark OOMs without caching

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-1343. --- Resolution: Fixed Fix Version/s: 0.9.0 1.0.0 Target Version/s:

[jira] [Commented] (SPARK-1343) PySpark OOMs without caching

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077012#comment-14077012 ] Davies Liu commented on SPARK-1343: --- https://github.com/apache/spark/pull/1460

[jira] [Commented] (SPARK-1687) Support NamedTuples in RDDs

2014-07-28 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077019#comment-14077019 ] Kan Zhang commented on SPARK-1687: -- Sure, pls go ahead and feel free to take over this

[jira] [Commented] (SPARK-2023) PySpark reduce does a map side reduce and then sends the results to the driver for final reduce, instead do this more like Scala Spark.

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077075#comment-14077075 ] Davies Liu commented on SPARK-2023: --- In most cases, the result of reduce will be small,

[jira] [Created] (SPARK-2719) Add Mima binary checks to Flume-Sink

2014-07-28 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-2719: Summary: Add Mima binary checks to Flume-Sink Key: SPARK-2719 URL: https://issues.apache.org/jira/browse/SPARK-2719 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2022) Spark 1.0.0 is failing if mesos.coarse set to true

2014-07-28 Thread Timothy Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077109#comment-14077109 ] Timothy Chen commented on SPARK-2022: - Github PR:

[jira] [Commented] (SPARK-2022) Spark 1.0.0 is failing if mesos.coarse set to true

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077110#comment-14077110 ] Apache Spark commented on SPARK-2022: - User 'tnachen' has created a pull request for

[jira] [Commented] (SPARK-1649) Figure out Nullability semantics for Array elements and Map values

2014-07-28 Thread Robbie Russo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077115#comment-14077115 ] Robbie Russo commented on SPARK-1649: - Thrift also supports null values in a map and

[jira] [Commented] (SPARK-1649) Figure out Nullability semantics for Array elements and Map values

2014-07-28 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077127#comment-14077127 ] Yin Huai commented on SPARK-1649: - [~rrusso2007] Can you open a JIRA for the issue of

[jira] [Commented] (SPARK-1687) Support NamedTuples in RDDs

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077136#comment-14077136 ] Apache Spark commented on SPARK-1687: - User 'davies' has created a pull request for

[jira] [Created] (SPARK-2720) spark-examples should depend on HBase modules for HBase 0.96+

2014-07-28 Thread Ted Yu (JIRA)
Ted Yu created SPARK-2720: - Summary: spark-examples should depend on HBase modules for HBase 0.96+ Key: SPARK-2720 URL: https://issues.apache.org/jira/browse/SPARK-2720 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1649) Figure out Nullability semantics for Array elements and Map values

2014-07-28 Thread Robbie Russo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077150#comment-14077150 ] Robbie Russo commented on SPARK-1649: - Just opened

[jira] [Created] (SPARK-2721) Fix MapType compatibility issues with reading Parquet datasets

2014-07-28 Thread Robbie Russo (JIRA)
Robbie Russo created SPARK-2721: --- Summary: Fix MapType compatibility issues with reading Parquet datasets Key: SPARK-2721 URL: https://issues.apache.org/jira/browse/SPARK-2721 Project: Spark

[jira] [Commented] (SPARK-2550) Support regularization and intercept in pyspark's linear methods

2014-07-28 Thread Michael Yannakopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077155#comment-14077155 ] Michael Yannakopoulos commented on SPARK-2550: -- Please ignore the previous

[jira] [Commented] (SPARK-2550) Support regularization and intercept in pyspark's linear methods

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077158#comment-14077158 ] Apache Spark commented on SPARK-2550: - User 'miccagiann' has created a pull request

[jira] [Commented] (SPARK-2382) build error:

2014-07-28 Thread Mukul Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077161#comment-14077161 ] Mukul Jain commented on SPARK-2382: --- how to open PR ? I am planning to close this issue

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

2014-07-28 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077179#comment-14077179 ] Ted Malaska commented on SPARK-2447: Making good progress. Just FYI it may take a

[jira] [Commented] (SPARK-2580) broken pipe collecting schemardd results

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077193#comment-14077193 ] Apache Spark commented on SPARK-2580: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-2305) pyspark - depend on py4j 0.8.1

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077202#comment-14077202 ] Apache Spark commented on SPARK-2305: - User 'JoshRosen' has created a pull request for

[jira] [Created] (SPARK-2722) Mechanism for escaping spark configs is not consistent

2014-07-28 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2722: Summary: Mechanism for escaping spark configs is not consistent Key: SPARK-2722 URL: https://issues.apache.org/jira/browse/SPARK-2722 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2722) Mechanism for escaping spark configs is not consistent

2014-07-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2722: - Description: Currently, you can specify a spark config in spark-defaults.conf as follows: {code}

[jira] [Commented] (SPARK-791) [pyspark] operator.getattr not serialized

2014-07-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077239#comment-14077239 ] Davies Liu commented on SPARK-791: -- This will be fixed by PR-1627[1] [1]

[jira] [Commented] (SPARK-1138) Spark 0.9.0 does not work with Hadoop / HDFS

2014-07-28 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077270#comment-14077270 ] Russell Jurney commented on SPARK-1138: --- I built spark master with 'sbt/sbt assembly

[jira] [Commented] (SPARK-1138) Spark 0.9.0 does not work with Hadoop / HDFS

2014-07-28 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077271#comment-14077271 ] Russell Jurney commented on SPARK-1138: --- See

[jira] [Created] (SPARK-2723) Block Manager should catch exceptions in putValues

2014-07-28 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-2723: Summary: Block Manager should catch exceptions in putValues Key: SPARK-2723 URL: https://issues.apache.org/jira/browse/SPARK-2723 Project: Spark

[jira] [Reopened] (SPARK-2512) Stratified sampling

2014-07-28 Thread Doris Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Xin reopened SPARK-2512: -- Stratified sampling --- Key: SPARK-2512 URL:

[jira] [Created] (SPARK-2724) Python version of Random RDD without support for arbitrary distribution

2014-07-28 Thread Doris Xin (JIRA)
Doris Xin created SPARK-2724: Summary: Python version of Random RDD without support for arbitrary distribution Key: SPARK-2724 URL: https://issues.apache.org/jira/browse/SPARK-2724 Project: Spark

[jira] [Commented] (SPARK-2724) Python version of Random RDD without support for arbitrary distribution

2014-07-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077286#comment-14077286 ] Apache Spark commented on SPARK-2724: - User 'dorx' has created a pull request for this

[jira] [Updated] (SPARK-2134) Report metrics before application finishes

2014-07-28 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2134: - Assignee: Rahul Singhal Report metrics before application finishes

[jira] [Created] (SPARK-2726) Remove SortOrder in ShuffleDependency and HashShuffleReader

2014-07-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2726: -- Summary: Remove SortOrder in ShuffleDependency and HashShuffleReader Key: SPARK-2726 URL: https://issues.apache.org/jira/browse/SPARK-2726 Project: Spark Issue

[jira] [Created] (SPARK-2727) HashShuffleReader should do in-place sort

2014-07-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2727: -- Summary: HashShuffleReader should do in-place sort Key: SPARK-2727 URL: https://issues.apache.org/jira/browse/SPARK-2727 Project: Spark Issue Type: Improvement