[jira] [Updated] (SPARK-2963) There no documentation for building about SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Description: Currently, if we'd like to use ThriftServer or CLI for SparkSQL, we need to use

[jira] [Updated] (SPARK-2963) There no documentation about building ThriftServer and CLI for SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: There no documentation about building ThriftServer and CLI for SparkSQL (was: There

[jira] [Updated] (SPARK-2963) There no documentation about building to use HiveServer and CLI for SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: There no documentation about building to use HiveServer and CLI for SparkSQL (was:

[jira] [Updated] (SPARK-2963) There no documentation about building to use HiveServer and CLI for SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Description: Currently, if we'd like to use HiveServer or CLI for SparkSQL, we need to use

[jira] [Comment Edited] (SPARK-2204) Scheduler for Mesos in fine-grained mode launches tasks on wrong executors

2014-08-11 Thread Xu Zhongxing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092376#comment-14092376 ] Xu Zhongxing edited comment on SPARK-2204 at 8/11/14 6:49 AM: --

[jira] [Created] (SPARK-2964) Wrong silent option in spark-sql script

2014-08-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2964: - Summary: Wrong silent option in spark-sql script Key: SPARK-2964 URL: https://issues.apache.org/jira/browse/SPARK-2964 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2965) Fix HashOuterJoin output nullabilities.

2014-08-11 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2965: Summary: Fix HashOuterJoin output nullabilities. Key: SPARK-2965 URL: https://issues.apache.org/jira/browse/SPARK-2965 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2966) Add an approximation algorithm for hierarchical clustering to MLlib

2014-08-11 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2966: --- Summary: Add an approximation algorithm for hierarchical clustering to MLlib (was: Add an

[jira] [Created] (SPARK-2967) Several SQL unit test failed when sort-based shuffle is enabled

2014-08-11 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-2967: -- Summary: Several SQL unit test failed when sort-based shuffle is enabled Key: SPARK-2967 URL: https://issues.apache.org/jira/browse/SPARK-2967 Project: Spark

[jira] [Updated] (SPARK-2969) Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull.

2014-08-11 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-2969: - Description: Make {{ScalaReflection}} be able to handle like: - Seq\[Int] as

[jira] [Created] (SPARK-2969) Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull.

2014-08-11 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2969: Summary: Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull. Key: SPARK-2969 URL: https://issues.apache.org/jira/browse/SPARK-2969

[jira] [Commented] (SPARK-2969) Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull.

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092671#comment-14092671 ] Apache Spark commented on SPARK-2969: - User 'ueshin' has created a pull request for

[jira] [Commented] (SPARK-2878) Inconsistent Kryo serialisation with custom Kryo Registrator

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092677#comment-14092677 ] Apache Spark commented on SPARK-2878: - User 'GrahamDennis' has created a pull request

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092705#comment-14092705 ] Kousuke Saruta commented on SPARK-2970: --- I noticed it's not caused by the reason

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092710#comment-14092710 ] Apache Spark commented on SPARK-2970: - User 'sarutak' has created a pull request for

[jira] [Created] (SPARK-2971) Orphaned YARN ApplicationMaster lingers forever

2014-08-11 Thread Shay Rojansky (JIRA)
Shay Rojansky created SPARK-2971: Summary: Orphaned YARN ApplicationMaster lingers forever Key: SPARK-2971 URL: https://issues.apache.org/jira/browse/SPARK-2971 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

2014-08-11 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092807#comment-14092807 ] Mridul Muralidharan commented on SPARK-2962: On further investigation : a)

[jira] [Commented] (SPARK-1777) Pass cached blocks directly to disk if memory is not large enough

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092826#comment-14092826 ] Apache Spark commented on SPARK-1777: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092860#comment-14092860 ] Cheng Lian commented on SPARK-2970: --- [~sarutak] Would you mind to update the issue

[jira] [Updated] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2970: -- Description: When spark-sql script run with spark.eventLog.enabled set true, it ends with

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092879#comment-14092879 ] Kousuke Saruta commented on SPARK-2970: --- [~liancheng] Thank you pointing my mistake.

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

2014-08-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092881#comment-14092881 ] Thomas Graves commented on SPARK-2089: -- Sandy, just wondering if you have any ETA on

[jira] [Comment Edited] (SPARK-2963) There no documentation about building to use HiveServer and CLI for SparkSQL

2014-08-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092889#comment-14092889 ] Cheng Lian edited comment on SPARK-2963 at 8/11/14 3:31 PM:

[jira] [Commented] (SPARK-2963) The description about building to use HiveServer and CLI is imcomplete

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092894#comment-14092894 ] Kousuke Saruta commented on SPARK-2963: --- I've updated this title and Github's one.

[jira] [Updated] (SPARK-2963) The description about building to use HiveServer and CLI is imcomplete

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Description: Currently, if we'd like to use HiveServer or CLI for SparkSQL, we need to use

[jira] [Updated] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-1297: -- Attachment: spark-1297-v4.txt Patch v4 adds two profiles to examples/pom.xml : hbase-hadoop1 (default)

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092967#comment-14092967 ] Sean Owen commented on SPARK-1297: -- I think you may want to open a PR rather than post

[jira] [Created] (SPARK-2972) APPLICATION_COMPLETE not created in Python unless context explicitly stopped

2014-08-11 Thread Shay Rojansky (JIRA)
Shay Rojansky created SPARK-2972: Summary: APPLICATION_COMPLETE not created in Python unless context explicitly stopped Key: SPARK-2972 URL: https://issues.apache.org/jira/browse/SPARK-2972 Project:

[jira] [Created] (SPARK-2973) Add a way to show tables without executing a job

2014-08-11 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-2973: - Summary: Add a way to show tables without executing a job Key: SPARK-2973 URL: https://issues.apache.org/jira/browse/SPARK-2973 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092988#comment-14092988 ] Ted Yu commented on SPARK-1297: --- HBase client doesn't need to specify dependency on

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093012#comment-14093012 ] Ted Yu commented on SPARK-1297: --- https://github.com/apache/spark/pull/1893 Upgrade HBase

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093018#comment-14093018 ] Apache Spark commented on SPARK-1297: - User 'tedyu' has created a pull request for

[jira] [Created] (SPARK-2974) Utils.getLocalDir() may return non-existent spark.local.dir directory

2014-08-11 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-2974: - Summary: Utils.getLocalDir() may return non-existent spark.local.dir directory Key: SPARK-2974 URL: https://issues.apache.org/jira/browse/SPARK-2974 Project: Spark

[jira] [Updated] (SPARK-2717) BasicBlockFetchIterator#next should log when it gets stuck

2014-08-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2717: --- Priority: Major (was: Blocker) BasicBlockFetchIterator#next should log when it gets stuck

[jira] [Updated] (SPARK-2717) BasicBlockFetchIterator#next should log when it gets stuck

2014-08-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2717: --- Priority: Critical (was: Major) BasicBlockFetchIterator#next should log when it gets stuck

[jira] [Updated] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2931: --- Target Version/s: 1.1.0 getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

[jira] [Updated] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2931: --- Fix Version/s: (was: 1.1.0) getAllowedLocalityLevel() throws

[jira] [Updated] (SPARK-2963) The description about building to use HiveServer and CLI is incomplete

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: The description about building to use HiveServer and CLI is incomplete (was: The

[jira] [Created] (SPARK-2976) There are too many tabs in some source files

2014-08-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2976: - Summary: There are too many tabs in some source files Key: SPARK-2976 URL: https://issues.apache.org/jira/browse/SPARK-2976 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2890) Spark SQL should allow SELECT with duplicated columns

2014-08-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093119#comment-14093119 ] Yin Huai commented on SPARK-2890: - What is the semantic when you have columns with same

[jira] [Commented] (SPARK-2790) PySpark zip() doesn't work properly if RDDs have different serializers

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093133#comment-14093133 ] Apache Spark commented on SPARK-2790: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-1284) pyspark hangs after IOError on Executor

2014-08-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093137#comment-14093137 ] Davies Liu commented on SPARK-1284: --- [~jblomo], could you reproduce this on master or

[jira] [Commented] (SPARK-2700) Hidden files (such as .impala_insert_staging) should be filtered out by sqlContext.parquetFile

2014-08-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093150#comment-14093150 ] Yin Huai commented on SPARK-2700: - Can we resolve it? Hidden files (such as

[jira] [Resolved] (SPARK-2948) PySpark doesn't work on Python 2.6

2014-08-11 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2948. --- Resolution: Fixed Fix Version/s: 1.1.0 PySpark doesn't work on Python 2.6

[jira] [Resolved] (SPARK-2954) PySpark MLlib serialization tests fail on Python 2.6

2014-08-11 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2954. --- Resolution: Fixed Fix Version/s: 1.1.0 PySpark MLlib serialization tests fail on Python 2.6

[jira] [Created] (SPARK-2977) Fix handling of short shuffle manager names in ShuffleBlockManager

2014-08-11 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-2977: - Summary: Fix handling of short shuffle manager names in ShuffleBlockManager Key: SPARK-2977 URL: https://issues.apache.org/jira/browse/SPARK-2977 Project: Spark

[jira] [Resolved] (SPARK-2101) Python unit tests fail on Python 2.6 because of lack of unittest.skipIf()

2014-08-11 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2101. --- Resolution: Fixed Fix Version/s: 1.1.0 Python unit tests fail on Python 2.6 because of lack

[jira] [Commented] (SPARK-2976) There are too many tabs in some source files

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093175#comment-14093175 ] Apache Spark commented on SPARK-2976: - User 'sarutak' has created a pull request for

[jira] [Updated] (SPARK-2420) Dependency changes for compatibility with Hive

2014-08-11 Thread Brock Noland (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated SPARK-2420: Labels: Hive (was: ) Dependency changes for compatibility with Hive

[jira] [Commented] (SPARK-1284) pyspark hangs after IOError on Executor

2014-08-11 Thread Jim Blomo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093219#comment-14093219 ] Jim Blomo commented on SPARK-1284: -- I will try to reproduce on the 1.1 branch later this

[jira] [Resolved] (SPARK-2891) Daemon failed to launch worker

2014-08-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-2891. --- Resolution: Duplicate Fix Version/s: 1.1.0 duplicated to 2898 Daemon failed to launch

[jira] [Commented] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093295#comment-14093295 ] Apache Spark commented on SPARK-2931: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-1065) PySpark runs out of memory with large broadcast variables

2014-08-11 Thread Vlad Frolov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093413#comment-14093413 ] Vlad Frolov commented on SPARK-1065: I am facing the same issue in my project, where I

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093466#comment-14093466 ] Ted Yu commented on SPARK-1297: --- w.r.t. build, by default, hbase-hadoop1 would be used. If

[jira] [Commented] (SPARK-1297) Upgrade HBase dependency to 0.98.0

2014-08-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093468#comment-14093468 ] Sean Owen commented on SPARK-1297: -- Yes I think you'd need to reflect that in changes to

[jira] [Updated] (SPARK-2975) SPARK_LOCAL_DIRS may cause problems when running in local mode

2014-08-11 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2975: -- Priority: Critical (was: Minor) I'm raising the priority of this issue to 'critical', since it causes

[jira] [Created] (SPARK-2978) Provide an MR-style shuffle transformation

2014-08-11 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-2978: - Summary: Provide an MR-style shuffle transformation Key: SPARK-2978 URL: https://issues.apache.org/jira/browse/SPARK-2978 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-2978) Provide an MR-style shuffle transformation

2014-08-11 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2978: -- Description: For Hive on Spark joins in particular, and for running legacy MR code in general, I

[jira] [Updated] (SPARK-2978) Provide an MR-style shuffle transformation

2014-08-11 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2978: -- Description: For Hive on Spark joins in particular, and for running legacy MR code in general, I

[jira] [Updated] (SPARK-2978) Provide an MR-style shuffle transformation

2014-08-11 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2978: -- Description: For Hive on Spark joins in particular, and for running legacy MR code in general, I

[jira] [Created] (SPARK-2979) Improve the convergence rate by minimize the condition number in LOR with LBFGS

2014-08-11 Thread DB Tsai (JIRA)
DB Tsai created SPARK-2979: -- Summary: Improve the convergence rate by minimize the condition number in LOR with LBFGS Key: SPARK-2979 URL: https://issues.apache.org/jira/browse/SPARK-2979 Project: Spark

[jira] [Commented] (SPARK-2979) Improve the convergence rate by minimize the condition number in LOR with LBFGS

2014-08-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093604#comment-14093604 ] Apache Spark commented on SPARK-2979: - User 'dbtsai' has created a pull request for

[jira] [Updated] (SPARK-2979) Improve the convergence rate by minimizing the condition number in LOR with LBFGS

2014-08-11 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-2979: --- Summary: Improve the convergence rate by minimizing the condition number in LOR with LBFGS (was: Improve

[jira] [Updated] (SPARK-2515) Hypothesis testing

2014-08-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2515: - Fix Version/s: 1.1.0 Hypothesis testing -- Key: SPARK-2515

[jira] [Updated] (SPARK-2515) Chi-squared test

2014-08-11 Thread Doris Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Xin updated SPARK-2515: - Summary: Chi-squared test (was: Hypothesis testing) Chi-squared test

[jira] [Closed] (SPARK-2515) Chi-squared test

2014-08-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-2515. Resolution: Implemented Target Version/s: 1.1.0 Chi-squared test

[jira] [Created] (SPARK-2980) Python support for chi-squared test

2014-08-11 Thread Doris Xin (JIRA)
Doris Xin created SPARK-2980: Summary: Python support for chi-squared test Key: SPARK-2980 URL: https://issues.apache.org/jira/browse/SPARK-2980 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-2980) Python support for chi-squared test

2014-08-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2980: - Assignee: (was: Doris Xin) Python support for chi-squared test

[jira] [Updated] (SPARK-2934) Adding LogisticRegressionWithLBFGS for training with LBFGS Optimizer

2014-08-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2934: - Assignee: DB Tsai Adding LogisticRegressionWithLBFGS for training with LBFGS Optimizer

[jira] [Resolved] (SPARK-2844) Existing JVM Hive Context not correctly used in Python Hive Context

2014-08-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2844. - Resolution: Fixed Fix Version/s: 1.1.0 Existing JVM Hive Context not correctly

[jira] [Resolved] (SPARK-2590) Add config property to disable incremental collection used in Thrift server

2014-08-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2590. - Resolution: Fixed Fix Version/s: 1.1.0 Add config property to disable

[jira] [Resolved] (SPARK-2965) Fix HashOuterJoin output nullabilities.

2014-08-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2965. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Takuya Ueshin Fix

[jira] [Resolved] (SPARK-2968) Fix nullabilities of Explode.

2014-08-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2968. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Takuya Ueshin Fix

[jira] [Resolved] (SPARK-2650) Caching tables larger than memory causes OOMs

2014-08-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2650. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Michael

[jira] [Created] (SPARK-2981) PartitionStrategy: VertexID hash overflow

2014-08-11 Thread Larry Xiao (JIRA)
Larry Xiao created SPARK-2981: - Summary: PartitionStrategy: VertexID hash overflow Key: SPARK-2981 URL: https://issues.apache.org/jira/browse/SPARK-2981 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2981) PartitionStrategy: VertexID hash overflow

2014-08-11 Thread Larry Xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Larry Xiao updated SPARK-2981: -- Description: In PartitionStrategy.scala a PartitionID is calculated by multiplying VertexId with a

[jira] [Resolved] (SPARK-2826) Reduce the Memory Copy for HashOuterJoin

2014-08-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2826. - Resolution: Fixed Fix Version/s: 1.1.0 Reduce the Memory Copy for HashOuterJoin

[jira] [Resolved] (SPARK-2934) Adding LogisticRegressionWithLBFGS for training with LBFGS Optimizer

2014-08-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2934. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1862

[jira] [Created] (SPARK-2982) Glitch of spark streaming

2014-08-11 Thread dai zhiyuan (JIRA)
dai zhiyuan created SPARK-2982: -- Summary: Glitch of spark streaming Key: SPARK-2982 URL: https://issues.apache.org/jira/browse/SPARK-2982 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2890) Spark SQL should allow SELECT with duplicated columns

2014-08-11 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093746#comment-14093746 ] Jianshi Huang commented on SPARK-2890: -- My use case: The result will be parsed into

[jira] [Resolved] (SPARK-2923) Implement some basic linalg operations in MLlib

2014-08-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2923. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1849