[jira] [Commented] (SPARK-2923) Implement some basic linalg operations in MLlib

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090352#comment-14090352 ] Apache Spark commented on SPARK-2923: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-2878) Inconsistent Kryo serialisation with custom Kryo Registrator

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090354#comment-14090354 ] Apache Spark commented on SPARK-2878: - User 'ash211' has created a pull request for

[jira] [Commented] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090397#comment-14090397 ] Reynold Xin commented on SPARK-2911: Would you like to change the usage of

[jira] [Created] (SPARK-2924) Remove use of default arguments where disallowed by 2.11

2014-08-08 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2924: -- Summary: Remove use of default arguments where disallowed by 2.11 Key: SPARK-2924 URL: https://issues.apache.org/jira/browse/SPARK-2924 Project: Spark

[jira] [Updated] (SPARK-2924) Remove use of default arguments where disallowed by 2.11

2014-08-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2924: --- Priority: Blocker (was: Major) Target Version/s: 1.1.0 Remove use of default

[jira] [Created] (SPARK-2925) bin/spark-sql shell throw unrecognized option error when set --driver-java-options

2014-08-08 Thread wangfei (JIRA)
wangfei created SPARK-2925: -- Summary: bin/spark-sql shell throw unrecognized option error when set --driver-java-options Key: SPARK-2925 URL: https://issues.apache.org/jira/browse/SPARK-2925 Project: Spark

[jira] [Updated] (SPARK-2643) Stages web ui has ERROR when pool name is None

2014-08-08 Thread YanTang Zhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YanTang Zhai updated SPARK-2643: Description: 14/07/23 16:01:44 WARN servlet.ServletHandler: /stages/

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

2014-08-08 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090473#comment-14090473 ] Alexander Ulanov commented on SPARK-1473: - I've implemented Chi-Squared and added

[jira] [Comment Edited] (SPARK-1473) Feature selection for high dimensional datasets

2014-08-08 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090473#comment-14090473 ] Alexander Ulanov edited comment on SPARK-1473 at 8/8/14 8:27 AM:

[jira] [Updated] (SPARK-2643) Stages web ui has ERROR when pool name is None

2014-08-08 Thread YanTang Zhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YanTang Zhai updated SPARK-2643: Description: 14/07/23 16:01:44 WARN servlet.ServletHandler: /stages/

[jira] [Updated] (SPARK-2590) Add config property to disable incremental collection used in Thrift server

2014-08-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2590: -- Description: {{SparkSQLOperationManager}} uses {{RDD.toLocalIterator}} to collect the result set one

[jira] [Commented] (SPARK-2590) Add config property to disable incremental collection used in Thrift server

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090492#comment-14090492 ] Apache Spark commented on SPARK-2590: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-2885) All-pairs similarity via DIMSUM

2014-08-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2885: - Assignee: Reza Zadeh All-pairs similarity via DIMSUM ---

[jira] [Commented] (SPARK-2916) [MLlib] While running regression tests with dense vectors of length greater than 1000, the treeAggregate blows up after several iterations

2014-08-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090498#comment-14090498 ] Burak Yavuz commented on SPARK-2916: will do [MLlib] While running regression tests

[jira] [Updated] (SPARK-2643) Stages web ui has ERROR when pool name is None

2014-08-08 Thread YanTang Zhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YanTang Zhai updated SPARK-2643: Description: 14/07/23 16:01:44 WARN servlet.ServletHandler: /stages/

[jira] [Commented] (SPARK-2922) spark web ui: Internal Error: Missing Template ERR_DNS_FAIL

2014-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090558#comment-14090558 ] Sean Owen commented on SPARK-2922: -- This doesn't seem to be anything to do with Spark.

[jira] [Commented] (SPARK-2906) FileLogger throws a invocation target exception.

2014-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090563#comment-14090563 ] Sean Owen commented on SPARK-2906: -- I think this is a duplicate of a couple JIRAs already

[jira] [Updated] (SPARK-2906) FileLogger throws a invocation target exception.

2014-08-08 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2906: --- Description: {noformat} 14/08/08 00:04:22 INFO ui.SparkUI: Stopped Spark web UI at

[jira] [Commented] (SPARK-2643) Stages web ui has ERROR when pool name is None

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090672#comment-14090672 ] Apache Spark commented on SPARK-2643: - User 'YanTangZhai' has created a pull request

[jira] [Commented] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-08 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090734#comment-14090734 ] Erik Erlandson commented on SPARK-2911: --- OK, shall I do it as part of this jira or

[jira] [Created] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-08-08 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-2926: -- Summary: Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle Key: SPARK-2926 URL: https://issues.apache.org/jira/browse/SPARK-2926 Project: Spark

[jira] [Updated] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-08-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-2926: --- Attachment: SortBasedShuffleRead.pdf A rough design doc is uploaded. Any comments would be greatly

[jira] [Updated] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-08-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-2926: --- Description: Currently Spark has already integrated sort-based shuffle write, which greatly improve

[jira] [Commented] (SPARK-2880) spark-submit processes app cmdline options

2014-08-08 Thread Shay Rojansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090880#comment-14090880 ] Shay Rojansky commented on SPARK-2880: -- It's indeed a duplicate of that bug, great to

[jira] [Created] (SPARK-2927) Add a conf to always read Binary columns stored in Parquet as String columns

2014-08-08 Thread Yin Huai (JIRA)
Yin Huai created SPARK-2927: --- Summary: Add a conf to always read Binary columns stored in Parquet as String columns Key: SPARK-2927 URL: https://issues.apache.org/jira/browse/SPARK-2927 Project: Spark

[jira] [Updated] (SPARK-2927) Add a conf to configure if we always read Binary columns stored in Parquet as String columns

2014-08-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2927: Summary: Add a conf to configure if we always read Binary columns stored in Parquet as String columns

[jira] [Commented] (SPARK-2927) Add a conf to configure if we always read Binary columns stored in Parquet as String columns

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090990#comment-14090990 ] Apache Spark commented on SPARK-2927: - User 'yhuai' has created a pull request for

[jira] [Updated] (SPARK-2890) Spark SQL should allow SELECT with duplicated columns

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2890: Target Version/s: 1.1.0 Spark SQL should allow SELECT with duplicated columns

[jira] [Updated] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2846: Target Version/s: 1.1.0 Spark SQL hive implementation bypass StorageHandler which breaks

[jira] [Updated] (SPARK-2721) Fix MapType compatibility issues with reading Parquet datasets

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2721: Target Version/s: 1.2.0 Fix MapType compatibility issues with reading Parquet datasets

[jira] [Updated] (SPARK-2928) TorrentBroadcast doesn't use the user specified serializer

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2928: --- Component/s: Spark Core TorrentBroadcast doesn't use the user specified serializer

[jira] [Created] (SPARK-2928) TorrentBroadcast doesn't use the user specified serializer

2014-08-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2928: -- Summary: TorrentBroadcast doesn't use the user specified serializer Key: SPARK-2928 URL: https://issues.apache.org/jira/browse/SPARK-2928 Project: Spark Issue

[jira] [Updated] (SPARK-2928) TorrentBroadcast should use the user specified serializer

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2928: --- Summary: TorrentBroadcast should use the user specified serializer (was: TorrentBroadcast doesn't

[jira] [Updated] (SPARK-2888) Fix addColumnMetadataToConf in HiveTableScan

2014-08-08 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2888: Summary: Fix addColumnMetadataToConf in HiveTableScan (was: Fix fixAddColumnMetadataToConf in

[jira] [Updated] (SPARK-2920) TorrentBroadcast does not support broadcast compression

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2920: --- Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0) TorrentBroadcast does not support broadcast

[jira] [Updated] (SPARK-2928) TorrentBroadcast should use the user specified serializer

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2928: --- Assignee: Guoqiang Li TorrentBroadcast should use the user specified serializer

[jira] [Created] (SPARK-2929) Rewrite HiveThriftServer2Suite and CliSuite

2014-08-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-2929: - Summary: Rewrite HiveThriftServer2Suite and CliSuite Key: SPARK-2929 URL: https://issues.apache.org/jira/browse/SPARK-2929 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-2897) org.apache.spark.broadcast.TorrentBroadcast does use the serializer class specified in the spark option spark.serializer

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2897: --- Description: HTTPBroadcast will changes the serializer according to the setting in

[jira] [Updated] (SPARK-2920) TorrentBroadcast does not support broadcast compression

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2920: --- Description: TorrentBroadcast always broadcast uncompressed content. The spark option

[jira] [Resolved] (SPARK-2908) JsonRDD.nullTypeToStringType does not convert all NullType to StringType

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2908. - Resolution: Fixed Fix Version/s: 1.1.0 JsonRDD.nullTypeToStringType does not

[jira] [Resolved] (SPARK-2877) MetastoreRelation should use SparkClassLoader when creating the tableDesc

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2877. - Resolution: Fixed Fix Version/s: 1.1.0 MetastoreRelation should use

[jira] [Commented] (SPARK-2929) Rewrite HiveThriftServer2Suite and CliSuite

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091077#comment-14091077 ] Apache Spark commented on SPARK-2929: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-1807) Modify SPARK_EXECUTOR_URI to allow for script execution in Mesos.

2014-08-08 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091096#comment-14091096 ] Matthew Farrellee commented on SPARK-1807: -- i disagree. SPARK_EXECUTOR_URI has

[jira] [Updated] (SPARK-2902) Enable compression for in-memory columnar storage by default

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2902: Target Version/s: 1.2.0 (was: 1.1.0) Enable compression for in-memory columnar storage

[jira] [Resolved] (SPARK-2919) Basic support for analyze command in HiveQl

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2919. - Resolution: Fixed Fix Version/s: 1.1.0 Basic support for analyze command in

[jira] [Resolved] (SPARK-2854) Finalize _acceptable_types in pyspark.sql

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2854. - Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Yin Huai Finalize

[jira] [Commented] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091102#comment-14091102 ] Michael Armbrust commented on SPARK-2846: - Hi [~alexliu68], Could you submit this

[jira] [Created] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

2014-08-08 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-2930: Summary: clarify docs on using webhdfs with spark.yarn.access.namenodes Key: SPARK-2930 URL: https://issues.apache.org/jira/browse/SPARK-2930 Project: Spark

[jira] [Created] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-08 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-2931: - Summary: getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException Key: SPARK-2931 URL: https://issues.apache.org/jira/browse/SPARK-2931 Project: Spark

[jira] [Commented] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091182#comment-14091182 ] Reynold Xin commented on SPARK-2911: We can do it as part of this ticket. provide

[jira] [Commented] (SPARK-1997) Update breeze to version 0.8.1

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091186#comment-14091186 ] Apache Spark commented on SPARK-1997: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-1997) Update breeze to version 0.9

2014-08-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091188#comment-14091188 ] Xiangrui Meng commented on SPARK-1997: -- breeze 0.9 is released. scalalogging was

[jira] [Updated] (SPARK-1997) Update breeze to version 0.9

2014-08-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1997: - Summary: Update breeze to version 0.9 (was: Update breeze to version 0.8.1) Update breeze to

[jira] [Commented] (SPARK-2924) Remove use of default arguments where disallowed by 2.11

2014-08-08 Thread Anand Avati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091193#comment-14091193 ] Anand Avati commented on SPARK-2924: PR: https://github.com/apache/spark/pull/1704

[jira] [Commented] (SPARK-2805) update akka to version 2.3

2014-08-08 Thread Anand Avati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091195#comment-14091195 ] Anand Avati commented on SPARK-2805: [~pwend...@gmail.com] ping update akka to

[jira] [Commented] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091215#comment-14091215 ] Apache Spark commented on SPARK-2911: - User 'erikerlandson' has created a pull request

[jira] [Created] (SPARK-2932) Move MasterFailureTest out of main source directory

2014-08-08 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-2932: - Summary: Move MasterFailureTest out of main source directory Key: SPARK-2932 URL: https://issues.apache.org/jira/browse/SPARK-2932 Project: Spark Issue

[jira] [Created] (SPARK-2933) Cleanup unnecessary and duplicated code in Yarn module

2014-08-08 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-2933: - Summary: Cleanup unnecessary and duplicated code in Yarn module Key: SPARK-2933 URL: https://issues.apache.org/jira/browse/SPARK-2933 Project: Spark Issue

[jira] [Commented] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091252#comment-14091252 ] Josh Rosen commented on SPARK-2931: --- It's pretty quick to set up a local spark-perf that

[jira] [Commented] (SPARK-2700) Hidden files (such as .impala_insert_staging) should be filtered out by sqlContext.parquetFile

2014-08-08 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091256#comment-14091256 ] Teng Qiu commented on SPARK-2700: - Hi [~srowen] and [~marmbrus] , what do you think about

[jira] [Updated] (SPARK-2933) Cleanup unnecessary and duplicated code in Yarn module

2014-08-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-2933: -- Component/s: YARN Cleanup unnecessary and duplicated code in Yarn module

[jira] [Updated] (SPARK-2932) Move MasterFailureTest out of main source directory

2014-08-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-2932: -- Component/s: Streaming Move MasterFailureTest out of main source directory

[jira] [Commented] (SPARK-2700) Hidden files (such as .impala_insert_staging) should be filtered out by sqlContext.parquetFile

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091263#comment-14091263 ] Michael Armbrust commented on SPARK-2700: - I actually just merged it. Thanks!

[jira] [Commented] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-08 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091259#comment-14091259 ] Kay Ousterhout commented on SPARK-2931: --- I tried doing something similar to

[jira] [Updated] (SPARK-2700) Hidden files (such as .impala_insert_staging) should be filtered out by sqlContext.parquetFile

2014-08-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2700: Target Version/s: 1.1.0 Fix Version/s: 1.1.0 Hidden files (such as

[jira] [Commented] (SPARK-2700) Hidden files (such as .impala_insert_staging) should be filtered out by sqlContext.parquetFile

2014-08-08 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091268#comment-14091268 ] Teng Qiu commented on SPARK-2700: - Oh, great, thanks :) Hidden files (such as

[jira] [Commented] (SPARK-1766) Move reduceByKey definitions next to each other in PairRDDFunctions

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091279#comment-14091279 ] Apache Spark commented on SPARK-1766: - User 'copester' has created a pull request for

[jira] [Updated] (SPARK-2916) [MLlib] While running regression tests with dense vectors of length greater than 1000, the treeAggregate blows up after several iterations

2014-08-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-2916: --- Description: While running any of the regression algorithms with gradient descent, the

[jira] [Updated] (SPARK-2916) [MLlib] While running regression tests with dense vectors of length greater than 1000, the treeAggregate blows up after several iterations

2014-08-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-2916: --- Component/s: Spark Core [MLlib] While running regression tests with dense vectors of length greater

[jira] [Commented] (SPARK-2678) `Spark-submit` overrides user application options

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091318#comment-14091318 ] Apache Spark commented on SPARK-2678: - User 'chutium' has created a pull request for

[jira] [Resolved] (SPARK-1997) Update breeze to version 0.9

2014-08-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1997. -- Resolution: Fixed Issue resolved by pull request 1857

[jira] [Updated] (SPARK-2706) Enable Spark to support Hive 0.13

2014-08-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2706: -- Attachment: hive.diff mvn -Phive -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package

[jira] [Updated] (SPARK-2706) Enable Spark to support Hive 0.13

2014-08-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2706: -- Attachment: (was: hive.diff) Enable Spark to support Hive 0.13 -

[jira] [Updated] (SPARK-2706) Enable Spark to support Hive 0.13

2014-08-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2706: -- Attachment: hive.diff Patch to the latest spark trunk. I only test with following compilation mvn

[jira] [Issue Comment Deleted] (SPARK-2706) Enable Spark to support Hive 0.13

2014-08-08 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2706: -- Comment: was deleted (was: mvn -Phive -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean

[jira] [Resolved] (SPARK-2851) Check API consistency for decision tree

2014-08-08 Thread Doris Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Xin resolved SPARK-2851. -- Resolution: Done Check API consistency for decision tree ---

[jira] [Created] (SPARK-2934) Adding LogisticRegressionWithLBFGS for training with LBFGS Optimizer

2014-08-08 Thread DB Tsai (JIRA)
DB Tsai created SPARK-2934: -- Summary: Adding LogisticRegressionWithLBFGS for training with LBFGS Optimizer Key: SPARK-2934 URL: https://issues.apache.org/jira/browse/SPARK-2934 Project: Spark

[jira] [Commented] (SPARK-2934) Adding LogisticRegressionWithLBFGS for training with LBFGS Optimizer

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091448#comment-14091448 ] Apache Spark commented on SPARK-2934: - User 'dbtsai' has created a pull request for

[jira] [Created] (SPARK-2935) Failure with push down of conjunctive parquet predicates

2014-08-08 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2935: --- Summary: Failure with push down of conjunctive parquet predicates Key: SPARK-2935 URL: https://issues.apache.org/jira/browse/SPARK-2935 Project: Spark

[jira] [Resolved] (SPARK-2897) org.apache.spark.broadcast.TorrentBroadcast does use the serializer class specified in the spark option spark.serializer

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2897. Resolution: Fixed Fix Version/s: 1.0.3 1.1.0

[jira] [Resolved] (SPARK-2928) TorrentBroadcast should use the user specified serializer

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2928. Resolution: Fixed Fix Version/s: 1.0.3 1.1.0 TorrentBroadcast should

[jira] [Resolved] (SPARK-2920) TorrentBroadcast does not support broadcast compression

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2920. Resolution: Fixed Fix Version/s: 1.0.3 1.1.0 TorrentBroadcast does not

[jira] [Commented] (SPARK-2935) Failure with push down of conjunctive parquet predicates

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091476#comment-14091476 ] Apache Spark commented on SPARK-2935: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-08-08 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091489#comment-14091489 ] Sandy Ryza commented on SPARK-2926: --- Hi Saisai, This seems like a very useful addition.

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

2014-08-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091514#comment-14091514 ] Ted Yu commented on SPARK-2706: --- Running Hive test, I got: {code} ^[[31m*** RUN ABORTED

[jira] [Commented] (SPARK-2894) spark-shell doesn't accept flags

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091531#comment-14091531 ] Apache Spark commented on SPARK-2894: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-2812) convert maven to archetype based build

2014-08-08 Thread Anand Avati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091561#comment-14091561 ] Anand Avati commented on SPARK-2812: According to

[jira] [Commented] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091581#comment-14091581 ] Josh Rosen commented on SPARK-2931: --- This isn't the easiest bug to reproduce. I tried

[jira] [Created] (SPARK-2936) Move Netty network module from Java to Scala

2014-08-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2936: -- Summary: Move Netty network module from Java to Scala Key: SPARK-2936 URL: https://issues.apache.org/jira/browse/SPARK-2936 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2936) Migrate Netty network module from Java to Scala

2014-08-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2936: --- Summary: Migrate Netty network module from Java to Scala (was: Move Netty network module from Java

[jira] [Commented] (SPARK-2936) Migrate Netty network module from Java to Scala

2014-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091654#comment-14091654 ] Apache Spark commented on SPARK-2936: - User 'rxin' has created a pull request for this

[jira] [Resolved] (SPARK-2635) Fix race condition at SchedulerBackend.isReady in standalone mode

2014-08-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2635. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1525

[jira] [Updated] (SPARK-2635) Fix race condition at SchedulerBackend.isReady in standalone mode

2014-08-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2635: --- Assignee: Zhihui Fix race condition at SchedulerBackend.isReady in standalone mode