[jira] [Commented] (SPARK-3061) Maven build fails in Windows OS

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098223#comment-14098223 ] Patrick Wendell commented on SPARK-3061: At this time, I'm not sure we intend to

[jira] [Created] (SPARK-3062) ShutdownHookManager is only available in Hadoop 2.x

2014-08-15 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3062: - Summary: ShutdownHookManager is only available in Hadoop 2.x Key: SPARK-3062 URL: https://issues.apache.org/jira/browse/SPARK-3062 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-2931) getAllowedLocalityLevel() throws ArrayIndexOutOfBoundsException

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2931. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Josh Rosen This was fixed

[jira] [Created] (SPARK-3063) ExistingRdd should convert Map to catalyst Map.

2014-08-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3063: Summary: ExistingRdd should convert Map to catalyst Map. Key: SPARK-3063 URL: https://issues.apache.org/jira/browse/SPARK-3063 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3063) ExistingRdd should convert Map to catalyst Map.

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098265#comment-14098265 ] Apache Spark commented on SPARK-3063: - User 'ueshin' has created a pull request for

[jira] [Created] (SPARK-3064) It would be very useful to specifies line terminate when use the textFile function

2014-08-15 Thread yangping wu (JIRA)
yangping wu created SPARK-3064: -- Summary: It would be very useful to specifies line terminate when use the textFile function Key: SPARK-3064 URL: https://issues.apache.org/jira/browse/SPARK-3064

[jira] [Created] (SPARK-3065) Add Locale setting to HiveCompatibilitySuite

2014-08-15 Thread luogankun (JIRA)
luogankun created SPARK-3065: Summary: Add Locale setting to HiveCompatibilitySuite Key: SPARK-3065 URL: https://issues.apache.org/jira/browse/SPARK-3065 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3012) Standardized Distance Functions between two Vectors for MLlib

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098312#comment-14098312 ] Apache Spark commented on SPARK-3012: - User 'yu-iskw' has created a pull request for

[jira] [Created] (SPARK-3066) Support recommendAll in matrix factorization model

2014-08-15 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3066: Summary: Support recommendAll in matrix factorization model Key: SPARK-3066 URL: https://issues.apache.org/jira/browse/SPARK-3066 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3061) Maven build fails in Windows OS

2014-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098343#comment-14098343 ] Sean Owen commented on SPARK-3061: -- At least, you almost certainly need to use Cygwin on

[jira] [Created] (SPARK-3067) JobProgressPage could not show Fair Scheduler Pools section sometimes

2014-08-15 Thread YanTang Zhai (JIRA)
YanTang Zhai created SPARK-3067: --- Summary: JobProgressPage could not show Fair Scheduler Pools section sometimes Key: SPARK-3067 URL: https://issues.apache.org/jira/browse/SPARK-3067 Project: Spark

[jira] [Created] (SPARK-3068) when run with jvm 1.8, should not set MaxPermSize

2014-08-15 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-3068: -- Summary: when run with jvm 1.8, should not set MaxPermSize Key: SPARK-3068 URL: https://issues.apache.org/jira/browse/SPARK-3068 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3068) when run with jvm 1.8, should not set MaxPermSize

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098368#comment-14098368 ] Apache Spark commented on SPARK-3068: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-3064) It would be very useful to specifies line terminate when use the textFile function

2014-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098369#comment-14098369 ] Sean Owen commented on SPARK-3064: -- Basically a duplicate of

[jira] [Commented] (SPARK-3068) when run with jvm 1.8, should not set MaxPermSize

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098376#comment-14098376 ] Apache Spark commented on SPARK-3068: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-3064) It would be very useful to specifies line terminate when use the textFile function

2014-08-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098388#comment-14098388 ] Cheng Lian commented on SPARK-3064: --- {{SparkContext.textFile}} uses {{TextInputFormat}}

[jira] [Commented] (SPARK-3065) Add Locale setting to HiveCompatibilitySuite

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098395#comment-14098395 ] Apache Spark commented on SPARK-3065: - User 'luogankun' has created a pull request for

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

2014-08-15 Thread Bertrand Bossy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098398#comment-14098398 ] Bertrand Bossy commented on SPARK-3039: --- Also need to update the README: See

[jira] [Commented] (SPARK-1861) ArrayIndexOutOfBoundsException when reading bzip2 files

2014-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098417#comment-14098417 ] Sean Owen commented on SPARK-1861: -- I don't think there is any Fix version since it was

[jira] [Commented] (SPARK-1861) ArrayIndexOutOfBoundsException when reading bzip2 files

2014-08-15 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098468#comment-14098468 ] sam commented on SPARK-1861: Thanks @srowen. So if I use org.apache.hadoop % hadoop-common %

[jira] [Comment Edited] (SPARK-1861) ArrayIndexOutOfBoundsException when reading bzip2 files

2014-08-15 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098468#comment-14098468 ] sam edited comment on SPARK-1861 at 8/15/14 11:28 AM: -- Thanks

[jira] [Commented] (SPARK-1861) ArrayIndexOutOfBoundsException when reading bzip2 files

2014-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098474#comment-14098474 ] Sean Owen commented on SPARK-1861: -- It matters what's running on your cluster. I believe

[jira] [Commented] (SPARK-1861) ArrayIndexOutOfBoundsException when reading bzip2 files

2014-08-15 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098477#comment-14098477 ] sam commented on SPARK-1861: OK, so what I need to do is ask my DevOps to upgrade our cluster

[jira] [Commented] (SPARK-2927) Add a conf to configure if we always read Binary columns stored in Parquet as String columns

2014-08-15 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098573#comment-14098573 ] Teng Qiu commented on SPARK-2927: - SPARK-2699 could also be closed, these two ticket are

[jira] [Commented] (SPARK-1828) Created forked version of hive-exec that doesn't bundle other dependencies

2014-08-15 Thread Maxim Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098656#comment-14098656 ] Maxim Ivanov commented on SPARK-1828: - Because of this change any incompatibilities

[jira] [Created] (SPARK-3070) Kry deserialization without using the custom registrator

2014-08-15 Thread Andras Nemeth (JIRA)
Andras Nemeth created SPARK-3070: Summary: Kry deserialization without using the custom registrator Key: SPARK-3070 URL: https://issues.apache.org/jira/browse/SPARK-3070 Project: Spark Issue

[jira] [Updated] (SPARK-3070) Kry deserialization without using the custom registrator

2014-08-15 Thread Andras Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Nemeth updated SPARK-3070: - Description: If an RDD partition is cached on executor1 and used by a task on executor2 then

[jira] [Updated] (SPARK-3070) Kryo deserialization without using the custom registrator

2014-08-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-3070: -- Summary: Kryo deserialization without using the custom registrator (was: Kry deserialization

[jira] [Created] (SPARK-3071) Increase default driver memory

2014-08-15 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3071: Summary: Increase default driver memory Key: SPARK-3071 URL: https://issues.apache.org/jira/browse/SPARK-3071 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

2014-08-15 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3072: Summary: Yarn AM not always properly exiting after unregistering from RM Key: SPARK-3072 URL: https://issues.apache.org/jira/browse/SPARK-3072 Project: Spark

[jira] [Resolved] (SPARK-2865) Potential deadlock: tasks could hang forever waiting to fetch a remote block even though most tasks finish

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2865. Resolution: Fixed I believe this has been resolved by virtue of other patches to the

[jira] [Resolved] (SPARK-2924) Remove use of default arguments where disallowed by 2.11

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2924. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1704

[jira] [Commented] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

2014-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098705#comment-14098705 ] Thomas Graves commented on SPARK-3072: -- Note that in yarn-cluster mode the client

[jira] [Created] (SPARK-3073) improve large sort (external sort)

2014-08-15 Thread Davies Liu (JIRA)
Davies Liu created SPARK-3073: - Summary: improve large sort (external sort) Key: SPARK-3073 URL: https://issues.apache.org/jira/browse/SPARK-3073 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3074) support groupByKey() with hot keys

2014-08-15 Thread Davies Liu (JIRA)
Davies Liu created SPARK-3074: - Summary: support groupByKey() with hot keys Key: SPARK-3074 URL: https://issues.apache.org/jira/browse/SPARK-3074 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3073) improve large sort (external sort)

2014-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098727#comment-14098727 ] Sean Owen commented on SPARK-3073: -- What does this refer to, and is it not the same as

[jira] [Commented] (SPARK-1828) Created forked version of hive-exec that doesn't bundle other dependencies

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098752#comment-14098752 ] Patrick Wendell commented on SPARK-1828: Maxim - I think what you are pointing out

[jira] [Commented] (SPARK-975) Spark Replay Debugger

2014-08-15 Thread Phuoc Do (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098758#comment-14098758 ] Phuoc Do commented on SPARK-975: Cheng Lian, I saw that latest UI displays stack trace for

[jira] [Commented] (SPARK-1828) Created forked version of hive-exec that doesn't bundle other dependencies

2014-08-15 Thread Maxim Ivanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098774#comment-14098774 ] Maxim Ivanov commented on SPARK-1828: - I don't have a pull request at hand if you are

[jira] [Updated] (SPARK-3073) improve large sort (external sort) for PySpark

2014-08-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3073: -- Summary: improve large sort (external sort) for PySpark (was: improve large sort (external sort))

[jira] [Commented] (SPARK-3062) ShutdownHookManager is only available in Hadoop 2.x

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098776#comment-14098776 ] Apache Spark commented on SPARK-3062: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098777#comment-14098777 ] Apache Spark commented on SPARK-2970: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-3073) improve large sort (external sort) for PySpark

2014-08-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098785#comment-14098785 ] Davies Liu commented on SPARK-3073: --- This is for PySpark, currently we do not support

[jira] [Updated] (SPARK-3074) support groupByKey() with hot keys in PySpark

2014-08-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3074: -- Summary: support groupByKey() with hot keys in PySpark (was: support groupByKey() with hot keys)

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098805#comment-14098805 ] Apache Spark commented on SPARK-2468: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-3046) Set executor's class loader as the default serializer class loader

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098823#comment-14098823 ] Apache Spark commented on SPARK-3046: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098824#comment-14098824 ] Josh Rosen commented on SPARK-922: -- Updated script, which also updates numpy: {code} yum

[jira] [Comment Edited] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098824#comment-14098824 ] Josh Rosen edited comment on SPARK-922 at 8/15/14 6:05 PM: ---

[jira] [Comment Edited] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098824#comment-14098824 ] Josh Rosen edited comment on SPARK-922 at 8/15/14 6:10 PM: ---

[jira] [Updated] (SPARK-3075) Expose a way for users to parse event logs

2014-08-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3075: - Description: Both ReplayListenerBus and util.JsonProtocol are private[spark], so the user wants to parse

[jira] [Created] (SPARK-3075) Expose a way for users to parse event logs

2014-08-15 Thread Andrew Or (JIRA)
Andrew Or created SPARK-3075: Summary: Expose a way for users to parse event logs Key: SPARK-3075 URL: https://issues.apache.org/jira/browse/SPARK-3075 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3075) Expose a way for users to parse event logs

2014-08-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3075: - Fix Version/s: 1.2.0 Expose a way for users to parse event logs

[jira] [Updated] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3028: --- Assignee: Sandy Ryza sparkEventToJson should support SparkListenerExecutorMetricsUpdate

[jira] [Resolved] (SPARK-3028) sparkEventToJson should support SparkListenerExecutorMetricsUpdate

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3028. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1961

[jira] [Resolved] (SPARK-2110) Misleading help displayed for interactive mode pyspark --help

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2110. --- Resolution: Fixed Fix Version/s: 1.1.0 I think this was fixed by SPARK-2678: these options

[jira] [Resolved] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2911. --- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Erik Erlandson Marking as 'fixed'

[jira] [Resolved] (SPARK-2717) BasicBlockFetchIterator#next should log when it gets stuck

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2717. --- Resolution: Won't Fix This is subsumed by the patch that adds timeouts to BasicBlockFetchIterator.

[jira] [Commented] (SPARK-3034) [HIve] java.sql.Date cannot be cast to java.sql.Timestamp

2014-08-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098985#comment-14098985 ] Michael Armbrust commented on SPARK-3034: - Can you provide the query? [HIve]

[jira] [Commented] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

2014-08-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098986#comment-14098986 ] Michael Armbrust commented on SPARK-3033: - Can you provide the query? [Hive]

[jira] [Updated] (SPARK-1477) Add the lifecycle interface

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1477: -- Target Version/s: 1.2.0 (was: 1.1.0) Retargeting this to 1.2.0. Add the lifecycle interface

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099049#comment-14099049 ] Nicholas Chammas commented on SPARK-922: Josh, at the end of your updated script do

[jira] [Created] (SPARK-3076) Gracefully report build timeouts in Jenkins

2014-08-15 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3076: --- Summary: Gracefully report build timeouts in Jenkins Key: SPARK-3076 URL: https://issues.apache.org/jira/browse/SPARK-3076 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

2014-08-15 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099072#comment-14099072 ] Mridul Muralidharan commented on SPARK-1476: Based on discussions we had with

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099089#comment-14099089 ] Josh Rosen commented on SPARK-922: -- Yeah, you still need to set PYSPARK_PYTHON since this

[jira] [Comment Edited] (SPARK-2858) Default log4j configuration no longer seems to work

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098148#comment-14098148 ] Patrick Wendell edited comment on SPARK-2858 at 8/15/14 8:48 PM:

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

2014-08-15 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099115#comment-14099115 ] Mridul Muralidharan commented on SPARK-2089: For a general case, wont

[jira] [Updated] (SPARK-3075) Expose a way for users to parse event logs

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3075: --- Target Version/s: 1.2.0 Fix Version/s: (was: 1.2.0) Expose a way for users to

[jira] [Updated] (SPARK-2532) Fix issues with consolidated shuffle

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2532: --- Target Version/s: 1.2.0 (was: 1.1.0) Fix issues with consolidated shuffle

[jira] [Updated] (SPARK-2977) Fix handling of short shuffle manager names in ShuffleBlockManager

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2977: --- Priority: Critical (was: Major) Fix handling of short shuffle manager names in

[jira] [Updated] (SPARK-2044) Pluggable interface for shuffles

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2044: --- Target Version/s: 1.2.0 (was: 1.1.0) Pluggable interface for shuffles

[jira] [Commented] (SPARK-2044) Pluggable interface for shuffles

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099136#comment-14099136 ] Patrick Wendell commented on SPARK-2044: A lot of this has been fixed in 1.1 so I

[jira] [Resolved] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

2014-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3022. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1950

[jira] [Updated] (SPARK-3022) FindBinsForLevel in decision tree should call findBin only once for each feature

2014-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3022: - Target Version/s: 1.1.0 (was: 1.0.2) FindBinsForLevel in decision tree should call findBin

[jira] [Resolved] (SPARK-3041) DecisionTree: isSampleValid indexing incorrect

2014-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3041. -- Resolution: Fixed Fix Version/s: 1.1.0 DecisionTree: isSampleValid indexing incorrect

[jira] [Updated] (SPARK-2585) Remove special handling of Hadoop JobConf

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2585: --- Target Version/s: 1.2.0 (was: 1.1.0) Remove special handling of Hadoop JobConf

[jira] [Updated] (SPARK-2546) Configuration object thread safety issue

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2546: --- Target Version/s: 1.2.0 (was: 1.1.0) Configuration object thread safety issue

[jira] [Commented] (SPARK-2585) Remove special handling of Hadoop JobConf

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099234#comment-14099234 ] Patrick Wendell commented on SPARK-2585: Unfortunately after a lot of effort we

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099239#comment-14099239 ] Patrick Wendell commented on SPARK-2546: Hey Andrew I think due to us cutting

[jira] [Updated] (SPARK-2914) spark.*.extraJavaOptions are evaluated too many times

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2914: --- Priority: Blocker (was: Major) spark.*.extraJavaOptions are evaluated too many times

[jira] [Updated] (SPARK-2914) spark.*.extraJavaOptions are evaluated too many times

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2914: --- Priority: Critical (was: Blocker) spark.*.extraJavaOptions are evaluated too many times

[jira] [Created] (SPARK-3077) ChiSqTest bugs

2014-08-15 Thread Doris Xin (JIRA)
Doris Xin created SPARK-3077: Summary: ChiSqTest bugs Key: SPARK-3077 URL: https://issues.apache.org/jira/browse/SPARK-3077 Project: Spark Issue Type: Bug Components: MLlib

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

2014-08-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099251#comment-14099251 ] Andrew Ash commented on SPARK-2546: --- Ok I'll stay on the lookout for this bug and ping

[jira] [Created] (SPARK-3078) Make LRWithLBFGS API consistent with others

2014-08-15 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-3078: Summary: Make LRWithLBFGS API consistent with others Key: SPARK-3078 URL: https://issues.apache.org/jira/browse/SPARK-3078 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3078) Make LRWithLBFGS API consistent with others

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099271#comment-14099271 ] Apache Spark commented on SPARK-3078: - User 'mengxr' has created a pull request for

[jira] [Updated] (SPARK-3025) Allow JDBC clients to set a fair scheduler pool

2014-08-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3025: --- Priority: Blocker (was: Major) Allow JDBC clients to set a fair scheduler pool

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-08-15 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099275#comment-14099275 ] Zhan Zhang commented on SPARK-2883: --- Spark with Hive12 can operate Orc table through

[jira] [Commented] (SPARK-3076) Gracefully report build timeouts in Jenkins

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099289#comment-14099289 ] Apache Spark commented on SPARK-3076: - User 'nchammas' has created a pull request for

[jira] [Updated] (SPARK-2406) Partitioned Parquet Support

2014-08-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2406: Target Version/s: 1.1.0 (was: 1.2.0) Partitioned Parquet Support

[jira] [Updated] (SPARK-2406) Partitioned Parquet Support

2014-08-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2406: Priority: Blocker (was: Critical) Partitioned Parquet Support

[jira] [Commented] (SPARK-3042) DecisionTree filtering is very inefficient

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099303#comment-14099303 ] Apache Spark commented on SPARK-3042: - User 'jkbradley' has created a pull request for

[jira] [Created] (SPARK-3079) Hive build should depend on parquet serdes

2014-08-15 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-3079: -- Summary: Hive build should depend on parquet serdes Key: SPARK-3079 URL: https://issues.apache.org/jira/browse/SPARK-3079 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2944) sc.makeRDD doesn't distribute partitions evenly

2014-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2944: - Priority: Major (was: Blocker) sc.makeRDD doesn't distribute partitions evenly

[jira] [Commented] (SPARK-2944) sc.makeRDD doesn't distribute partitions evenly

2014-08-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099317#comment-14099317 ] Xiangrui Meng commented on SPARK-2944: -- I changed the priority to Major because I

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

2014-08-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099333#comment-14099333 ] Reynold Xin commented on SPARK-1476: Let's work together to get something for 1.2 or

[jira] [Updated] (SPARK-3046) Set executor's class loader as the default serializer class loader

2014-08-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3046: --- Component/s: Spark Core Set executor's class loader as the default serializer class loader

[jira] [Created] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-08-15 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-3080: -- Summary: ArrayIndexOutOfBoundsException in ALS for Large datasets Key: SPARK-3080 URL: https://issues.apache.org/jira/browse/SPARK-3080 Project: Spark Issue

[jira] [Updated] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-08-15 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-3080: --- Description: The stack trace is below: {quote} java.lang.ArrayIndexOutOfBoundsException: 2716

[jira] [Updated] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-08-15 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-3080: --- Description: The stack trace is below: {quote} java.lang.ArrayIndexOutOfBoundsException: 2716

[jira] [Commented] (SPARK-3081) Rename RandomRDDGenerators to RandomRDDs

2014-08-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099449#comment-14099449 ] Apache Spark commented on SPARK-3081: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-1987) More memory-efficient graph construction

2014-08-15 Thread Larry Xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099476#comment-14099476 ] Larry Xiao commented on SPARK-1987: --- ok. I understand. I'll try to implement it More

  1   2   >