[jira] [Commented] (SPARK-4122) Add library to write data back to Kafka

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188052#comment-14188052 ] Apache Spark commented on SPARK-4122: - User 'harishreedharan' has created a pull

[jira] [Created] (SPARK-4130) loadLibSVMFile does not handle extra whitespace

2014-10-29 Thread Joseph E. Gonzalez (JIRA)
Joseph E. Gonzalez created SPARK-4130: - Summary: loadLibSVMFile does not handle extra whitespace Key: SPARK-4130 URL: https://issues.apache.org/jira/browse/SPARK-4130 Project: Spark

[jira] [Updated] (SPARK-4130) loadLibSVMFile does not handle extra whitespace

2014-10-29 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-4130: -- Description: When testing MLlib on the splice site data

[jira] [Updated] (SPARK-4130) loadLibSVMFile does not handle extra whitespace

2014-10-29 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-4130: -- Description: When testing MLlib on the splice site data

[jira] [Updated] (SPARK-4130) loadLibSVMFile does not handle extra whitespace

2014-10-29 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-4130: -- Description: When testing MLlib on the splice site data

[jira] [Updated] (SPARK-4130) loadLibSVMFile does not handle extra whitespace

2014-10-29 Thread Joseph E. Gonzalez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph E. Gonzalez updated SPARK-4130: -- Description: When testing MLlib on the splice site data

[jira] [Commented] (SPARK-4124) Simplify serialization and call API in MLlib Python

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188104#comment-14188104 ] Apache Spark commented on SPARK-4124: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188105#comment-14188105 ] Davies Liu commented on SPARK-3683: --- [~jamborta] It seems that this is a feature, not a

[jira] [Commented] (SPARK-4130) loadLibSVMFile does not handle extra whitespace

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188110#comment-14188110 ] Apache Spark commented on SPARK-4130: - User 'jegonzal' has created a pull request for

[jira] [Updated] (SPARK-1442) Add Window function support

2014-10-29 Thread guowei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guowei updated SPARK-1442: -- Attachment: (was: Window Function.pdf) Add Window function support ---

[jira] [Updated] (SPARK-1442) Add Window function support

2014-10-29 Thread guowei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guowei updated SPARK-1442: -- Attachment: Window Function.pdf Add Window function support ---

[jira] [Updated] (SPARK-4131) support “Writing data into the filesystem from queries”

2014-10-29 Thread XiaoJing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoJing wang updated SPARK-4131: - Summary: support “Writing data into the filesystem from queries” (was: support “insert overwrite

[jira] [Updated] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread XiaoJing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoJing wang updated SPARK-4131: - Summary: Support Writing data into the filesystem from queries (was: support “Writing data into

[jira] [Commented] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread Ravindra Pesala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188147#comment-14188147 ] Ravindra Pesala commented on SPARK-4131: I will work on this issue. Support

[jira] [Updated] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread XiaoJing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoJing wang updated SPARK-4131: - Description: Writing data into the filesystem from queries,SparkSql is not support . eg: pre

[jira] [Created] (SPARK-4132) Spark uses incompatible HDFS API

2014-10-29 Thread kuromatsu nobuyuki (JIRA)
kuromatsu nobuyuki created SPARK-4132: - Summary: Spark uses incompatible HDFS API Key: SPARK-4132 URL: https://issues.apache.org/jira/browse/SPARK-4132 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread XiaoJing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoJing wang updated SPARK-4131: - Description: Writing data into the filesystem from queries,SparkSql is not support . eg: pre

[jira] [Updated] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread XiaoJing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoJing wang updated SPARK-4131: - Description: Writing data into the filesystem from queries,SparkSql is not support . eg:

[jira] [Commented] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188158#comment-14188158 ] Apache Spark commented on SPARK-4131: - User 'wangxiaojing' has created a pull request

[jira] [Updated] (SPARK-4131) Support Writing data into the filesystem from queries

2014-10-29 Thread XiaoJing wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoJing wang updated SPARK-4131: - Description: Writing data into the filesystem from queries,SparkSql is not support . eg:

[jira] [Resolved] (SPARK-4132) Spark uses incompatible HDFS API

2014-10-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4132. -- Resolution: Duplicate I'm all but certain you're describing the same thing as SPARK-4078 Spark uses

[jira] [Commented] (SPARK-683) Spark 0.7 with Hadoop 1.0 does not work with current AMI's HDFS installation

2014-10-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188160#comment-14188160 ] Sean Owen commented on SPARK-683: - PS I think this also turns out to be the same as

[jira] [Commented] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-29 Thread Tamas Jambor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188185#comment-14188185 ] Tamas Jambor commented on SPARK-3683: - Thanks for the comments. From my perspective

[jira] [Commented] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188194#comment-14188194 ] Cheng Lian commented on SPARK-3683: --- [~jamborta] Your concern is legitimate. However,

[jira] [Closed] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-29 Thread Tamas Jambor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Jambor closed SPARK-3683. --- Resolution: Not a Problem PySpark Hive query generates NULL instead of None

[jira] [Commented] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-29 Thread Tamas Jambor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188227#comment-14188227 ] Tamas Jambor commented on SPARK-3683: - OK, makes sense. Thanks. PySpark Hive query

[jira] [Created] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-10-29 Thread Antonio Jesus Navarro (JIRA)
Antonio Jesus Navarro created SPARK-4133: Summary: PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0 Key: SPARK-4133 URL: https://issues.apache.org/jira/browse/SPARK-4133 Project:

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-10-29 Thread Antonio Jesus Navarro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188284#comment-14188284 ] Antonio Jesus Navarro commented on SPARK-4133: -- Existing Spark Streaming app

[jira] [Updated] (SPARK-2429) Hierarchical Implementation of KMeans

2014-10-29 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Ishikawa updated SPARK-2429: --- Attachment: benchmark-result.2014-10-29.html I added a new performance test results named

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

2014-10-29 Thread RJ Nowling (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188325#comment-14188325 ] RJ Nowling commented on SPARK-2429: --- The sparsity tests look good. Have you compared

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-10-29 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188354#comment-14188354 ] Ilya Ganelin commented on SPARK-3080: - Hello Xiangrui - happy to hear that you're on

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-29 Thread Michael Griffiths (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188357#comment-14188357 ] Michael Griffiths commented on SPARK-3398: -- Hi Nicholas, Thanks for the thorough

[jira] [Comment Edited] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-29 Thread Michael Griffiths (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188357#comment-14188357 ] Michael Griffiths edited comment on SPARK-3398 at 10/29/14 1:58 PM:

[jira] [Created] (SPARK-4134) Tone down scary executor lost messages when killing on purpose

2014-10-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-4134: Summary: Tone down scary executor lost messages when killing on purpose Key: SPARK-4134 URL: https://issues.apache.org/jira/browse/SPARK-4134 Project: Spark Issue

[jira] [Commented] (SPARK-3182) Twitter Streaming Geoloaction Filter

2014-10-29 Thread Brennon York (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188544#comment-14188544 ] Brennon York commented on SPARK-3182: - Hey all, looking to contribute back to Spark :)

[jira] [Resolved] (SPARK-4129) Performance tuning in MultivariateOnlineSummarizer

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4129. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2992

[jira] [Updated] (SPARK-4129) Performance tuning in MultivariateOnlineSummarizer

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4129: - Assignee: DB Tsai Performance tuning in MultivariateOnlineSummarizer

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-10-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188665#comment-14188665 ] Josh Rosen commented on SPARK-4133: --- Since you mentioned that you see a similar issue

[jira] [Updated] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

2014-10-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3958: -- Affects Version/s: 1.1.0 Adding 1.1.0 as an affected version, since a user has observed this in 1.1.0,

[jira] [Updated] (SPARK-4081) Categorical feature indexing

2014-10-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4081: - Description: DecisionTree and RandomForest require that categorical features and labels

[jira] [Resolved] (SPARK-4003) Add {Big Decimal, Timestamp, Date} types to Java SqlContext

2014-10-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4003. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2850

[jira] [Commented] (SPARK-4081) Categorical feature indexing

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188810#comment-14188810 ] Apache Spark commented on SPARK-4081: - User 'jkbradley' has created a pull request for

[jira] [Updated] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3080: - Target Version/s: 1.2.0 Affects Version/s: 1.1.0 ArrayIndexOutOfBoundsException in ALS for

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-10-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188815#comment-14188815 ] Josh Rosen commented on SPARK-4133: --- Also, can you paste more of the log leading up to

[jira] [Created] (SPARK-4135) Error reading Parquet file generated with SparkSQL

2014-10-29 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-4135: - Summary: Error reading Parquet file generated with SparkSQL Key: SPARK-4135 URL: https://issues.apache.org/jira/browse/SPARK-4135 Project: Spark Issue

[jira] [Updated] (SPARK-4135) Error reading Parquet file generated with SparkSQL

2014-10-29 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-4135: -- Attachment: _metadata part-r-1.parquet Files generated by SparkSQL that cannot

[jira] [Created] (SPARK-4136) Under dynamic allocation, cancel outstanding executor requests when pending task queue is empty

2014-10-29 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-4136: - Summary: Under dynamic allocation, cancel outstanding executor requests when pending task queue is empty Key: SPARK-4136 URL: https://issues.apache.org/jira/browse/SPARK-4136

[jira] [Updated] (SPARK-4136) Under dynamic allocation, cancel outstanding executor requests when pending task queue is empty

2014-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4136: - Affects Version/s: (was: 1.1.0) 1.2.0 Under dynamic allocation, cancel

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2014-10-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188910#comment-14188910 ] Josh Rosen commented on SPARK-3630: --- *Decompression errors during shuffle fetching*: If

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2014-10-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188916#comment-14188916 ] Josh Rosen commented on SPARK-4105: --- It seems plausible that SPARK-4107 could have

[jira] [Commented] (SPARK-3573) Dataset

2014-10-29 Thread Evan Sparks (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188919#comment-14188919 ] Evan Sparks commented on SPARK-3573: This comment originally appeared on the PR

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188928#comment-14188928 ] Xiangrui Meng commented on SPARK-3080: -- SimpleALS is not merged yet. You need to

[jira] [Created] (SPARK-4137) Relative paths don't get handled correctly by spark-ec2

2014-10-29 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-4137: --- Summary: Relative paths don't get handled correctly by spark-ec2 Key: SPARK-4137 URL: https://issues.apache.org/jira/browse/SPARK-4137 Project: Spark

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-10-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188938#comment-14188938 ] Nicholas Chammas commented on SPARK-3398: - No problem. I've opened [SPARK-4137] to

[jira] [Commented] (SPARK-3796) Create shuffle service for external block storage

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188947#comment-14188947 ] Apache Spark commented on SPARK-3796: - User 'aarondav' has created a pull request for

[jira] [Created] (SPARK-4138) Guard against incompatible settings on the number of executors

2014-10-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-4138: Summary: Guard against incompatible settings on the number of executors Key: SPARK-4138 URL: https://issues.apache.org/jira/browse/SPARK-4138 Project: Spark Issue

[jira] [Created] (SPARK-4139) Start the number of executors at the max if dynamic allocation is enabled

2014-10-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-4139: Summary: Start the number of executors at the max if dynamic allocation is enabled Key: SPARK-4139 URL: https://issues.apache.org/jira/browse/SPARK-4139 Project: Spark

[jira] [Commented] (SPARK-4139) Start the number of executors at the max if dynamic allocation is enabled

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188961#comment-14188961 ] Apache Spark commented on SPARK-4139: - User 'andrewor14' has created a pull request

[jira] [Commented] (SPARK-4138) Guard against incompatible settings on the number of executors

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188960#comment-14188960 ] Apache Spark commented on SPARK-4138: - User 'andrewor14' has created a pull request

[jira] [Created] (SPARK-4140) Document the dynamic allocation feature

2014-10-29 Thread Andrew Or (JIRA)
Andrew Or created SPARK-4140: Summary: Document the dynamic allocation feature Key: SPARK-4140 URL: https://issues.apache.org/jira/browse/SPARK-4140 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3573) Dataset

2014-10-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188980#comment-14188980 ] Joseph K. Bradley commented on SPARK-3573: -- [~sparks] Trying to simplify things,

[jira] [Closed] (SPARK-3822) Expose a mechanism for SparkContext to ask for / remove Yarn containers

2014-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3822. Resolution: Fixed Fix Version/s: 1.2.0 Expose a mechanism for SparkContext to ask for / remove Yarn

[jira] [Closed] (SPARK-4126) Do not set `spark.executor.instances` if not needed (yarn-cluster)

2014-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4126. Resolution: Won't Fix superseded by SPARK-4138 Do not set `spark.executor.instances` if not needed

[jira] [Commented] (SPARK-3466) Limit size of results that a driver collects for each action

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188998#comment-14188998 ] Apache Spark commented on SPARK-3466: - User 'davies' has created a pull request for

[jira] [Created] (SPARK-4141) Hide Accumulators column on stage page when no accumulators exist

2014-10-29 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-4141: - Summary: Hide Accumulators column on stage page when no accumulators exist Key: SPARK-4141 URL: https://issues.apache.org/jira/browse/SPARK-4141 Project: Spark

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-10-29 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189014#comment-14189014 ] Josh Rosen commented on SPARK-4133: --- Also, could you enable debug logging and share the

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-10-29 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189039#comment-14189039 ] Ilya Ganelin commented on SPARK-3080: - Hi all - I have managed to make some

[jira] [Resolved] (SPARK-4097) Race condition in org.apache.spark.ComplexFutureAction.cancel

2014-10-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-4097. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Shixiong

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189093#comment-14189093 ] Xiangrui Meng commented on SPARK-3080: -- Btw, the `ArrayIndexOutOfBoundsException` is

[jira] [Created] (SPARK-4142) Bad Default for GraphLoader Edge Partitions

2014-10-29 Thread Joseph E. Gonzalez (JIRA)
Joseph E. Gonzalez created SPARK-4142: - Summary: Bad Default for GraphLoader Edge Partitions Key: SPARK-4142 URL: https://issues.apache.org/jira/browse/SPARK-4142 Project: Spark Issue

[jira] [Commented] (SPARK-2672) Support compression in wholeFile()

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189320#comment-14189320 ] Apache Spark commented on SPARK-2672: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-4142) Bad Default for GraphLoader Edge Partitions

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189347#comment-14189347 ] Apache Spark commented on SPARK-4142: - User 'jegonzal' has created a pull request for

[jira] [Commented] (SPARK-4132) Spark uses incompatible HDFS API

2014-10-29 Thread kuromatsu nobuyuki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189351#comment-14189351 ] kuromatsu nobuyuki commented on SPARK-4132: --- Owen, thank you for your

[jira] [Created] (SPARK-4143) Move inner class DeferredObjectAdapter to top level

2014-10-29 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-4143: Summary: Move inner class DeferredObjectAdapter to top level Key: SPARK-4143 URL: https://issues.apache.org/jira/browse/SPARK-4143 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4143) Move inner class DeferredObjectAdapter to top level

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189380#comment-14189380 ] Apache Spark commented on SPARK-4143: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-4144) Support incremental model training of Naive Bayes classifier

2014-10-29 Thread Chris Fregly (JIRA)
Chris Fregly created SPARK-4144: --- Summary: Support incremental model training of Naive Bayes classifier Key: SPARK-4144 URL: https://issues.apache.org/jira/browse/SPARK-4144 Project: Spark

[jira] [Closed] (SPARK-3795) Add scheduler hooks/heuristics for adding and removing executors

2014-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3795. Resolution: Fixed Fix Version/s: 1.2.0 Add scheduler hooks/heuristics for adding and removing

[jira] [Created] (SPARK-4145) Create jobs overview and job details pages on the web UI

2014-10-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4145: - Summary: Create jobs overview and job details pages on the web UI Key: SPARK-4145 URL: https://issues.apache.org/jira/browse/SPARK-4145 Project: Spark Issue Type:

[jira] [Closed] (SPARK-4053) Block generator throttling in NetworkReceiverSuite is flaky

2014-10-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4053. Resolution: Fixed Fix Version/s: 1.2.0 Block generator throttling in NetworkReceiverSuite is flaky

[jira] [Commented] (SPARK-4145) Create jobs overview and job details pages on the web UI

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189429#comment-14189429 ] Apache Spark commented on SPARK-4145: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-10-29 Thread Jason Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dai updated SPARK-2926: - Assignee: Saisai Shao Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[jira] [Updated] (SPARK-4094) checkpoint should still be available after rdd actions

2014-10-29 Thread Jason Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dai updated SPARK-4094: - Assignee: Zhang, Liye checkpoint should still be available after rdd actions

[jira] [Updated] (SPARK-4078) New FsPermission instance w/o FsPermission.createImmutable in eventlog

2014-10-29 Thread Jason Dai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dai updated SPARK-4078: - Assignee: Jason Dai New FsPermission instance w/o FsPermission.createImmutable in eventlog

[jira] [Created] (SPARK-4146) [GraphX] Modify option name according to example doc in SynthBenchmark

2014-10-29 Thread Jie Huang (JIRA)
Jie Huang created SPARK-4146: Summary: [GraphX] Modify option name according to example doc in SynthBenchmark Key: SPARK-4146 URL: https://issues.apache.org/jira/browse/SPARK-4146 Project: Spark

[jira] [Updated] (SPARK-4146) [GraphX] Modify option name according to example doc in SynthBenchmark

2014-10-29 Thread Jie Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Huang updated SPARK-4146: - Affects Version/s: 1.1.1 [GraphX] Modify option name according to example doc in SynthBenchmark

[jira] [Resolved] (SPARK-4146) [GraphX] Modify option name according to example doc in SynthBenchmark

2014-10-29 Thread Jie Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Huang resolved SPARK-4146. -- Resolution: Fixed [GraphX] Modify option name according to example doc in SynthBenchmark

[jira] [Updated] (SPARK-4144) Support incremental model training of Naive Bayes classifier

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4144: - Assignee: Liquan Pei Support incremental model training of Naive Bayes classifier

[jira] [Updated] (SPARK-4146) [GraphX] Modify option name according to example doc in SynthBenchmark

2014-10-29 Thread Jie Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Huang updated SPARK-4146: - Fix Version/s: 1.2.0 [GraphX] Modify option name according to example doc in SynthBenchmark

[jira] [Created] (SPARK-4148) PySpark's sample uses the same seed for all partitions

2014-10-29 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-4148: Summary: PySpark's sample uses the same seed for all partitions Key: SPARK-4148 URL: https://issues.apache.org/jira/browse/SPARK-4148 Project: Spark Issue

[jira] [Updated] (SPARK-4148) PySpark's sample uses the same seed for all partitions

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4148: - Affects Version/s: (was: 1.0.0) 1.0.2 PySpark's sample uses the same

[jira] [Commented] (SPARK-4148) PySpark's sample uses the same seed for all partitions

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189579#comment-14189579 ] Apache Spark commented on SPARK-4148: - User 'mengxr' has created a pull request for

[jira] [Updated] (SPARK-4148) PySpark's sample uses the same seed for all partitions

2014-10-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4148: - Description: The current way of seed distribution makes the random sequences from partition i

[jira] [Created] (SPARK-4150) rdd.setName returns None in PySpark

2014-10-29 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-4150: Summary: rdd.setName returns None in PySpark Key: SPARK-4150 URL: https://issues.apache.org/jira/browse/SPARK-4150 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4150) rdd.setName returns None in PySpark

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189592#comment-14189592 ] Apache Spark commented on SPARK-4150: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-4149) ISO 8601 support for json date time strings

2014-10-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189618#comment-14189618 ] Apache Spark commented on SPARK-4149: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

2014-10-29 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189651#comment-14189651 ] Zhan Zhang commented on SPARK-1537: --- Hi Marcelo, Do you have update on this? If you