[jira] [Commented] (SPARK-2432) Apriori algorithm for frequent itemset mining

2014-11-04 Thread Varadharajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195879#comment-14195879 ] Varadharajan commented on SPARK-2432: - Do we have any updates on this? If its not

[jira] [Comment Edited] (SPARK-2468) Netty-based block server / client module

2014-11-04 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14189825#comment-14189825 ] zzc edited comment on SPARK-2468 at 11/4/14 8:37 AM: - Hi, Reynold Xin,

[jira] [Resolved] (SPARK-3002) Maintains a connection pool and reuse clients in BlockClientFactory

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3002. Resolution: Fixed Fix Version/s: 1.2.0 Maintains a connection pool and reuse clients in

[jira] [Resolved] (SPARK-3453) Refactor Netty module to use BlockTransferService

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3453. Resolution: Fixed Fix Version/s: 1.2.0 Refactor Netty module to use BlockTransferService

[jira] [Resolved] (SPARK-3049) Make sure client doesn't block when server/connection has error(s)

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3049. Resolution: Fixed Fix Version/s: 1.2.0 Make sure client doesn't block when

[jira] [Resolved] (SPARK-3016) Client should be able to put blocks in addition to fetch blocks

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3016. Resolution: Fixed Fix Version/s: 1.2.0 Client should be able to put blocks in addition to

[jira] [Resolved] (SPARK-3017) Implement unit/integration tests for connection failures

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3017. Resolution: Fixed Fix Version/s: 1.2.0 Implement unit/integration tests for connection

[jira] [Resolved] (SPARK-3503) Disable thread local cache in PooledByteBufAllocator

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3503. Resolution: Fixed Fix Version/s: 1.2.0 Disable thread local cache in PooledByteBufAllocator

[jira] [Resolved] (SPARK-3502) SO_RCVBUF and SO_SNDBUF should be bootstrap childOption, not option

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3502. Resolution: Fixed Fix Version/s: 1.2.0 SO_RCVBUF and SO_SNDBUF should be bootstrap

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195884#comment-14195884 ] Reynold Xin commented on SPARK-2468: Are you running on YARN? It seems like YARN just

[jira] [Resolved] (SPARK-2432) Apriori algorithm for frequent itemset mining

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2432. -- Resolution: Duplicate Resolving as duplicate of the later issue with more discussion. Apriori

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-04 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195952#comment-14195952 ] zzc commented on SPARK-2468: Yes, running on yarn client mode, but application is running, not

[jira] [Commented] (SPARK-3954) source code optimization

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195956#comment-14195956 ] Sean Owen commented on SPARK-3954: -- source code optimization is not a good JIRA title.

[jira] [Updated] (SPARK-3954) Optimization to FileInputDStream

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3954: - Summary: Optimization to FileInputDStream (was: source code optimization) Optimization to

[jira] [Commented] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196217#comment-14196217 ] Apache Spark commented on SPARK-3033: - User 'pengyanhong' has created a pull request

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-04 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196251#comment-14196251 ] zzc commented on SPARK-2468: If I use less data, it can run successfully, such as 24G snappy

[jira] [Updated] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-11-04 Thread Vitaliy Migov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitaliy Migov updated SPARK-4133: - Attachment: spark_ex.logs PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

[jira] [Updated] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

2014-11-04 Thread Vitaliy Migov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitaliy Migov updated SPARK-3958: - Attachment: spark_ex.logs Possible stream-corruption issues in TorrentBroadcast

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

2014-11-04 Thread Vitaliy Migov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196288#comment-14196288 ] Vitaliy Migov commented on SPARK-3958: -- Observed the same exception ( Spark 1.1.0 ).

[jira] [Created] (SPARK-4220) Spark New Feature

2014-11-04 Thread Tao Li (JIRA)
Tao Li created SPARK-4220: - Summary: Spark New Feature Key: SPARK-4220 URL: https://issues.apache.org/jira/browse/SPARK-4220 Project: Spark Issue Type: New Feature Reporter: Tao Li

[jira] [Created] (SPARK-4219) Spark New Feature

2014-11-04 Thread Tao Li (JIRA)
Tao Li created SPARK-4219: - Summary: Spark New Feature Key: SPARK-4219 URL: https://issues.apache.org/jira/browse/SPARK-4219 Project: Spark Issue Type: New Feature Reporter: Tao Li

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-11-04 Thread Vitaliy Migov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196386#comment-14196386 ] Vitaliy Migov commented on SPARK-4133: -- After additional investigation of this issue

[jira] [Commented] (SPARK-3694) Allow printing object graph of tasks/RDD's with a debug flag

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196411#comment-14196411 ] Apache Spark commented on SPARK-3694: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3964) Python API for Hypothesis testing

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196412#comment-14196412 ] Apache Spark commented on SPARK-3964: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-11-04 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196455#comment-14196455 ] Josh Rosen commented on SPARK-4133: --- Spark doesn't support multiple active SparkContexts

[jira] [Resolved] (SPARK-4060) MLlib, exposing special rdd functions to the public

2014-11-04 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4060. -- Resolution: Fixed Issue resolved by pull request 2907

[jira] [Comment Edited] (SPARK-4214) With dynamic allocation, avoid outstanding requests for more executors than pending tasks need

2014-11-04 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196464#comment-14196464 ] Sandy Ryza edited comment on SPARK-4214 at 11/4/14 6:00 PM: We

[jira] [Commented] (SPARK-4214) With dynamic allocation, avoid outstanding requests for more executors than pending tasks need

2014-11-04 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196464#comment-14196464 ] Sandy Ryza commented on SPARK-4214: --- We can implement this in either a weak way or a

[jira] [Comment Edited] (SPARK-4214) With dynamic allocation, avoid outstanding requests for more executors than pending tasks need

2014-11-04 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196464#comment-14196464 ] Sandy Ryza edited comment on SPARK-4214 at 11/4/14 6:00 PM: We

[jira] [Commented] (SPARK-2691) Allow Spark on Mesos to be launched with Docker

2014-11-04 Thread Tom Arnfeld (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196468#comment-14196468 ] Tom Arnfeld commented on SPARK-2691: [~ChrisHeller] I noticed there's also a reference

[jira] [Commented] (SPARK-3639) Kinesis examples set master as local

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196527#comment-14196527 ] Apache Spark commented on SPARK-3639: - User 'aniketbhatnagar' has created a pull

[jira] [Resolved] (SPARK-4220) Spark New Feature

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4220. -- Resolution: Invalid Given two empty JIRAs were opened, I assume these were accidental. Spark New

[jira] [Resolved] (SPARK-4219) Spark New Feature

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4219. -- Resolution: Invalid Given two empty JIRAs were opened, I assume these were accidental. Spark New

[jira] [Commented] (SPARK-3640) KinesisUtils should accept a credentials object instead of forcing DefaultCredentialsProvider

2014-11-04 Thread Aniket Bhatnagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196534#comment-14196534 ] Aniket Bhatnagar commented on SPARK-3640: - Thanks Chris for looking into this.

[jira] [Created] (SPARK-4222) FixedLengthBinaryRecordReader should readFully

2014-11-04 Thread Jascha Swisher (JIRA)
Jascha Swisher created SPARK-4222: - Summary: FixedLengthBinaryRecordReader should readFully Key: SPARK-4222 URL: https://issues.apache.org/jira/browse/SPARK-4222 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4196) Streaming + checkpointing yields NotSerializableException for Hadoop Configuration from saveAsNewAPIHadoopFiles ?

2014-11-04 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196641#comment-14196641 ] Cody Koeninger commented on SPARK-4196: --- Have you tried replacing

[jira] [Commented] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-04 Thread Venkata Ramana G (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196674#comment-14196674 ] Venkata Ramana G commented on SPARK-4217: - I have executed this on Hive and

[jira] [Created] (SPARK-4223) Support * (meaning all users) as part of the acls

2014-11-04 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-4223: Summary: Support * (meaning all users) as part of the acls Key: SPARK-4223 URL: https://issues.apache.org/jira/browse/SPARK-4223 Project: Spark Issue Type:

[jira] [Created] (SPARK-4224) Support group acls

2014-11-04 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-4224: Summary: Support group acls Key: SPARK-4224 URL: https://issues.apache.org/jira/browse/SPARK-4224 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4225) jdbc/odbc error when using maven build spark

2014-11-04 Thread wangfei (JIRA)
wangfei created SPARK-4225: -- Summary: jdbc/odbc error when using maven build spark Key: SPARK-4225 URL: https://issues.apache.org/jira/browse/SPARK-4225 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4225) jdbc/odbc error when using maven build spark

2014-11-04 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196693#comment-14196693 ] wangfei commented on SPARK-4225: it seems there is some difference between using sbt and

[jira] [Comment Edited] (SPARK-4225) jdbc/odbc error when using maven build spark

2014-11-04 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196693#comment-14196693 ] wangfei edited comment on SPARK-4225 at 11/4/14 7:46 PM: - it seems

[jira] [Created] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2014-11-04 Thread Terry Siu (JIRA)
Terry Siu created SPARK-4226: Summary: SparkSQL - Add support for subqueries in predicates Key: SPARK-4226 URL: https://issues.apache.org/jira/browse/SPARK-4226 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4196) Streaming + checkpointing yields NotSerializableException for Hadoop Configuration from saveAsNewAPIHadoopFiles ?

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196762#comment-14196762 ] Sean Owen commented on SPARK-4196: -- Same problem I'm afraid. The serialization error

[jira] [Created] (SPARK-4227) Document external shuffle service

2014-11-04 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-4227: - Summary: Document external shuffle service Key: SPARK-4227 URL: https://issues.apache.org/jira/browse/SPARK-4227 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4100) JSON RDD schema inference causes whole RDD to be realized

2014-11-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196852#comment-14196852 ] Yin Huai commented on SPARK-4100: - I am not sure I understand the description correctly.

[jira] [Created] (SPARK-4228) Save a ScheamRDD in JSON format

2014-11-04 Thread Yin Huai (JIRA)
Yin Huai created SPARK-4228: --- Summary: Save a ScheamRDD in JSON format Key: SPARK-4228 URL: https://issues.apache.org/jira/browse/SPARK-4228 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-4229) Create hadoop configuration in a consistent way

2014-11-04 Thread Cody Koeninger (JIRA)
Cody Koeninger created SPARK-4229: - Summary: Create hadoop configuration in a consistent way Key: SPARK-4229 URL: https://issues.apache.org/jira/browse/SPARK-4229 Project: Spark Issue Type:

[jira] [Updated] (SPARK-4229) Create hadoop configuration in a consistent way

2014-11-04 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-4229: -- Description: Some places use SparkHadoopUtil.get.conf, some create a new hadoop config.

[jira] [Created] (SPARK-4230) Doc for spark.default.parallelism is incorrect

2014-11-04 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-4230: - Summary: Doc for spark.default.parallelism is incorrect Key: SPARK-4230 URL: https://issues.apache.org/jira/browse/SPARK-4230 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4230) Doc for spark.default.parallelism is incorrect

2014-11-04 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4230: -- Description: The default default parallelism for shuffle transformations is actually the maximum

[jira] [Commented] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-04 Thread peter.zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197281#comment-14197281 ] peter.zhang commented on SPARK-4217: It's impossible I tested this SQL both in my dev

[jira] [Comment Edited] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-04 Thread peter.zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197281#comment-14197281 ] peter.zhang edited comment on SPARK-4217 at 11/5/14 1:17 AM: -

[jira] [Created] (SPARK-4231) Add RankingMetrics to examples.MovieLensALS

2014-11-04 Thread Debasish Das (JIRA)
Debasish Das created SPARK-4231: --- Summary: Add RankingMetrics to examples.MovieLensALS Key: SPARK-4231 URL: https://issues.apache.org/jira/browse/SPARK-4231 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4231) Add RankingMetrics to examples.MovieLensALS

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197315#comment-14197315 ] Apache Spark commented on SPARK-4231: - User 'debasish83' has created a pull request

[jira] [Created] (SPARK-4232) Truncate table not works when specific the table from non-current database session

2014-11-04 Thread shengli (JIRA)
shengli created SPARK-4232: -- Summary: Truncate table not works when specific the table from non-current database session Key: SPARK-4232 URL: https://issues.apache.org/jira/browse/SPARK-4232 Project: Spark

[jira] [Commented] (SPARK-3530) Pipeline and Parameters

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197338#comment-14197338 ] Apache Spark commented on SPARK-3530: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-3936) Incorrect result in GraphX BytecodeUtils with closures + class/object methods

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197351#comment-14197351 ] Apache Spark commented on SPARK-3936: - User 'ankurdave' has created a pull request for

[jira] [Updated] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-04 Thread peter.zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peter.zhang updated SPARK-4217: --- Attachment: (was: TestScript.sql) Result of SparkSQL is incorrect after a table join and group

[jira] [Created] (SPARK-4233) Simplify the Aggregation Function implementation

2014-11-04 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-4233: Summary: Simplify the Aggregation Function implementation Key: SPARK-4233 URL: https://issues.apache.org/jira/browse/SPARK-4233 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-04 Thread shengli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197386#comment-14197386 ] shengli commented on SPARK-4217: I also test the script both on pure-hive and spark-sql.

[jira] [Created] (SPARK-4234) Always do paritial aggregation

2014-11-04 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-4234: Summary: Always do paritial aggregation Key: SPARK-4234 URL: https://issues.apache.org/jira/browse/SPARK-4234 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4235) Add union data type support

2014-11-04 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-4235: Summary: Add union data type support Key: SPARK-4235 URL: https://issues.apache.org/jira/browse/SPARK-4235 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4236) External shuffle service must cleanup its shuffle files

2014-11-04 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-4236: - Summary: External shuffle service must cleanup its shuffle files Key: SPARK-4236 URL: https://issues.apache.org/jira/browse/SPARK-4236 Project: Spark

[jira] [Created] (SPARK-4237) add Manifest File for Maven building

2014-11-04 Thread wangfei (JIRA)
wangfei created SPARK-4237: -- Summary: add Manifest File for Maven building Key: SPARK-4237 URL: https://issues.apache.org/jira/browse/SPARK-4237 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-4238) Perform network-level retry of shuffle file fetches

2014-11-04 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-4238: - Summary: Perform network-level retry of shuffle file fetches Key: SPARK-4238 URL: https://issues.apache.org/jira/browse/SPARK-4238 Project: Spark Issue

[jira] [Commented] (SPARK-4238) Perform network-level retry of shuffle file fetches

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197417#comment-14197417 ] Apache Spark commented on SPARK-4238: - User 'aarondav' has created a pull request for

[jira] [Commented] (SPARK-4100) JSON RDD schema inference causes whole RDD to be realized

2014-11-04 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197445#comment-14197445 ] Kuldeep commented on SPARK-4100: Sorry if I was not clear. Yes, this is what i meant. Is

[jira] [Commented] (SPARK-4237) add Manifest File for Maven building

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197452#comment-14197452 ] Apache Spark commented on SPARK-4237: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-4225) jdbc/odbc error when using maven build spark

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197453#comment-14197453 ] Apache Spark commented on SPARK-4225: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-4174) Streaming: Optionally provide notifications to Receivers when DStream has been generated

2014-11-04 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197464#comment-14197464 ] Hari Shreedharan commented on SPARK-4174: - I will write up a doc soon, but here

[jira] [Commented] (SPARK-4197) Gradient Boosting API cleanups

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197622#comment-14197622 ] Apache Spark commented on SPARK-4197: - User 'jkbradley' has created a pull request for

[jira] [Resolved] (SPARK-3964) Python API for Hypothesis testing

2014-11-04 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3964. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3091

[jira] [Commented] (SPARK-4148) PySpark's sample uses the same seed for all partitions

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197645#comment-14197645 ] Apache Spark commented on SPARK-4148: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-4225) jdbc/odbc error when using maven build spark

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197647#comment-14197647 ] Apache Spark commented on SPARK-4225: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-4148) PySpark's sample uses the same seed for all partitions

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197648#comment-14197648 ] Apache Spark commented on SPARK-4148: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0

2014-11-04 Thread Vitaliy Migov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197655#comment-14197655 ] Vitaliy Migov commented on SPARK-4133: -- The problem is that if we mistakenly pass

[jira] [Created] (SPARK-4239) support view in HiveQL

2014-11-04 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-4239: -- Summary: support view in HiveQL Key: SPARK-4239 URL: https://issues.apache.org/jira/browse/SPARK-4239 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-4239) support view in HiveQL

2014-11-04 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-4239: --- Component/s: SQL support view in HiveQL -- Key: SPARK-4239

[jira] [Updated] (SPARK-3984) Display task deserialization time in the UI

2014-11-04 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-3984: -- Description: Right now, the UI does not display the time to deserialize the task, which can be

[jira] [Commented] (SPARK-4237) add Manifest File for Maven building

2014-11-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197698#comment-14197698 ] Sean Owen commented on SPARK-4237: -- How does the PR address this? the manifest file is

[jira] [Commented] (SPARK-4237) add Manifest File for Maven building

2014-11-04 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197715#comment-14197715 ] wangfei commented on SPARK-4237: The title is not correct, should be generate right

[jira] [Updated] (SPARK-4237) Generate right Manifest File for maven building

2014-11-04 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-4237: --- Summary: Generate right Manifest File for maven building (was: add Manifest File for Maven building)

[jira] [Created] (SPARK-4240) Refine Tree Predictions in Gradient Boosting to Improve Prediction Accuracy.

2014-11-04 Thread Sung Chung (JIRA)
Sung Chung created SPARK-4240: - Summary: Refine Tree Predictions in Gradient Boosting to Improve Prediction Accuracy. Key: SPARK-4240 URL: https://issues.apache.org/jira/browse/SPARK-4240 Project: Spark

[jira] [Commented] (SPARK-4230) Doc for spark.default.parallelism is incorrect

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197794#comment-14197794 ] Apache Spark commented on SPARK-4230: - User 'sryza' has created a pull request for

[jira] [Updated] (SPARK-4240) Refine Tree Predictions in Gradient Boosting to Improve Prediction Accuracy.

2014-11-04 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4240: - Fix Version/s: (was: 1.3.0) Refine Tree Predictions in Gradient Boosting to Improve

[jira] [Commented] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-04 Thread Venkata Ramana G (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197799#comment-14197799 ] Venkata Ramana G commented on SPARK-4217: - I executed them on Hive 0.12 (from Hive

[jira] [Created] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2014-11-04 Thread Haitao Yao (JIRA)
Haitao Yao created SPARK-4241: - Summary: spark_ec2.py support China AWS region: cn-north-1 Key: SPARK-4241 URL: https://issues.apache.org/jira/browse/SPARK-4241 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4241) spark_ec2.py support China AWS region: cn-north-1

2014-11-04 Thread Haitao Yao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197803#comment-14197803 ] Haitao Yao commented on SPARK-4241: --- In order to see region: cn-north-1 , you will have

[jira] [Created] (SPARK-4242) Add SASL to external shuffle service

2014-11-04 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-4242: - Summary: Add SASL to external shuffle service Key: SPARK-4242 URL: https://issues.apache.org/jira/browse/SPARK-4242 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4242) Add SASL to external shuffle service

2014-11-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197808#comment-14197808 ] Apache Spark commented on SPARK-4242: - User 'aarondav' has created a pull request for