[jira] [Resolved] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13050. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10954 [https://github.

[jira] [Commented] (SPARK-13074) Add getPersistentRDDs() API to JavaSparkContext

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123111#comment-15123111 ] Apache Spark commented on SPARK-13074: -- User 'flyjy' has created a pull request for

[jira] [Assigned] (SPARK-13074) Add getPersistentRDDs() API to JavaSparkContext

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13074: Assignee: (was: Apache Spark) > Add getPersistentRDDs() API to JavaSparkContext >

[jira] [Assigned] (SPARK-13074) Add getPersistentRDDs() API to JavaSparkContext

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13074: Assignee: Apache Spark > Add getPersistentRDDs() API to JavaSparkContext > ---

[jira] [Created] (SPARK-13074) Add getPersistentRDDs() API to JavaSparkContext

2016-01-28 Thread Junyang Shen (JIRA)
Junyang Shen created SPARK-13074: Summary: Add getPersistentRDDs() API to JavaSparkContext Key: SPARK-13074 URL: https://issues.apache.org/jira/browse/SPARK-13074 Project: Spark Issue Type: N

[jira] [Updated] (SPARK-12913) Reimplement stat functions as declarative function

2016-01-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12913: --- Summary: Reimplement stat functions as declarative function (was: Reimplement stddev/variance as dec

[jira] [Resolved] (SPARK-13067) DataFrameSuite.simple explode fail locally

2016-01-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13067. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10970 [https://github.

[jira] [Commented] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123057#comment-15123057 ] Apache Spark commented on SPARK-13031: -- User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-12968) Implement command to set current database

2016-01-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12968. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 > Implement comm

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

2016-01-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123043#comment-15123043 ] Saisai Shao commented on SPARK-3374: +1 to address this in the upcoming 2.0 release.

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

2016-01-28 Thread Jayadevan M (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123023#comment-15123023 ] Jayadevan M commented on SPARK-12982: - I tried your scenario in spark 1.6.0. But I ca

[jira] [Commented] (SPARK-5891) Add Binarizer

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123017#comment-15123017 ] Apache Spark commented on SPARK-5891: - User 'seddonm1' has created a pull request for

[jira] [Commented] (SPARK-13072) simplify and improve murmur3 hash expression codegen

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122959#comment-15122959 ] Apache Spark commented on SPARK-13072: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-13072) simplify and improve murmur3 hash expression codegen

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13072: Assignee: (was: Apache Spark) > simplify and improve murmur3 hash expression codegen >

[jira] [Assigned] (SPARK-13072) simplify and improve murmur3 hash expression codegen

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13072: Assignee: Apache Spark > simplify and improve murmur3 hash expression codegen > --

[jira] [Created] (SPARK-13073) creating R like summary for logistic Regression in Spark - Scala

2016-01-28 Thread Samsudhin (JIRA)
Samsudhin created SPARK-13073: - Summary: creating R like summary for logistic Regression in Spark - Scala Key: SPARK-13073 URL: https://issues.apache.org/jira/browse/SPARK-13073 Project: Spark I

[jira] [Issue Comment Deleted] (SPARK-12746) ArrayType(_, true) should also accept ArrayType(_, false)

2016-01-28 Thread Earthson Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earthson Lu updated SPARK-12746: Comment: was deleted (was: Hi Joseph, what is the status of nullability now? It seems someone has

[jira] [Commented] (SPARK-12746) ArrayType(_, true) should also accept ArrayType(_, false)

2016-01-28 Thread Earthson Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122870#comment-15122870 ] Earthson Lu commented on SPARK-12746: - Hi Joseph, what is the status of nullability n

[jira] [Commented] (SPARK-12746) ArrayType(_, true) should also accept ArrayType(_, false)

2016-01-28 Thread Earthson Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122871#comment-15122871 ] Earthson Lu commented on SPARK-12746: - Hi Joseph, what is the status of nullability n

[jira] [Created] (SPARK-13072) simplify and improve murmur3 hash expression codegen

2016-01-28 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-13072: --- Summary: simplify and improve murmur3 hash expression codegen Key: SPARK-13072 URL: https://issues.apache.org/jira/browse/SPARK-13072 Project: Spark Issue Type

[jira] [Assigned] (SPARK-13071) Coalescing HadoopRDD overwrites existing input metrics

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13071: Assignee: Apache Spark (was: Andrew Or) > Coalescing HadoopRDD overwrites existing input

[jira] [Commented] (SPARK-13071) Coalescing HadoopRDD overwrites existing input metrics

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122824#comment-15122824 ] Apache Spark commented on SPARK-13071: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-13071) Coalescing HadoopRDD overwrites existing input metrics

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13071: Assignee: Andrew Or (was: Apache Spark) > Coalescing HadoopRDD overwrites existing input

[jira] [Updated] (SPARK-13071) Coalescing HadoopRDD overwrites existing input metrics

2016-01-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13071: -- Description: Currently if we do the following in Hadoop 2.5+: {code} sc.textFile(..., 4).coalesce(2).co

[jira] [Created] (SPARK-13071) Coalescing HadoopRDD overwrites existing input metrics

2016-01-28 Thread Andrew Or (JIRA)
Andrew Or created SPARK-13071: - Summary: Coalescing HadoopRDD overwrites existing input metrics Key: SPARK-13071 URL: https://issues.apache.org/jira/browse/SPARK-13071 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4846) When the vocabulary size is large, Word2Vec may yield "OutOfMemoryError: Requested array size exceeds VM limit"

2016-01-28 Thread Joseph Tang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122775#comment-15122775 ] Joseph Tang commented on SPARK-4846: Hi Tung, As far as I can remember, the data is s

[jira] [Updated] (SPARK-13070) Points out which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13070: --- Summary: Points out which physical file is the trouble maker when Parquet schema merging fails (was:

[jira] [Assigned] (SPARK-13070) Points which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13070: Assignee: Apache Spark (was: Cheng Lian) > Points which physical file is the trouble make

[jira] [Commented] (SPARK-13070) Points which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122752#comment-15122752 ] Apache Spark commented on SPARK-13070: -- User 'liancheng' has created a pull request

[jira] [Assigned] (SPARK-13070) Points which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13070: Assignee: Cheng Lian (was: Apache Spark) > Points which physical file is the trouble make

[jira] [Created] (SPARK-13070) Points which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13070: -- Summary: Points which physical file is the trouble maker when Parquet schema merging fails Key: SPARK-13070 URL: https://issues.apache.org/jira/browse/SPARK-13070 Project

[jira] [Updated] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13050: --- Assignee: Alex Bozarth > Scalatest tags fail builds with the addition of the sketch module >

[jira] [Commented] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122723#comment-15122723 ] Reynold Xin commented on SPARK-13031: - What happened with the commit? If you revert a

[jira] [Commented] (SPARK-12196) Store/retrieve blocks in different speed storage devices by hierarchy way

2016-01-28 Thread yucai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122684#comment-15122684 ] yucai commented on SPARK-12196: --- Thanks Holden. Usually, customers have 7-11 HDDs installed

[jira] [Assigned] (SPARK-12913) Reimplement stddev/variance as declarative function

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12913: Assignee: Apache Spark > Reimplement stddev/variance as declarative function > ---

[jira] [Assigned] (SPARK-12913) Reimplement stddev/variance as declarative function

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12913: Assignee: (was: Apache Spark) > Reimplement stddev/variance as declarative function >

[jira] [Commented] (SPARK-12913) Reimplement stddev/variance as declarative function

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122645#comment-15122645 ] Apache Spark commented on SPARK-12913: -- User 'davies' has created a pull request for

[jira] [Commented] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2016-01-28 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122631#comment-15122631 ] Shivaram Venkataraman commented on SPARK-12526: --- Yeah this fix was checked

[jira] [Commented] (SPARK-12526) `ifelse`, `when`, `otherwise` unable to take Column as value

2016-01-28 Thread Deborah Siegel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122611#comment-15122611 ] Deborah Siegel commented on SPARK-12526: I'm trying to use ifelse (or when and ot

[jira] [Resolved] (SPARK-11955) Mark one side fields in merging schema for safely pushdowning filters in parquet

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11955. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9940 [https://github.c

[jira] [Updated] (SPARK-11955) Mark one side fields in merging schema for safely pushdowning filters in parquet

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11955: --- Assignee: Liang-Chi Hsieh > Mark one side fields in merging schema for safely pushdowning filters in

[jira] [Created] (SPARK-13069) ActorHelper is not throttled by rate limiter

2016-01-28 Thread Lin Zhao (JIRA)
Lin Zhao created SPARK-13069: Summary: ActorHelper is not throttled by rate limiter Key: SPARK-13069 URL: https://issues.apache.org/jira/browse/SPARK-13069 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13055) SQLHistoryListener throws ClassCastException

2016-01-28 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13055: -- Component/s: SQL > SQLHistoryListener throws ClassCastException > -

[jira] [Commented] (SPARK-13066) Specify types for per-model/estimator params in ML to allow automatic type conversion

2016-01-28 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122567#comment-15122567 ] holdenk commented on SPARK-13066: - See https://issues.apache.org/jira/browse/SPARK-13068

[jira] [Commented] (SPARK-13055) SQLHistoryListener throws ClassCastException

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122545#comment-15122545 ] Apache Spark commented on SPARK-13055: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-13055) SQLHistoryListener throws ClassCastException

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13055: Assignee: Andrew Or (was: Apache Spark) > SQLHistoryListener throws ClassCastException >

[jira] [Assigned] (SPARK-13055) SQLHistoryListener throws ClassCastException

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13055: Assignee: Apache Spark (was: Andrew Or) > SQLHistoryListener throws ClassCastException >

[jira] [Commented] (SPARK-13068) Extend pyspark ml paramtype conversion to support lists

2016-01-28 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122536#comment-15122536 ] holdenk commented on SPARK-13068: - cc [~sethah] > Extend pyspark ml paramtype conversion

[jira] [Created] (SPARK-13068) Extend pyspark ml paramtype conversion to support lists

2016-01-28 Thread holdenk (JIRA)
holdenk created SPARK-13068: --- Summary: Extend pyspark ml paramtype conversion to support lists Key: SPARK-13068 URL: https://issues.apache.org/jira/browse/SPARK-13068 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-12944) CrossValidator doesn't accept a Pipeline as an estimator

2016-01-28 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122518#comment-15122518 ] Seth Hendrickson commented on SPARK-12944: -- Wanted to update - There are two pa

[jira] [Resolved] (SPARK-12749) Spark SQL JSON schema infernce should allow floating-point numbers as BigDecimal

2016-01-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12749. - Resolution: Fixed Assignee: Brandon Bradley Fix Version/s: 2.0.0 > Spark SQL JSON

[jira] [Resolved] (SPARK-12401) Add support for enums in postgres

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12401. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10596 [https://github.

[jira] [Updated] (SPARK-12401) Add support for enums in postgres

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12401: --- Assignee: Takeshi Yamamuro > Add support for enums in postgres > - >

[jira] [Updated] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13050: --- Description: Builds fail at the new sketch module when a scalatest tag is used. Found when using "-D

[jira] [Commented] (SPARK-13066) Specify types for per-model/estimator params in ML to allow automatic type conversion

2016-01-28 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122503#comment-15122503 ] Seth Hendrickson commented on SPARK-13066: -- Posted this on a [previous PR|https

[jira] [Assigned] (SPARK-13067) DataFrameSuite.simple explode fail locally

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13067: Assignee: (was: Apache Spark) > DataFrameSuite.simple explode fail locally > -

[jira] [Assigned] (SPARK-13067) DataFrameSuite.simple explode fail locally

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13067: Assignee: Apache Spark > DataFrameSuite.simple explode fail locally >

[jira] [Commented] (SPARK-13067) DataFrameSuite.simple explode fail locally

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122501#comment-15122501 ] Apache Spark commented on SPARK-13067: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-13067) DataFrameSuite.simple explode fail locally

2016-01-28 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-13067: --- Summary: DataFrameSuite.simple explode fail locally Key: SPARK-13067 URL: https://issues.apache.org/jira/browse/SPARK-13067 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6328) Python API for StreamingListener

2016-01-28 Thread Daniel Jalova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122473#comment-15122473 ] Daniel Jalova commented on SPARK-6328: -- [~apachespark] Hi, can you change the assigne

[jira] [Issue Comment Deleted] (SPARK-6328) Python API for StreamingListener

2016-01-28 Thread Daniel Jalova (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Jalova updated SPARK-6328: - Comment: was deleted (was: @tdas Could you change the assignee to me please?) > Python API for St

[jira] [Resolved] (SPARK-9835) Iteratively reweighted least squares solver for GLMs

2016-01-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9835. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10639 [https://gith

[jira] [Resolved] (SPARK-12926) SQLContext to display warning message when non-sql configs are being set

2016-01-28 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-12926. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10849 [htt

[jira] [Resolved] (SPARK-13031) Improve test coverage for whole stage codegen

2016-01-28 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13031. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10944 [https://github.

[jira] [Created] (SPARK-13066) Specify types for per-model/estimator params in ML to allow automatic type conversion

2016-01-28 Thread holdenk (JIRA)
holdenk created SPARK-13066: --- Summary: Specify types for per-model/estimator params in ML to allow automatic type conversion Key: SPARK-13066 URL: https://issues.apache.org/jira/browse/SPARK-13066 Project:

[jira] [Commented] (SPARK-10009) PySpark Param of Vector type can be set with Python array or numpy.array

2016-01-28 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122358#comment-15122358 ] holdenk commented on SPARK-10009: - We could handle this the param type conversion code th

[jira] [Created] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

2016-01-28 Thread Andrew Davidson (JIRA)
Andrew Davidson created SPARK-13065: --- Summary: streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream() Key: SPARK-13065 URL: https://issues.apache.org/jira/browse/SPARK-13065

[jira] [Commented] (SPARK-9301) collect_set and collect_list aggregate functions

2016-01-28 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122291#comment-15122291 ] Herman van Hovell commented on SPARK-9301: -- You could implement this as an {{Impe

[jira] [Commented] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

2016-01-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122187#comment-15122187 ] Shixiong Zhu commented on SPARK-12739: -- [~jlaskowski] could you submit a PR soon? Sh

[jira] [Commented] (SPARK-13039) Spark Streaming with Mesos shutdown without any reason on logs

2016-01-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122166#comment-15122166 ] Shixiong Zhu commented on SPARK-13039: -- May be killed by Mesos because exceeding the

[jira] [Commented] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

2016-01-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122169#comment-15122169 ] Shixiong Zhu commented on SPARK-13027: -- Could you submit a PR? > Add API for update

[jira] [Commented] (SPARK-12727) SQL generation support for distinct aggregation patterns that fit DistinctAggregationRewriter analysis rule

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122124#comment-15122124 ] Cheng Lian commented on SPARK-12727: It would be nice if we can recover the original

[jira] [Commented] (SPARK-12723) Comprehensive SQL generation support for expressions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122102#comment-15122102 ] Cheng Lian commented on SPARK-12723: Most of this issue is probably fixed in SPARK-12

[jira] [Updated] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12720: --- Description: {{HiveCompatibilitySuite}} can be useful for bootstrapping test coverage. Please refer

[jira] [Updated] (SPARK-12721) SQL generation support for script transformation

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12721: --- Description: {{HiveCompatibilitySuite}} can be useful for bootstrapping test coverage. Please refer

[jira] [Updated] (SPARK-12719) SQL generation support for generators (including UDTF)

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12719: --- Description: {{HiveCompatibilitySuite}} can be useful for bootstrapping test coverage. Please refer

[jira] [Updated] (SPARK-12718) SQL generation support for window functions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12718: --- Description: {{HiveWindowFunctionQuerySuite}} and {{HiveWindowFunctionQueryFileSuite}} can be useful

[jira] [Updated] (SPARK-11012) Canonicalize view definitions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11012: --- Description: In SPARK-10337, we added the first step of supporting view natively, which is basically

[jira] [Commented] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122086#comment-15122086 ] Marcelo Vanzin commented on SPARK-13064: The real reason it's not set is because

[jira] [Assigned] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13053: Assignee: Andrew Or (was: Apache Spark) > Rectify ignored tests in InternalAccumulatorSui

[jira] [Assigned] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13053: Assignee: Apache Spark (was: Andrew Or) > Rectify ignored tests in InternalAccumulatorSui

[jira] [Commented] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

2016-01-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122081#comment-15122081 ] Apache Spark commented on SPARK-13053: -- User 'andrewor14' has created a pull request

[jira] [Commented] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122077#comment-15122077 ] Thomas Graves commented on SPARK-13064: --- thanks [~vanzin] We can just have the res

[jira] [Updated] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuo Liu updated SPARK-13064: - Issue Type: Improvement (was: Bug) > api/v1/application/jobs/attempt lacks "attempId" field for spark-sh

[jira] [Commented] (SPARK-10251) Some internal spark classes are not registered with kryo

2016-01-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122076#comment-15122076 ] Shixiong Zhu commented on SPARK-10251: -- Current KryoSerializer doesn't provide an ap

[jira] [Updated] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuo Liu updated SPARK-13064: - Description: For any application launches with spark-shell will not have attemptId field in their rest A

[jira] [Updated] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Zhuo Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuo Liu updated SPARK-13064: - Description: For any application launches with spark-shell will not have attemptId field in their rest A

[jira] [Commented] (SPARK-12196) Store/retrieve blocks in different speed storage devices by hierarchy way

2016-01-28 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122073#comment-15122073 ] holdenk commented on SPARK-12196: - This sounds interesting, I've definitely run into clus

[jira] [Updated] (SPARK-11012) Canonicalize view definitions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11012: --- Description: In SPARK-10337, we added the first step of supporting view natively, which is basically

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122043#comment-15122043 ] Cheng Lian commented on SPARK-12725: Thanks, this also sounds good to me. Will try th

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-28 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122016#comment-15122016 ] Michael Armbrust commented on SPARK-12725: -- Why don't we just add a flag to Attr

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122013#comment-15122013 ] Cheng Lian commented on SPARK-12725: One possible solution I was thinking about is th

[jira] [Commented] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121972#comment-15121972 ] Marcelo Vanzin commented on SPARK-13064: IIRC no application launched in client m

[jira] [Created] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Zhuo Liu (JIRA)
Zhuo Liu created SPARK-13064: Summary: api/v1/application/jobs/attempt lacks "attempId" field for spark-shell Key: SPARK-13064 URL: https://issues.apache.org/jira/browse/SPARK-13064 Project: Spark

[jira] [Commented] (SPARK-9301) collect_set and collect_list aggregate functions

2016-01-28 Thread Cristian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121968#comment-15121968 ] Cristian commented on SPARK-9301: - Seconded, looks like MutableAggregationBuffer is not so

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

2016-01-28 Thread Andres Ivaldi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121822#comment-15121822 ] Andres Ivaldi commented on SPARK-10528: --- You are right, I didn't pay attention to t

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

2016-01-28 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121792#comment-15121792 ] Michel Lemay commented on SPARK-10528: -- Your issue is about deleting temp files at t

[jira] [Assigned] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-01-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1239: Assignee: Thomas Graves > Don't fetch all map output statuses at each reducer during shuffl

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

2016-01-28 Thread Andres Ivaldi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121608#comment-15121608 ] Andres Ivaldi commented on SPARK-10528: --- Ok, this issue is about the same exception

[jira] [Commented] (SPARK-13062) Overwriting same file with new schema destroys original file.

2016-01-28 Thread Vincent Warmerdam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121590#comment-15121590 ] Vincent Warmerdam commented on SPARK-13062: --- Target version -> my bad. Apologi

  1   2   >