[jira] [Created] (SPARK-13188) There is OMM when a receiver stores block

2016-02-04 Thread KaiXinXIaoLei (JIRA)
KaiXinXIaoLei created SPARK-13188: - Summary: There is OMM when a receiver stores block Key: SPARK-13188 URL: https://issues.apache.org/jira/browse/SPARK-13188 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-13183) Bytebuffers occupy a large amount of heap memory

2016-02-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13183. --- Resolution: Not A Problem I don't think that's a problem per se unless you can establish some kind

[jira] [Resolved] (SPARK-13181) Spark delay in task scheduling within executor

2016-02-04 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph resolved SPARK-13181. --- Resolution: Not A Problem > Spark delay in task scheduling within executor >

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

2016-02-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132191#comment-15132191 ] Sean Owen commented on SPARK-12807: --- I don't think we can park at 2.2.x. Spark already uses 2.5 >

[jira] [Commented] (SPARK-13059) Sort inputsplits by size in HadoopRDD to avoid long tails

2016-02-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132200#comment-15132200 ] Sean Owen commented on SPARK-13059: --- The splits have a meaningful ordering for some input formats that

[jira] [Commented] (SPARK-13181) Spark delay in task scheduling within executor

2016-02-04 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132115#comment-15132115 ] Prabhu Joseph commented on SPARK-13181: --- Okay, the reason for the task delay within executor when

[jira] [Updated] (SPARK-13188) There is OMM when a receiver stores block

2016-02-04 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-13188: -- Description: 2016-02-04 15:13:27,349 | ERROR | [Thread-35] | Uncaught exception in thread

[jira] [Assigned] (SPARK-13136) Data exchange (shuffle, broadcast) should only be handled by the exchange operator

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13136: Assignee: Apache Spark > Data exchange (shuffle, broadcast) should only be handled by the

[jira] [Commented] (SPARK-13186) Migrate away from SynchronizedMap which is derpecated

2016-02-04 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133101#comment-15133101 ] Huaxin Gao commented on SPARK-13186: Hi Holden, I will work on this one. Thanks! > Migrate away

[jira] [Commented] (SPARK-13205) SQL generation support for self join

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133289#comment-15133289 ] Apache Spark commented on SPARK-13205: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13205) SQL generation support for self join

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13205: Assignee: Apache Spark > SQL generation support for self join >

[jira] [Assigned] (SPARK-13205) SQL generation support for self join

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13205: Assignee: (was: Apache Spark) > SQL generation support for self join >

[jira] [Assigned] (SPARK-13136) Data exchange (shuffle, broadcast) should only be handled by the exchange operator

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13136: Assignee: (was: Apache Spark) > Data exchange (shuffle, broadcast) should only be

[jira] [Commented] (SPARK-13136) Data exchange (shuffle, broadcast) should only be handled by the exchange operator

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133104#comment-15133104 ] Apache Spark commented on SPARK-13136: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Commented] (SPARK-13202) Jars specified with --jars do not exist on the worker classpath.

2016-02-04 Thread Michael Schmitz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133142#comment-15133142 ] Michael Schmitz commented on SPARK-13202: - This issue sounds very similar to what was reported on

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

2016-02-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133045#comment-15133045 ] Marcelo Vanzin commented on SPARK-12807: bq. I don't think we can park at 2.2.x. Spark already

[jira] [Commented] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

2016-02-04 Thread Aaditya Ramesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133087#comment-15133087 ] Aaditya Ramesh commented on SPARK-13027: [~zsxwing] Sorry for the late response, definitely, I

[jira] [Created] (SPARK-13204) Replace use of mutable.SynchronizedMap with ConcurrentHashMap

2016-02-04 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13204: -- Summary: Replace use of mutable.SynchronizedMap with ConcurrentHashMap Key: SPARK-13204 URL: https://issues.apache.org/jira/browse/SPARK-13204 Project: Spark Issue

[jira] [Commented] (SPARK-13116) TungstenAggregate though it is supposedly capable of all processing unsafe & safe rows, fails if the input is safe rows

2016-02-04 Thread Asif Hussain Shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133061#comment-15133061 ] Asif Hussain Shahid commented on SPARK-13116: - It seems our code of using relation provider

[jira] [Commented] (SPARK-13068) Extend pyspark ml paramtype conversion to support lists

2016-02-04 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133285#comment-15133285 ] Seth Hendrickson commented on SPARK-13068: -- Upon closer examination, I think the best way to

[jira] [Commented] (SPARK-13155) add runtime null check when convert catalyst array to external array

2016-02-04 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133324#comment-15133324 ] Huaxin Gao commented on SPARK-13155:

[jira] [Assigned] (SPARK-13203) Add scalastyle rule banning use of mutable.SynchronizedBuffer

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13203: Assignee: Apache Spark > Add scalastyle rule banning use of mutable.SynchronizedBuffer >

[jira] [Commented] (SPARK-12141) Use Jackson to serialize all events when writing event log

2016-02-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133048#comment-15133048 ] Marcelo Vanzin commented on SPARK-12141:

[jira] [Commented] (SPARK-13203) Add scalastyle rule banning use of mutable.SynchronizedBuffer

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133080#comment-15133080 ] Apache Spark commented on SPARK-13203: -- User 'ted-yu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13203) Add scalastyle rule banning use of mutable.SynchronizedBuffer

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13203: Assignee: (was: Apache Spark) > Add scalastyle rule banning use of

[jira] [Created] (SPARK-13203) Add scalastyle rule banning use of mutable.SynchronizedBuffer

2016-02-04 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13203: -- Summary: Add scalastyle rule banning use of mutable.SynchronizedBuffer Key: SPARK-13203 URL: https://issues.apache.org/jira/browse/SPARK-13203 Project: Spark Issue

[jira] [Created] (SPARK-13205) SQL generation support for self join

2016-02-04 Thread Xiao Li (JIRA)
Xiao Li created SPARK-13205: --- Summary: SQL generation support for self join Key: SPARK-13205 URL: https://issues.apache.org/jira/browse/SPARK-13205 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-13187) Add boolean/long/double type option functions in DataFrameReader/Writer

2016-02-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13187. - Resolution: Fixed Fix Version/s: 2.0.0 > Add boolean/long/double type option functions in

[jira] [Commented] (SPARK-13196) Optimize the option and flatten in Word2Vec to reduce the max memory consumption

2016-02-04 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133548#comment-15133548 ] yuhao yang commented on SPARK-13196: https://github.com/apache/spark/pull/11078 > Optimize the

[jira] [Commented] (SPARK-13185) Improve the performance of DateTimeUtils by reusing TimeZone and Calendar objects

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133771#comment-15133771 ] Apache Spark commented on SPARK-13185: -- User 'carsonwang' has created a pull request for this issue:

[jira] [Updated] (SPARK-13185) Improve the performance of DateTimeUtils by reusing Calendar objects

2016-02-04 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-13185: Summary: Improve the performance of DateTimeUtils by reusing Calendar objects (was: Improve the

[jira] [Resolved] (SPARK-13208) Replace Pair with tuples

2016-02-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13208. - Resolution: Fixed Assignee: Jakob Odersky Fix Version/s: 2.0.0 > Replace Pair

[jira] [Created] (SPARK-13210) NPE in Sort

2016-02-04 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13210: -- Summary: NPE in Sort Key: SPARK-13210 URL: https://issues.apache.org/jira/browse/SPARK-13210 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133764#comment-15133764 ] Xiao Li commented on SPARK-12720: - Almost done. Need to add more test cases to ensure it covers all the

[jira] [Updated] (SPARK-13185) Improve the performance of DateTimeUtils.StringToDate by reusing Calendar objects

2016-02-04 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-13185: Summary: Improve the performance of DateTimeUtils.StringToDate by reusing Calendar objects (was:

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-04 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133363#comment-15133363 ] Xusen Yin commented on SPARK-13178: --- Yes, it works, we can use read.json to load a DataFrame that

[jira] [Resolved] (SPARK-12850) Support bucket pruning (predicate pushdown for bucketed tables)

2016-02-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12850. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 > Support bucket

[jira] [Assigned] (SPARK-13207) _SUCCESS should not break partition discovery

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13207: Assignee: Apache Spark (was: Yin Huai) > _SUCCESS should not break partition discovery >

[jira] [Commented] (SPARK-13207) _SUCCESS should not break partition discovery

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133540#comment-15133540 ] Apache Spark commented on SPARK-13207: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13207) _SUCCESS should not break partition discovery

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13207: Assignee: Yin Huai (was: Apache Spark) > _SUCCESS should not break partition discovery >

[jira] [Comment Edited] (SPARK-13155) add runtime null check when convert catalyst array to external array

2016-02-04 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133324#comment-15133324 ] Huaxin Gao edited comment on SPARK-13155 at 2/4/16 11:54 PM: - [~cloud_fan]

[jira] [Commented] (SPARK-13206) Subquery Alias in Hive Parser

2016-02-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133520#comment-15133520 ] Liang-Chi Hsieh commented on SPARK-13206: - I tried your query. Looks like it can be parsed. The

[jira] [Comment Edited] (SPARK-13206) Subquery Alias in Hive Parser

2016-02-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133520#comment-15133520 ] Liang-Chi Hsieh edited comment on SPARK-13206 at 2/5/16 2:14 AM: - I tried

[jira] [Closed] (SPARK-13206) Subquery Alias in Hive Parser

2016-02-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-13206. --- Resolution: Cannot Reproduce > Subquery Alias in Hive Parser > - > >

[jira] [Commented] (SPARK-13206) Subquery Alias in Hive Parser

2016-02-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133528#comment-15133528 ] Xiao Li commented on SPARK-13206: - True... I am unable to reproduce it. Thank you! Let me close it. >

[jira] [Updated] (SPARK-13157) ADD JAR command cannot handle path with @ character

2016-02-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13157: --- Assignee: Herman van Hovell > ADD JAR command cannot handle path with @ character >

[jira] [Created] (SPARK-13207) _SUCCESS should not break partition discovery

2016-02-04 Thread Yin Huai (JIRA)
Yin Huai created SPARK-13207: Summary: _SUCCESS should not break partition discovery Key: SPARK-13207 URL: https://issues.apache.org/jira/browse/SPARK-13207 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13207) _SUCCESS should not break partition discovery

2016-02-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133529#comment-15133529 ] Yin Huai commented on SPARK-13207: -- It will be better to let partitioning discovery ignore files/dirs

[jira] [Updated] (SPARK-13208) Replace Pair with tuples

2016-02-04 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated SPARK-13208: -- Priority: Trivial (was: Major) > Replace Pair with tuples > > >

[jira] [Updated] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-02-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13052: -- Assignee: Raafat Akkad > waitingApps metric doesn't show the number of apps currently in the WAITING

[jira] [Resolved] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

2016-02-04 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13052. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > waitingApps metric

[jira] [Updated] (SPARK-13207) _SUCCESS should not break partition discovery

2016-02-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-13207: - Description: Partitioning discovery will fail with the following case {code} test("_SUCCESS should not

[jira] [Assigned] (SPARK-13179) pyspark row name collision 'count'

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13179: Assignee: Apache Spark > pyspark row name collision 'count' >

[jira] [Updated] (SPARK-12957) Derive and propagate data constrains in logical plan

2016-02-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12957: - Fix Version/s: (was: 2.0.0) > Derive and propagate data constrains in logical plan >

[jira] [Commented] (SPARK-12957) Derive and propagate data constrains in logical plan

2016-02-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133546#comment-15133546 ] Yin Huai commented on SPARK-12957: -- Update: https://github.com/apache/spark/pull/10844 is for

[jira] [Commented] (SPARK-13196) Optimize the option and flatten in Word2Vec to reduce the max memory consumption

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133547#comment-15133547 ] Apache Spark commented on SPARK-13196: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13196) Optimize the option and flatten in Word2Vec to reduce the max memory consumption

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13196: Assignee: (was: Apache Spark) > Optimize the option and flatten in Word2Vec to reduce

[jira] [Issue Comment Deleted] (SPARK-13196) Optimize the option and flatten in Word2Vec to reduce the max memory consumption

2016-02-04 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-13196: --- Comment: was deleted (was: https://github.com/apache/spark/pull/11078) > Optimize the option and

[jira] [Commented] (SPARK-13090) Add initial support for constraint propagation in SparkSQL

2016-02-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133549#comment-15133549 ] Yin Huai commented on SPARK-13090: -- Issue resolved by pull request 10844

[jira] [Assigned] (SPARK-13196) Optimize the option and flatten in Word2Vec to reduce the max memory consumption

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13196: Assignee: Apache Spark > Optimize the option and flatten in Word2Vec to reduce the max

[jira] [Comment Edited] (SPARK-13155) add runtime null check when convert catalyst array to external array

2016-02-04 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133324#comment-15133324 ] Huaxin Gao edited comment on SPARK-13155 at 2/4/16 11:43 PM: - @Wenchen Fan

[jira] [Created] (SPARK-13206) Subquery Alias in Join in Hive Parser

2016-02-04 Thread Xiao Li (JIRA)
Xiao Li created SPARK-13206: --- Summary: Subquery Alias in Join in Hive Parser Key: SPARK-13206 URL: https://issues.apache.org/jira/browse/SPARK-13206 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-13206) Subquery Alias in Join in Hive Parser

2016-02-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133452#comment-15133452 ] Xiao Li commented on SPARK-13206: - [~viirya] This is the one I mentioned just now. Could you take a look

[jira] [Updated] (SPARK-13206) Subquery Alias in Hive Parser

2016-02-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13206: Summary: Subquery Alias in Hive Parser (was: Subquery Alias in Join in Hive Parser) > Subquery Alias in

[jira] [Commented] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133459#comment-15133459 ] Xiao Li commented on SPARK-12720: - Since Subquery Alias does not work well, my PR has to wait until

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133498#comment-15133498 ] Apache Spark commented on SPARK-13171: -- User 'jodersky' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13171: Assignee: (was: Apache Spark) > Update promise & future to Promise and Future as the

[jira] [Assigned] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13171: Assignee: Apache Spark > Update promise & future to Promise and Future as the old ones

[jira] [Assigned] (SPARK-13145) checkAnswer in SQL query suites should tolerate small float number error

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13145: Assignee: (was: Apache Spark) > checkAnswer in SQL query suites should tolerate small

[jira] [Commented] (SPARK-13145) checkAnswer in SQL query suites should tolerate small float number error

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133531#comment-15133531 ] Apache Spark commented on SPARK-13145: -- User 'bomeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13145) checkAnswer in SQL query suites should tolerate small float number error

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13145: Assignee: Apache Spark > checkAnswer in SQL query suites should tolerate small float

[jira] [Commented] (SPARK-13179) pyspark row name collision 'count'

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133538#comment-15133538 ] Apache Spark commented on SPARK-13179: -- User 'shoelsch' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13179) pyspark row name collision 'count'

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13179: Assignee: (was: Apache Spark) > pyspark row name collision 'count' >

[jira] [Created] (SPARK-13208) Replace Pair with tuples

2016-02-04 Thread Jakob Odersky (JIRA)
Jakob Odersky created SPARK-13208: - Summary: Replace Pair with tuples Key: SPARK-13208 URL: https://issues.apache.org/jira/browse/SPARK-13208 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-13208) Replace Pair with tuples

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133574#comment-15133574 ] Apache Spark commented on SPARK-13208: -- User 'jodersky' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13208) Replace Pair with tuples

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13208: Assignee: (was: Apache Spark) > Replace Pair with tuples > >

[jira] [Assigned] (SPARK-13208) Replace Pair with tuples

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13208: Assignee: Apache Spark > Replace Pair with tuples > > >

[jira] [Created] (SPARK-13190) Update pom.xml to reference Scala 2.11

2016-02-04 Thread Luciano Resende (JIRA)
Luciano Resende created SPARK-13190: --- Summary: Update pom.xml to reference Scala 2.11 Key: SPARK-13190 URL: https://issues.apache.org/jira/browse/SPARK-13190 Project: Spark Issue Type:

[jira] [Updated] (SPARK-13195) mapWithState fails in case of timeout set without state update

2016-02-04 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuval Itzchakov updated SPARK-13195: Description: Using the new spark mapWithState API, I've encountered a bug when setting a

[jira] [Updated] (SPARK-13195) mapWithState fails in case of timeout set without state update

2016-02-04 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuval Itzchakov updated SPARK-13195: Description: Using the new spark mapWithState API, I've encountered a bug when setting a

[jira] [Created] (SPARK-13192) Some memory leak problem?

2016-02-04 Thread uncleGen (JIRA)
uncleGen created SPARK-13192: Summary: Some memory leak problem? Key: SPARK-13192 URL: https://issues.apache.org/jira/browse/SPARK-13192 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-12430) Temporary folders do not get deleted after Task completes causing problems with disk space.

2016-02-04 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132341#comment-15132341 ] Iulian Dragos commented on SPARK-12430: --- There is a long discussion about this in the dynamic

[jira] [Created] (SPARK-13195) mapWithState fails in case of timeout set without state update

2016-02-04 Thread Yuval Itzchakov (JIRA)
Yuval Itzchakov created SPARK-13195: --- Summary: mapWithState fails in case of timeout set without state update Key: SPARK-13195 URL: https://issues.apache.org/jira/browse/SPARK-13195 Project: Spark

[jira] [Updated] (SPARK-13195) PairDStreamFunctions.mapWithState fails in case timeout is set without state update

2016-02-04 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuval Itzchakov updated SPARK-13195: Summary: PairDStreamFunctions.mapWithState fails in case timeout is set without state

[jira] [Updated] (SPARK-13195) mapWithState fails in case of timeout set without state update

2016-02-04 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuval Itzchakov updated SPARK-13195: Description: Using the new spark mapWithState API, I've encountered a bug when setting a

[jira] [Created] (SPARK-13191) Update LICENSE with Scala 2.11 dependencies

2016-02-04 Thread Luciano Resende (JIRA)
Luciano Resende created SPARK-13191: --- Summary: Update LICENSE with Scala 2.11 dependencies Key: SPARK-13191 URL: https://issues.apache.org/jira/browse/SPARK-13191 Project: Spark Issue

[jira] [Commented] (SPARK-13189) Cleanup build references to Scala 2.10

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132277#comment-15132277 ] Apache Spark commented on SPARK-13189: -- User 'lresende' has created a pull request for this issue:

[jira] [Commented] (SPARK-13190) Update pom.xml to reference Scala 2.11

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132334#comment-15132334 ] Apache Spark commented on SPARK-13190: -- User 'lresende' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13190) Update pom.xml to reference Scala 2.11

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13190: Assignee: (was: Apache Spark) > Update pom.xml to reference Scala 2.11 >

[jira] [Assigned] (SPARK-13190) Update pom.xml to reference Scala 2.11

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13190: Assignee: Apache Spark > Update pom.xml to reference Scala 2.11 >

[jira] [Created] (SPARK-13194) Update release audit tools to use Scala 2.11

2016-02-04 Thread Luciano Resende (JIRA)
Luciano Resende created SPARK-13194: --- Summary: Update release audit tools to use Scala 2.11 Key: SPARK-13194 URL: https://issues.apache.org/jira/browse/SPARK-13194 Project: Spark Issue

[jira] [Updated] (SPARK-13195) PairDStreamFunctions.mapWithState fails in case timeout is set without updating State[S]

2016-02-04 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuval Itzchakov updated SPARK-13195: Description: Using the new spark mapWithState API, I've encountered a bug when setting a

[jira] [Created] (SPARK-13196) Optimize the option and flatten in Word2Vec to reduce the max memory consumption

2016-02-04 Thread yuhao yang (JIRA)
yuhao yang created SPARK-13196: -- Summary: Optimize the option and flatten in Word2Vec to reduce the max memory consumption Key: SPARK-13196 URL: https://issues.apache.org/jira/browse/SPARK-13196

[jira] [Commented] (SPARK-13193) Update Docker tests to use Scala 2.11

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132339#comment-15132339 ] Apache Spark commented on SPARK-13193: -- User 'lresende' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13193) Update Docker tests to use Scala 2.11

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13193: Assignee: (was: Apache Spark) > Update Docker tests to use Scala 2.11 >

[jira] [Assigned] (SPARK-13193) Update Docker tests to use Scala 2.11

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13193: Assignee: Apache Spark > Update Docker tests to use Scala 2.11 >

[jira] [Updated] (SPARK-13195) PairDStreamFunctions.mapWithState fails in case timeout is set without updating State[S]

2016-02-04 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuval Itzchakov updated SPARK-13195: Description: Using the new spark mapWithState API, I've encountered a bug when setting a

[jira] [Assigned] (SPARK-13189) Cleanup build references to Scala 2.10

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13189: Assignee: Apache Spark > Cleanup build references to Scala 2.10 >

[jira] [Assigned] (SPARK-13189) Cleanup build references to Scala 2.10

2016-02-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13189: Assignee: (was: Apache Spark) > Cleanup build references to Scala 2.10 >

[jira] [Updated] (SPARK-13192) Some memory leak problem?

2016-02-04 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-13192: - Description: In my Spark Streaming job, the executor container was killed by Yarn. The nodemanager log

  1   2   >