[jira] [Commented] (SPARK-4397) Reorganize 'implicit's to improve the API convenience

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211989#comment-14211989 ] Apache Spark commented on SPARK-4397: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-11-13 Thread Daniel Erenrich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211986#comment-14211986 ] Daniel Erenrich commented on SPARK-3080: Yes I understand. I am definitely doing i

[jira] [Created] (SPARK-4397) Reorganize 'implicit's to improve the API convenience

2014-11-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4397: --- Summary: Reorganize 'implicit's to improve the API convenience Key: SPARK-4397 URL: https://issues.apache.org/jira/browse/SPARK-4397 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1977) mutable.BitSet in ALS not serializable with KryoSerializer

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211915#comment-14211915 ] Apache Spark commented on SPARK-1977: - User 'nevillelyh' has created a pull request fo

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211898#comment-14211898 ] Xiangrui Meng commented on SPARK-3080: -- I see. If the procedure of sample negatives i

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

2014-11-13 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211896#comment-14211896 ] Nishkam Ravi commented on SPARK-2398: - [~srowen] yes, this has been resolved by modify

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-11-13 Thread Daniel Erenrich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211881#comment-14211881 ] Daniel Erenrich commented on SPARK-3080: Yes I am using random numbers. The random

[jira] [Commented] (SPARK-4348) pyspark.mllib.random conflicts with random module

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211768#comment-14211768 ] Xiangrui Meng commented on SPARK-4348: -- Note that after this fix, it is very likely t

[jira] [Commented] (SPARK-4396) Support lookup by index in Rating

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211754#comment-14211754 ] Apache Spark commented on SPARK-4396: - User 'mengxr' has created a pull request for th

[jira] [Created] (SPARK-4396) Support lookup by index in Rating

2014-11-13 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-4396: Summary: Support lookup by index in Rating Key: SPARK-4396 URL: https://issues.apache.org/jira/browse/SPARK-4396 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-3080) ArrayIndexOutOfBoundsException in ALS for Large datasets

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211702#comment-14211702 ] Xiangrui Meng commented on SPARK-3080: -- [~ilganeli] and [~derenrich], one more questi

[jira] [Created] (SPARK-4395) Running a Spark SQL SELECT command from PySpark causes a hang for ~ 1 hour

2014-11-13 Thread Sameer Farooqui (JIRA)
Sameer Farooqui created SPARK-4395: -- Summary: Running a Spark SQL SELECT command from PySpark causes a hang for ~ 1 hour Key: SPARK-4395 URL: https://issues.apache.org/jira/browse/SPARK-4395 Project:

[jira] [Commented] (SPARK-4394) Allow datasources to support IN and sizeInBytes

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211640#comment-14211640 ] Apache Spark commented on SPARK-4394: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

2014-11-13 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211624#comment-14211624 ] Josh Rosen commented on SPARK-3633: --- I've opened SPARK-4393 to address the memory leak i

[jira] [Commented] (SPARK-4393) Memory leak in connection manager timeout thread

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211620#comment-14211620 ] Apache Spark commented on SPARK-4393: - User 'JoshRosen' has created a pull request for

[jira] [Created] (SPARK-4394) Allow datasources to support IN and sizeInBytes

2014-11-13 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-4394: --- Summary: Allow datasources to support IN and sizeInBytes Key: SPARK-4394 URL: https://issues.apache.org/jira/browse/SPARK-4394 Project: Spark Issue Typ

[jira] [Updated] (SPARK-4393) Memory leak in connection manager timeout thread

2014-11-13 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4393: -- Description: This JIRA tracks a fix for a memory leak in ConnectionManager's TimerTasks, originally repo

[jira] [Created] (SPARK-4393) Memory leak in connection manager timeout thread

2014-11-13 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4393: - Summary: Memory leak in connection manager timeout thread Key: SPARK-4393 URL: https://issues.apache.org/jira/browse/SPARK-4393 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-4392) Event proration based on event timestamps

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211567#comment-14211567 ] Apache Spark commented on SPARK-4392: - User 'bijaybisht' has created a pull request fo

[jira] [Created] (SPARK-4392) Event proration based on event timestamps

2014-11-13 Thread Bijay Singh Bisht (JIRA)
Bijay Singh Bisht created SPARK-4392: Summary: Event proration based on event timestamps Key: SPARK-4392 URL: https://issues.apache.org/jira/browse/SPARK-4392 Project: Spark Issue Type: N

[jira] [Resolved] (SPARK-2816) Type-safe SQL queries

2014-11-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2816. - Resolution: Won't Fix This is going to happen in a library instead: https://github.com/ma

[jira] [Updated] (SPARK-4385) DataSource DDL Parser can't handle table names with '_'

2014-11-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4385: Assignee: Cheng Lian > DataSource DDL Parser can't handle table names with '_' > ---

[jira] [Commented] (SPARK-4391) Parquet Filter pushdown flag should be set with SQLConf

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211532#comment-14211532 ] Apache Spark commented on SPARK-4391: - User 'marmbrus' has created a pull request for

[jira] [Updated] (SPARK-3633) Fetches failure observed after SPARK-2711

2014-11-13 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3633: -- Priority: Blocker (was: Critical) This seems like a very serious issue, so I've upgraded it to a 1.2 bl

[jira] [Resolved] (SPARK-4355) OnlineSummarizer doesn't merge mean correctly

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4355. -- Resolution: Fixed Fix Version/s: (was: 1.2.0) 1.1.2 Issue resolved

[jira] [Reopened] (SPARK-4355) OnlineSummarizer doesn't merge mean correctly

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-4355: -- branch-1.0 pending > OnlineSummarizer doesn't merge mean correctly > --

[jira] [Updated] (SPARK-4355) OnlineSummarizer doesn't merge mean correctly

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4355: - Target Version/s: 1.2.0, 1.0.3, 1.1.2 (was: 1.2.0, 1.0.3) > OnlineSummarizer doesn't merge mean c

[jira] [Updated] (SPARK-4390) Bad casts to decimal throw instead of returning null

2014-11-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4390: Component/s: SQL > Bad casts to decimal throw instead of returning null > --

[jira] [Created] (SPARK-4391) Parquet Filter pushdown flag should be set with SQLConf

2014-11-13 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-4391: --- Summary: Parquet Filter pushdown flag should be set with SQLConf Key: SPARK-4391 URL: https://issues.apache.org/jira/browse/SPARK-4391 Project: Spark I

[jira] [Commented] (SPARK-4390) Bad casts to decimal throw instead of returning null

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211442#comment-14211442 ] Apache Spark commented on SPARK-4390: - User 'marmbrus' has created a pull request for

[jira] [Created] (SPARK-4390) Bad casts to decimal throw instead of returning null

2014-11-13 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-4390: --- Summary: Bad casts to decimal throw instead of returning null Key: SPARK-4390 URL: https://issues.apache.org/jira/browse/SPARK-4390 Project: Spark Issu

[jira] [Closed] (SPARK-4310) "Submitted" column in Stage page doesn't sort by time

2014-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-4310. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Shixiong Zhu Target Version/s:

[jira] [Updated] (SPARK-4379) RDD.checkpoint throws a general Exception (should be SparkException)

2014-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4379: - Summary: RDD.checkpoint throws a general Exception (should be SparkException) (was: RDD.checkpoint throws

[jira] [Updated] (SPARK-2996) Standalone and Yarn have different settings for adding the user classpath first

2014-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2996: - Affects Version/s: 1.0.0 > Standalone and Yarn have different settings for adding the user classpath > fi

[jira] [Updated] (SPARK-2996) Standalone and Yarn have different settings for adding the user classpath first

2014-11-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2996: - Target Version/s: 1.3.0 > Standalone and Yarn have different settings for adding the user classpath > fir

[jira] [Updated] (SPARK-4372) Make LR and SVM's default parameters consistent in Scala and Python

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4372: - Assignee: Xiangrui Meng > Make LR and SVM's default parameters consistent in Scala and Python > -

[jira] [Resolved] (SPARK-4372) Make LR and SVM's default parameters consistent in Scala and Python

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4372. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3232 [https://githu

[jira] [Updated] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT

2014-11-13 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4389: -- Description: We should set {{akka.remote.netty.tcp.bind-hostname="0.0.0.0"}} in our Akka configuration

[jira] [Updated] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT

2014-11-13 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4389: -- Priority: Minor (was: Major) > Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be locat

[jira] [Created] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT

2014-11-13 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4389: - Summary: Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT Key: SPARK-4389 URL: https://issues.apache.org/jira/browse/SPARK-4389 Project

[jira] [Resolved] (SPARK-4326) unidoc is broken on master

2014-11-13 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-4326. --- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Xiangrui Meng > unidoc is bro

[jira] [Commented] (SPARK-4387) Refactoring python profiling code to make it extensible

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211301#comment-14211301 ] Apache Spark commented on SPARK-4387: - User 'udnay' has created a pull request for thi

[jira] [Created] (SPARK-4388) Add tooltips to explain maxMemory / usedMemory columns in executor UI

2014-11-13 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4388: - Summary: Add tooltips to explain maxMemory / usedMemory columns in executor UI Key: SPARK-4388 URL: https://issues.apache.org/jira/browse/SPARK-4388 Project: Spark

[jira] [Created] (SPARK-4387) Refactoring python profiling code to make it extensible

2014-11-13 Thread Yandu Oppacher (JIRA)
Yandu Oppacher created SPARK-4387: - Summary: Refactoring python profiling code to make it extensible Key: SPARK-4387 URL: https://issues.apache.org/jira/browse/SPARK-4387 Project: Spark Issue

[jira] [Commented] (SPARK-874) Have a --wait flag in ./sbin/stop-all.sh that polls until Worker's are finished

2014-11-13 Thread Hari Dattada (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211287#comment-14211287 ] Hari Dattada commented on SPARK-874: I am not sure if this is still being worked on. If

[jira] [Commented] (SPARK-4386) Parquet file write performance improvement

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211271#comment-14211271 ] Apache Spark commented on SPARK-4386: - User 'jimfcarroll' has created a pull request f

[jira] [Updated] (SPARK-4326) unidoc is broken on master

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-4326: - Target Version/s: 1.2.0 (was: 1.3.0) > unidoc is broken on master > -- >

[jira] [Created] (SPARK-4386) Parquet file write performance improvement

2014-11-13 Thread Jim Carroll (JIRA)
Jim Carroll created SPARK-4386: -- Summary: Parquet file write performance improvement Key: SPARK-4386 URL: https://issues.apache.org/jira/browse/SPARK-4386 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4326) unidoc is broken on master

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211241#comment-14211241 ] Xiangrui Meng commented on SPARK-4326: -- Sorry, I pointed to the wrong commit ... I ju

[jira] [Created] (SPARK-4385) DataSource DDL Parser can't handle table names with '_'

2014-11-13 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-4385: --- Summary: DataSource DDL Parser can't handle table names with '_' Key: SPARK-4385 URL: https://issues.apache.org/jira/browse/SPARK-4385 Project: Spark I

[jira] [Commented] (SPARK-4326) unidoc is broken on master

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211237#comment-14211237 ] Apache Spark commented on SPARK-4326: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-4384) Too many open files during sort in pyspark

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211229#comment-14211229 ] Apache Spark commented on SPARK-4384: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-4384) Too many open files during sort in pyspark

2014-11-13 Thread Davies Liu (JIRA)
Davies Liu created SPARK-4384: - Summary: Too many open files during sort in pyspark Key: SPARK-4384 URL: https://issues.apache.org/jira/browse/SPARK-4384 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-4378) Make ALS more Java-friendly

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4378. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3240 [https://githu

[jira] [Created] (SPARK-4383) Delay scheduling doesn't work right when jobs have tasks with different locality levels

2014-11-13 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-4383: - Summary: Delay scheduling doesn't work right when jobs have tasks with different locality levels Key: SPARK-4383 URL: https://issues.apache.org/jira/browse/SPARK-4383

[jira] [Commented] (SPARK-4355) OnlineSummarizer doesn't merge mean correctly

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210169#comment-14210169 ] Apache Spark commented on SPARK-4355: - User 'mengxr' has created a pull request for th

[jira] [Resolved] (SPARK-4348) pyspark.mllib.random conflicts with random module

2014-11-13 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4348. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 3216 [https://githu

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210114#comment-14210114 ] Apache Spark commented on SPARK-4226: - User 'ravipesala' has created a pull request fo

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

2014-11-13 Thread Stephen Haberman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210100#comment-14210100 ] Stephen Haberman commented on SPARK-3633: - FWIW we've seen this issue several time

[jira] [Commented] (SPARK-4322) Struct fields can't be used as sub-expression of grouping fields

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210084#comment-14210084 ] Apache Spark commented on SPARK-4322: - User 'liancheng' has created a pull request for

[jira] [Comment Edited] (SPARK-2468) Netty-based block server / client module

2014-11-13 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210074#comment-14210074 ] Aaron Davidson edited comment on SPARK-2468 at 11/13/14 5:48 PM: ---

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-13 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210074#comment-14210074 ] Aaron Davidson commented on SPARK-2468: --- Here is my spark configuration (note 32 cor

[jira] [Updated] (SPARK-4322) Struct fields can't be used as sub-expression of grouping fields

2014-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4322: -- Description: Some examples: {code} sqlContext.jsonRDD(sc.parallelize("""{"a": {"b": [{"c": 1}]}}""" ::

[jira] [Updated] (SPARK-4322) Struct fields can't be used as sub-expression of grouping fields

2014-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4322: -- Summary: Struct fields can't be used as sub-expression of grouping fields (was: Analysis incorrectly re

[jira] [Commented] (SPARK-3066) Support recommendAll in matrix factorization model

2014-11-13 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209936#comment-14209936 ] Debasish Das commented on SPARK-3066: - On our internal datasets, flatMap is slow...I a

[jira] [Commented] (SPARK-4233) Simplify the Aggregation Function implementation

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209832#comment-14209832 ] Apache Spark commented on SPARK-4233: - User 'chenghao-intel' has created a pull reques

[jira] [Commented] (SPARK-4382) Add locations parameter to Twitter Stream

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209830#comment-14209830 ] Apache Spark commented on SPARK-4382: - User 'viirya' has created a pull request for th

[jira] [Created] (SPARK-4382) Add locations parameter to Twitter Stream

2014-11-13 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-4382: -- Summary: Add locations parameter to Twitter Stream Key: SPARK-4382 URL: https://issues.apache.org/jira/browse/SPARK-4382 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-2687) after receving allocated containers,amClient should remove ContainerRequest.

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209821#comment-14209821 ] Apache Spark commented on SPARK-2687: - User 'lianhuiwang' has created a pull request f

[jira] [Issue Comment Deleted] (SPARK-4371) Spark crashes with JBoss Logging 3.6.1

2014-11-13 Thread Florent Pellerin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florent Pellerin updated SPARK-4371: Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Spark

[jira] [Commented] (SPARK-4381) User should get warned when set spark.master with local in Spark Streaming

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209743#comment-14209743 ] Apache Spark commented on SPARK-4381: - User 'jerryshao' has created a pull request for

[jira] [Created] (SPARK-4381) User should get warned when set spark.master with local in Spark Streaming

2014-11-13 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-4381: -- Summary: User should get warned when set spark.master with local in Spark Streaming Key: SPARK-4381 URL: https://issues.apache.org/jira/browse/SPARK-4381 Project: Spark

[jira] [Commented] (SPARK-4380) Executor full of log "spilling in-memory map of 0 MB to disk"

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209699#comment-14209699 ] Apache Spark commented on SPARK-4380: - User 'shenh062326' has created a pull request f

[jira] [Commented] (SPARK-4380) Executor full of log "spilling in-memory map of 0 MB to disk"

2014-11-13 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209680#comment-14209680 ] Hong Shen commented on SPARK-4380: -- The reason is that it's MB but not B, when I change t

[jira] [Updated] (SPARK-4380) Executor full of log "spilling in-memory map of 0 MB to disk"

2014-11-13 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-4380: - Description: When I set spark.shuffle.manager = sort, Executor full of log, It confuse me a lot: 14/11/13

[jira] [Created] (SPARK-4380) Executor full of log "spilling in-memory map of 0 MB to disk"

2014-11-13 Thread Hong Shen (JIRA)
Hong Shen created SPARK-4380: Summary: Executor full of log "spilling in-memory map of 0 MB to disk" Key: SPARK-4380 URL: https://issues.apache.org/jira/browse/SPARK-4380 Project: Spark Issue Ty

[jira] [Commented] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2014-11-13 Thread SUMANTH B B N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209622#comment-14209622 ] SUMANTH B B N commented on SPARK-3717: -- i tried to implement the method rowToColumnSt

[jira] [Commented] (SPARK-4285) Transpose RDD[Vector] to column store for ML

2014-11-13 Thread SUMANTH B B N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209605#comment-14209605 ] SUMANTH B B N commented on SPARK-4285: -- [~josephkb] i have tried to implement this me

[jira] [Commented] (SPARK-4371) Spark crashes with JBoss Logging 3.6.1

2014-11-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209588#comment-14209588 ] Sean Owen commented on SPARK-4371: -- Aha, so it's a web app containing Spark. Although it'

[jira] [Commented] (SPARK-4371) Spark crashes with JBoss Logging 3.6.1

2014-11-13 Thread Florent Pellerin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209572#comment-14209572 ] Florent Pellerin commented on SPARK-4371: - I build a Spring Webapp, in which Spark

[jira] [Commented] (SPARK-4371) Spark crashes with JBoss Logging 3.6.1

2014-11-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209559#comment-14209559 ] Sean Owen commented on SPARK-4371: -- But you're bundling this in your Spark app right? you

[jira] [Commented] (SPARK-4379) RDD.checkpoint throws an Exception

2014-11-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209504#comment-14209504 ] Apache Spark commented on SPARK-4379: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-4379) RDD.checkpoint throws an Exception

2014-11-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4379: --- Summary: RDD.checkpoint throws an Exception Key: SPARK-4379 URL: https://issues.apache.org/jira/browse/SPARK-4379 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-13 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209489#comment-14209489 ] zzc commented on SPARK-2468: Aaron Davidson, I find that #3155 has merged into master, but sp

[jira] [Commented] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

2014-11-13 Thread Bertrand Bossy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209486#comment-14209486 ] Bertrand Bossy commented on SPARK-3039: --- Needs more fixes, since although I got the

[jira] [Commented] (SPARK-4359) Empty classifier in "avro-mapred" is misinterpreted in SBT

2014-11-13 Thread Bertrand Bossy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209480#comment-14209480 ] Bertrand Bossy commented on SPARK-4359: --- Is it possible that sbt doesn't recognize "

[jira] [Commented] (SPARK-2468) Netty-based block server / client module

2014-11-13 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209477#comment-14209477 ] zzc commented on SPARK-2468: Aaron Davidson, thank you for your reply. I will try it again. C