[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-20 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964672#comment-14964672 ] Sun Rui commented on SPARK-10903: - I figured out the reason. You have to export the S3 methods in

[jira] [Commented] (SPARK-11200) NettyRpcEnv endless message "cannot send ${message} because RpcEnv is closed"

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964729#comment-14964729 ] Sean Owen commented on SPARK-11200: --- I have never seen this message or heard reports of it. It sounds

[jira] [Resolved] (SPARK-11110) Scala 2.11 build fails due to compiler errors

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-0. --- Resolution: Fixed Fix Version/s: 1.6.0 Should be resolved by

[jira] [Updated] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11199: --- Assignee: Felix Cheung > Improve R context management story and add getOrCreate >

[jira] [Updated] (SPARK-10876) display total application time in spark history UI

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10876: -- Priority: Minor (was: Major) > display total application time in spark history UI >

[jira] [Resolved] (SPARK-10876) display total application time in spark history UI

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10876. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9059

[jira] [Reopened] (SPARK-11166) HIVE ON SPARK : yarn-cluster mode , if memory is busy,have no enough resource. app wil failed

2015-10-20 Thread yindu_asan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yindu_asan reopened SPARK-11166: The cause of the isse is not only resource not enough 1. jar error . eg : if build spark , add

[jira] [Commented] (SPARK-11205) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964821#comment-14964821 ] Jeff Zhang commented on SPARK-11205: Will create PR soon. > Delegate to scala DataFrame API rather

[jira] [Created] (SPARK-11206) Support SQL UI on the history server

2015-10-20 Thread Carson Wang (JIRA)
Carson Wang created SPARK-11206: --- Summary: Support SQL UI on the history server Key: SPARK-11206 URL: https://issues.apache.org/jira/browse/SPARK-11206 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-11205) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11205: Assignee: (was: Apache Spark) > Delegate to scala DataFrame API rather than print in

[jira] [Commented] (SPARK-11205) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964840#comment-14964840 ] Apache Spark commented on SPARK-11205: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Commented] (SPARK-11166) HIVE ON SPARK : yarn-cluster mode , if memory is busy,have no enough resource. app wil failed

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964850#comment-14964850 ] Sean Owen commented on SPARK-11166: --- [~673310629] please do not reopen issues if it has been closed by

[jira] [Updated] (SPARK-11179) Push filters through aggregate if filters are subset of 'group by' expressions

2015-10-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11179: Target Version/s: 1.6.0 > Push filters through aggregate if filters are subset of 'group by'

[jira] [Updated] (SPARK-10709) When loading a json dataset as a data frame, if the input path is wrong, the error message is very confusing

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10709: -- Target Version/s: (was: 1.6.0) Priority: Minor (was: Major) Issue Type:

[jira] [Created] (SPARK-11204) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-11204: -- Summary: Delegate to scala DataFrame API rather than print in python Key: SPARK-11204 URL: https://issues.apache.org/jira/browse/SPARK-11204 Project: Spark

[jira] [Updated] (SPARK-11205) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated SPARK-11205: --- Description: When I use DataFrame#explain(), I found the output is a little different from scala

[jira] [Created] (SPARK-11205) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Jeff Zhang (JIRA)
Jeff Zhang created SPARK-11205: -- Summary: Delegate to scala DataFrame API rather than print in python Key: SPARK-11205 URL: https://issues.apache.org/jira/browse/SPARK-11205 Project: Spark

[jira] [Assigned] (SPARK-11205) Delegate to scala DataFrame API rather than print in python

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11205: Assignee: Apache Spark > Delegate to scala DataFrame API rather than print in python >

[jira] [Commented] (SPARK-11206) Support SQL UI on the history server

2015-10-20 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964841#comment-14964841 ] Carson Wang commented on SPARK-11206: - I am working on this. I plan to add an onOtherEvent method to

[jira] [Closed] (SPARK-11166) HIVE ON SPARK : yarn-cluster mode , if memory is busy,have no enough resource. app wil failed

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-11166. - Resolution: Invalid > HIVE ON SPARK : yarn-cluster mode , if memory is busy,have no enough >

[jira] [Commented] (SPARK-2654) Leveled logging in PySpark

2015-10-20 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964586#comment-14964586 ] Jeff Zhang commented on SPARK-2654: --- [~davies] I think currently spark-core also don't have logging

[jira] [Created] (SPARK-11202) Unsupported dataType

2015-10-20 Thread whc (JIRA)
whc created SPARK-11202: --- Summary: Unsupported dataType Key: SPARK-11202 URL: https://issues.apache.org/jira/browse/SPARK-11202 Project: Spark Issue Type: Bug Affects Versions: 1.5.0

[jira] [Assigned] (SPARK-11201) StreamContext.getOrCreate is broken is yarn-client mode

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11201: Assignee: Apache Spark > StreamContext.getOrCreate is broken is yarn-client mode >

[jira] [Assigned] (SPARK-11201) StreamContext.getOrCreate is broken is yarn-client mode

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11201: Assignee: (was: Apache Spark) > StreamContext.getOrCreate is broken is yarn-client

[jira] [Updated] (SPARK-11202) Unsupported dataType

2015-10-20 Thread whc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] whc updated SPARK-11202: Component/s: SQL > Unsupported dataType > > > Key: SPARK-11202 >

[jira] [Commented] (SPARK-11201) StreamContext.getOrCreate is broken is yarn-client mode

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964608#comment-14964608 ] Apache Spark commented on SPARK-11201: -- User 'harishreedharan' has created a pull request for this

[jira] [Commented] (SPARK-11166) HIVE ON SPARK : yarn-cluster mode , if memory is busy,have no enough resource. app wil failed

2015-10-20 Thread yindu_asan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964857#comment-14964857 ] yindu_asan commented on SPARK-11166: thinks @Marcelo Vanzin @Sean Owen sorry!!! > HIVE ON SPARK

[jira] [Created] (SPARK-11207) Add test cases for normal LinearRegression solver as followup.

2015-10-20 Thread Kai Sasaki (JIRA)
Kai Sasaki created SPARK-11207: -- Summary: Add test cases for normal LinearRegression solver as followup. Key: SPARK-11207 URL: https://issues.apache.org/jira/browse/SPARK-11207 Project: Spark

[jira] [Commented] (SPARK-4368) Ceph integration?

2015-10-20 Thread Serge Smertin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964972#comment-14964972 ] Serge Smertin commented on SPARK-4368: -- if it's decided to be hosted outside of project - is there

[jira] [Assigned] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10971: Assignee: (was: Apache Spark) > sparkR: RRunner should allow setting path to Rscript

[jira] [Updated] (SPARK-11182) HDFS Delegation Token will be expired when calling "UserGroupInformation.getCurrentUser.addCredentials" in HA mode

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11182: --- Affects Version/s: 1.5.1 > HDFS Delegation Token will be expired when calling >

[jira] [Commented] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964983#comment-14964983 ] Apache Spark commented on SPARK-10971: -- User 'sun-rui' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10971: Assignee: Apache Spark > sparkR: RRunner should allow setting path to Rscript >

[jira] [Updated] (SPARK-11207) Add test cases for solver selection of LinearRegression as followup.

2015-10-20 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated SPARK-11207: --- Summary: Add test cases for solver selection of LinearRegression as followup. (was: Add test cases

[jira] [Created] (SPARK-11208) Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

2015-10-20 Thread Artem Aliev (JIRA)
Artem Aliev created SPARK-11208: --- Summary: Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config Key: SPARK-11208 URL: https://issues.apache.org/jira/browse/SPARK-11208 Project:

[jira] [Commented] (SPARK-11208) Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964974#comment-14964974 ] Apache Spark commented on SPARK-11208: -- User 'artem-aliev' has created a pull request for this

[jira] [Assigned] (SPARK-11208) Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11208: Assignee: (was: Apache Spark) > Filter out 'hive.metastore.rawstore.impl' from

[jira] [Created] (SPARK-11209) Add window functions into SparkR [step 1]

2015-10-20 Thread Sun Rui (JIRA)
Sun Rui created SPARK-11209: --- Summary: Add window functions into SparkR [step 1] Key: SPARK-11209 URL: https://issues.apache.org/jira/browse/SPARK-11209 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11209) Add window functions into SparkR [step 1]

2015-10-20 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964995#comment-14964995 ] Sun Rui commented on SPARK-11209: - I am working on this. > Add window functions into SparkR [step 1] >

[jira] [Commented] (SPARK-4368) Ceph integration?

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964994#comment-14964994 ] Steve Loughran commented on SPARK-4368: --- Looking at the ceph code, the strategy is 1. add the ceph

[jira] [Commented] (SPARK-10043) Add window functions into SparkR

2015-10-20 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964990#comment-14964990 ] Sun Rui commented on SPARK-10043: - Break this issue down to 2 sub-issues. > Add window functions into

[jira] [Commented] (SPARK-4368) Ceph integration?

2015-10-20 Thread Serge Smertin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964970#comment-14964970 ] Serge Smertin commented on SPARK-4368: -- here you go - Hadoop Filesystem implementation on top of

[jira] [Assigned] (SPARK-11208) Filter out 'hive.metastore.rawstore.impl' from executionHive temporary config

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11208: Assignee: Apache Spark > Filter out 'hive.metastore.rawstore.impl' from executionHive

[jira] [Created] (SPARK-11210) Add window functions into SparkR [step 2]

2015-10-20 Thread Sun Rui (JIRA)
Sun Rui created SPARK-11210: --- Summary: Add window functions into SparkR [step 2] Key: SPARK-11210 URL: https://issues.apache.org/jira/browse/SPARK-11210 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11206) Support SQL UI on the history server

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964997#comment-14964997 ] Steve Loughran commented on SPARK-11206: There's another strategy which would have to be taken up

[jira] [Commented] (SPARK-11207) Add test cases for solver selection of LinearRegression as followup.

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965117#comment-14965117 ] Apache Spark commented on SPARK-11207: -- User 'Lewuathe' has created a pull request for this issue:

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965118#comment-14965118 ] Jean-Baptiste Onofré commented on SPARK-11193: -- To thread safe, the HashMap

[jira] [Assigned] (SPARK-11207) Add test cases for solver selection of LinearRegression as followup.

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11207: Assignee: Apache Spark > Add test cases for solver selection of LinearRegression as

[jira] [Assigned] (SPARK-11207) Add test cases for solver selection of LinearRegression as followup.

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11207: Assignee: (was: Apache Spark) > Add test cases for solver selection of

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965125#comment-14965125 ] Jean-Baptiste Onofré commented on SPARK-11193: -- By the way, with spark 1.4.x, the

[jira] [Reopened] (SPARK-11173) Cannot save data via MSSQL JDBC

2015-10-20 Thread Gianluca Salvo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gianluca Salvo reopened SPARK-11173: Hi, I suppose I've found the problem: I downloaded the source code and followed the stack

[jira] [Created] (SPARK-11212) Make RDD's preferred locations support the executor location and fix ReceiverTracker for multiple executors in a host

2015-10-20 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-11212: Summary: Make RDD's preferred locations support the executor location and fix ReceiverTracker for multiple executors in a host Key: SPARK-11212 URL:

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965133#comment-14965133 ] Jean-Baptiste Onofré commented on SPARK-11193: -- I guess you used Scala 2.10.4 (default) for

[jira] [Comment Edited] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965133#comment-14965133 ] Jean-Baptiste Onofré edited comment on SPARK-11193 at 10/20/15 1:58 PM:

[jira] [Commented] (SPARK-11212) Make RDD's preferred locations support the executor location and fix ReceiverTracker for multiple executors in a host

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965136#comment-14965136 ] Apache Spark commented on SPARK-11212: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11212) Make RDD's preferred locations support the executor location and fix ReceiverTracker for multiple executors in a host

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11212: Assignee: (was: Apache Spark) > Make RDD's preferred locations support the executor

[jira] [Assigned] (SPARK-11212) Make RDD's preferred locations support the executor location and fix ReceiverTracker for multiple executors in a host

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11212: Assignee: Apache Spark > Make RDD's preferred locations support the executor location and

[jira] [Commented] (SPARK-11177) sc.wholeTextFiles throws ArrayIndexOutOfBoundsException when S3 file has zero bytes

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965007#comment-14965007 ] Steve Loughran commented on SPARK-11177: 0-byte files are a troublespot in object stores, as they

[jira] [Commented] (SPARK-11128) strange NPE when writing in non-existing S3 bucket

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965016#comment-14965016 ] Steve Loughran commented on SPARK-11128: Which Hadoop version is this? This JIRA could be moved

[jira] [Updated] (SPARK-11119) cleanup unsafe array and map

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9: -- Assignee: Wenchen Fan > cleanup unsafe array and map > > >

[jira] [Commented] (SPARK-11195) Exception thrown on executor throws ClassNotFound on driver

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965044#comment-14965044 ] Sean Owen commented on SPARK-11195: --- What is in your assembly jar? I assume you're sure it has

[jira] [Updated] (SPARK-11192) Spark sql seems to leak org.apache.spark.sql.execution.ui.SQLTaskMetrics objects over time

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11192: -- Target Version/s: (was: 1.5.1) [~blivingston] don't set Target; it can't target a released version

[jira] [Commented] (SPARK-11166) HIVE ON SPARK : yarn-cluster mode , if memory is busy,have no enough resource. app wil failed

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964998#comment-14964998 ] Steve Loughran commented on SPARK-11166: see also http://wiki.apache.org/hadoop/InvalidJiraIssues

[jira] [Updated] (SPARK-11195) Exception thrown on executor throws ClassNotFound on driver

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11195: -- Component/s: Spark Core > Exception thrown on executor throws ClassNotFound on driver >

[jira] [Commented] (SPARK-6527) sc.binaryFiles can not access files on s3

2015-10-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965006#comment-14965006 ] Steve Loughran commented on SPARK-6527: --- try using s3a instead of S3n (ideally, hadoop 2.7+); it may

[jira] [Updated] (SPARK-11190) SparkR support for cassandra collection types.

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11190: -- Target Version/s: (was: 1.5.1) Fix Version/s: (was: 1.5.2) Component/s: SparkR

[jira] [Commented] (SPARK-4368) Ceph integration?

2015-10-20 Thread Serge Smertin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965026#comment-14965026 ] Serge Smertin commented on SPARK-4368: -- Thank you Steve for bunch of details > Ceph integration? >

[jira] [Created] (SPARK-11211) Kafka - offsetOutOfRange forces to largest

2015-10-20 Thread Daniel Strassler (JIRA)
Daniel Strassler created SPARK-11211: Summary: Kafka - offsetOutOfRange forces to largest Key: SPARK-11211 URL: https://issues.apache.org/jira/browse/SPARK-11211 Project: Spark Issue

[jira] [Created] (SPARK-11203) UDF doesn't support charType column and lit function doesn't allow charType as argument

2015-10-20 Thread M Bharat lal (JIRA)
M Bharat lal created SPARK-11203: Summary: UDF doesn't support charType column and lit function doesn't allow charType as argument Key: SPARK-11203 URL: https://issues.apache.org/jira/browse/SPARK-11203

[jira] [Resolved] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8332. -- Resolution: Not A Problem > NoSuchMethodError: >

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-10-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964762#comment-14964762 ] Sean Owen commented on SPARK-11016: --- Yeah that's a good start. It's bridging Kryo with the particular

[jira] [Updated] (SPARK-11192) When graphite metric sink is enabled, spark sql leaks org.apache.spark.sql.execution.ui.SQLTaskMetrics objects over time

2015-10-20 Thread Blake Livingston (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Livingston updated SPARK-11192: - Description: Noticed that slowly, over the course of a day or two, heap memory usage on

[jira] [Updated] (SPARK-11192) Spark sql seems to leak org.apache.spark.sql.execution.ui.SQLTaskMetrics objects over time

2015-10-20 Thread Blake Livingston (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Livingston updated SPARK-11192: - Summary: Spark sql seems to leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964642#comment-14964642 ] Felix Cheung commented on SPARK-10903: -- Working from SparkR package version in this commit:

[jira] [Closed] (SPARK-8071) Run PySpark dataframe.rollup/cube test failed

2015-10-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-8071. - Resolution: Won't Fix > Run PySpark dataframe.rollup/cube test failed >

[jira] [Commented] (SPARK-9299) percentile and percentile_approx aggregate functions

2015-10-20 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964656#comment-14964656 ] Jeff Zhang commented on SPARK-9299: --- Link with SPARK-6761 as they can share the same algorithm >

[jira] [Closed] (SPARK-10045) Add support for DataFrameStatFunctions in SparkR

2015-10-20 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-10045. --- Resolution: Fixed all sub-tasks are finished. > Add support for DataFrameStatFunctions in SparkR >

[jira] [Created] (SPARK-11213) Documentation for remote spark Submit for R Scripts from 1.5 on CDH 5.4

2015-10-20 Thread Ankit (JIRA)
Ankit created SPARK-11213: - Summary: Documentation for remote spark Submit for R Scripts from 1.5 on CDH 5.4 Key: SPARK-11213 URL: https://issues.apache.org/jira/browse/SPARK-11213 Project: Spark

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965210#comment-14965210 ] Jean-Baptiste Onofré commented on SPARK-11193: -- It looks to work fine to me: {code}

[jira] [Resolved] (SPARK-10447) Upgrade pyspark to use py4j 0.9

2015-10-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10447. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8615

[jira] [Updated] (SPARK-10447) Upgrade pyspark to use py4j 0.9

2015-10-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10447: --- Assignee: holdenk > Upgrade pyspark to use py4j 0.9 > --- > >

[jira] [Assigned] (SPARK-11214) Join with Unicode-String results wrong empty

2015-10-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-11214: -- Assignee: Josh Rosen > Join with Unicode-String results wrong empty >

[jira] [Commented] (SPARK-11201) StreamContext.getOrCreate is broken is yarn-client mode

2015-10-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965529#comment-14965529 ] Marcelo Vanzin commented on SPARK-11201: As I mentioned in the PR, I believe the fix for

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965561#comment-14965561 ] Jean-Baptiste Onofré commented on SPARK-11193: -- I only have one warning during compilation

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965567#comment-14965567 ] Jean-Baptiste Onofré commented on SPARK-11193: -- No warning with Java7 neither. > Spark 1.5+

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread Phil Kallos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965599#comment-14965599 ] Phil Kallos commented on SPARK-11193: - Have you tried the last combination? scala 2.10.4 + java 7?

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965584#comment-14965584 ] Jean-Baptiste Onofré commented on SPARK-11193: -- OK, I have the same warning as you have if I

[jira] [Resolved] (SPARK-11105) Disitribute the log4j.properties files from the client to the executors

2015-10-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11105. Resolution: Fixed Assignee: Srinivasa Reddy Vundela Fix Version/s: 1.6.0 >

[jira] [Comment Edited] (SPARK-10309) Some tasks failed with Unable to acquire memory

2015-10-20 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965510#comment-14965510 ] Jerry Lam edited comment on SPARK-10309 at 10/20/15 6:25 PM: - Same issue, I

[jira] [Assigned] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11199: Assignee: Felix Cheung (was: Apache Spark) > Improve R context management story and add

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965605#comment-14965605 ] Jean-Baptiste Onofré commented on SPARK-11193: -- Can you provide: - Java version and provider

[jira] [Commented] (SPARK-10646) Bivariate Statistics: Pearson's Chi-Squared goodness of fit test

2015-10-20 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965298#comment-14965298 ] Jihong MA commented on SPARK-10646: --- [~mengxr] to add chi-squared test support through UDAF framework,

[jira] [Commented] (SPARK-11128) strange NPE when writing in non-existing S3 bucket

2015-10-20 Thread mathieu despriee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965351#comment-14965351 ] mathieu despriee commented on SPARK-11128: -- hadoop 2.4 just tested right now with hadoop 2.6

[jira] [Comment Edited] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954447#comment-14954447 ] Reynold Xin edited comment on SPARK-10953 at 10/20/15 7:13 PM: --- [~mengxr] I

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread Phil Kallos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965612#comment-14965612 ] Phil Kallos commented on SPARK-11193: - {noformat} $ java -version java version "1.7.0_75" Java(TM) SE

[jira] [Updated] (SPARK-3842) Remove the hacks for Python callback server in py4j

2015-10-20 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3842: -- Assignee: holdenk (was: Davies Liu) > Remove the hacks for Python callback server in py4j >

[jira] [Assigned] (SPARK-3842) Remove the hacks for Python callback server in py4j

2015-10-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3842: --- Assignee: Davies Liu (was: Apache Spark) > Remove the hacks for Python callback server in

[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer

2015-10-20 Thread Jonathan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965484#comment-14965484 ] Jonathan Kelly commented on SPARK-8332: --- Oh, you might be right. I was thinking that I had tried

[jira] [Commented] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

2015-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965530#comment-14965530 ] Jean-Baptiste Onofré commented on SPARK-11193: -- The warning is interesting, let me check if

[jira] [Commented] (SPARK-11201) StreamContext.getOrCreate is broken is yarn-client mode

2015-10-20 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965533#comment-14965533 ] Hari Shreedharan commented on SPARK-11201: -- Yep, closed the PR. So I will close this jira as a

  1   2   3   >