[jira] [Updated] (SPARK-15630) 2.0 python coverage ml root module

2016-07-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15630: -- Target Version/s: 2.0.0 Priority: Blocker (was: Major) > 2.0 python coverage ml root modul

[jira] [Commented] (SPARK-16465) Add nonnegative flag to mllib ALS

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369545#comment-15369545 ] Sean Owen commented on SPARK-16465: --- What are you referring to -- there has been a setN

[jira] [Updated] (SPARK-16470) ml.regression.LinearRegression training data do not check whether the result actually reach convergence

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16470: -- Affects Version/s: (was: 2.0.1) (was: 2.1.0) 2.0.0

[jira] [Resolved] (SPARK-15937) Spark declares a succeeding job to be failed in yarn-cluster mode if the job takes very small time (~ < 10 seconds) to finish

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15937. --- Resolution: Not A Problem Per JIRA discussion > Spark declares a succeeding job to be failed in yarn

[jira] [Resolved] (SPARK-16361) It takes a long time for gc when building cube with many fields

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16361. --- Resolution: Not A Problem > It takes a long time for gc when building cube with many fields > --

[jira] [Updated] (SPARK-16469) Long running Driver task while multiplying big matrices

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16469: -- Fix Version/s: (was: 2.0.0) > Long running Driver task while multiplying big matrices > ---

[jira] [Commented] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369686#comment-15369686 ] Sean Owen commented on SPARK-16474: --- I am not sure that is expected to work. You have d

[jira] [Commented] (SPARK-16478) strongly connected components doesn't cache returned RDD

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370409#comment-15370409 ] Sean Owen commented on SPARK-16478: --- You can cache the results if you want it cached. T

[jira] [Commented] (SPARK-16465) Add nonnegative flag to mllib ALS

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370423#comment-15370423 ] Sean Owen commented on SPARK-16465: --- Is this Python? You can set these values if you d

[jira] [Commented] (SPARK-14813) ML 2.0 QA: API: Python API coverage

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370577#comment-15370577 ] Sean Owen commented on SPARK-14813: --- OK, pedantically, I just want to make sure that: t

[jira] [Commented] (SPARK-16480) Streaming checkpointing does not work well with SIGTERM

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370602#comment-15370602 ] Sean Owen commented on SPARK-16480: --- [~skonto] can you edit this down a bit / format it

[jira] [Updated] (SPARK-13569) Kafka DStreams from wildcard topic filters

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13569: -- Assignee: Cody Koeninger > Kafka DStreams from wildcard topic filters > ---

[jira] [Updated] (SPARK-16479) Add Asynchronous Actions Example

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16479: -- Target Version/s: (was: 1.6.2) Labels: (was: beginner newbie) Fix Version/s:

[jira] [Commented] (SPARK-16480) Streaming checkpointing does not work well with SIGTERM

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370784#comment-15370784 ] Sean Owen commented on SPARK-16480: --- Hm, if it happens with SIGINT then SIGTERM isn't t

[jira] [Commented] (SPARK-16480) Streaming checkpointing does not work well with SIGTERM

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370989#comment-15370989 ] Sean Owen commented on SPARK-16480: --- I've caught SIGINT in a program with sun.misc.Sign

[jira] [Updated] (SPARK-16385) NoSuchMethodException thrown by Utils.waitForProcess

2016-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16385: -- Fix Version/s: 1.6.3 > NoSuchMethodException thrown by Utils.waitForProcess > -

[jira] [Commented] (SPARK-16480) Streaming checkpointing does not work well with SIGTERM

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372526#comment-15372526 ] Sean Owen commented on SPARK-16480: --- Huh, good point, I actually didn't realize Spark a

[jira] [Commented] (SPARK-16494) Upgrade breeze to 0.12

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372543#comment-15372543 ] Sean Owen commented on SPARK-16494: --- Seems OK for 2.1 but the question is always whethe

[jira] [Commented] (SPARK-16495) Add ADMM optimizer in mllib package

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372746#comment-15372746 ] Sean Owen commented on SPARK-16495: --- Have a look at https://cwiki.apache.org/confluenc

[jira] [Updated] (SPARK-16395) Fail if too many CheckpointWriteHandlers are queued up in the fixed thread pool

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16395: -- Summary: Fail if too many CheckpointWriteHandlers are queued up in the fixed thread pool (was: Show wa

[jira] [Resolved] (SPARK-16470) ml.regression.LinearRegression training data do not check whether the result actually reach convergence

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16470. --- Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 14122 [https://github.co

[jira] [Updated] (SPARK-16470) ml.regression.LinearRegression training data do not check whether the result actually reach convergence

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16470: -- Assignee: Weichen Xu Priority: Trivial (was: Minor) > ml.regression.LinearRegression training data

[jira] [Resolved] (SPARK-16449) unionAll raises "Task not serializable"

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16449. --- Resolution: Duplicate > unionAll raises "Task not serializable" > ---

[jira] [Commented] (SPARK-16500) Add LBFG training not convergence warning for all ML algorithm

2016-07-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373051#comment-15373051 ] Sean Owen commented on SPARK-16500: --- [~WeichenXu123] this should have been covered in S

[jira] [Commented] (SPARK-16521) Add support of parameterized configuration for SparkConf

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374606#comment-15374606 ] Sean Owen commented on SPARK-16521: --- This sounds almost exactly like https://issues.ap

[jira] [Commented] (SPARK-16513) Spark executor deadlocks itself in memory management

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374613#comment-15374613 ] Sean Owen commented on SPARK-16513: --- Can you try vs a later version, and, why do you th

[jira] [Resolved] (SPARK-16375) [Spark web UI]:The wrong value(numCompletedTasks) has been assigned to the variable numSkippedTasks

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16375. --- Resolution: Fixed Fix Version/s: 2.0.1 1.6.3 Issue resolved by pull request

[jira] [Updated] (SPARK-16375) [Spark web UI]:The wrong value(numCompletedTasks) has been assigned to the variable numSkippedTasks

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16375: -- Assignee: Alex Bozarth > [Spark web UI]:The wrong value(numCompletedTasks) has been assigned to the >

[jira] [Resolved] (SPARK-16439) Incorrect information in SQL Query details

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16439. --- Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 14142 [https://github.co

[jira] [Updated] (SPARK-16439) Incorrect information in SQL Query details

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16439: -- Assignee: Maciej Bryński Priority: Minor (was: Major) > Incorrect information in SQL Query details

[jira] [Resolved] (SPARK-16527) Scala Error:Type Arguments

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16527. --- Resolution: Invalid Fix Version/s: (was: 1.6.1) Target Version/s: (was: 1.6.1)

[jira] [Updated] (SPARK-16438) Add Asynchronous Actions documentation

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16438: -- Shepherd: (was: sandeep purohit) Assignee: sandeep purohit Priority: Trivial (

[jira] [Resolved] (SPARK-16440) Undeleted broadcast variables in Word2Vec causing OoM for long runs

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16440. --- Resolution: Fixed Fix Version/s: 2.0.1 1.6.3 Issue resolved by pull request

[jira] [Updated] (SPARK-16440) Undeleted broadcast variables in Word2Vec causing OoM for long runs

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16440: -- Assignee: Sean Owen > Undeleted broadcast variables in Word2Vec causing OoM for long runs > --

[jira] [Commented] (SPARK-16528) HiveClientImpl throws NPE when reading database from a custom metastore

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374961#comment-15374961 ] Sean Owen commented on SPARK-16528: --- If it should just be passes through as null in thi

[jira] [Resolved] (SPARK-16469) Long running Driver task while multiplying big matrices

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16469. --- Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 14068 [https://github.co

[jira] [Updated] (SPARK-16469) Long running Driver task while multiplying big matrices

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16469: -- Assignee: Ohad Raviv > Long running Driver task while multiplying big matrices > --

[jira] [Updated] (SPARK-16325) reduceByKey requires an implicit ordering which it never uses

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16325: -- Description: assume there is a case class as follows: {code} case class UnorderedPair[A](left: A, righ

[jira] [Commented] (SPARK-16325) reduceByKey requires an implicit ordering which it never uses

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375290#comment-15375290 ] Sean Owen commented on SPARK-16325: --- I can't reproduce this on master. What error do yo

[jira] [Resolved] (SPARK-14258) change scope of some functions in KafkaCluster

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14258. --- Resolution: Won't Fix > change scope of some functions in KafkaCluster >

[jira] [Resolved] (SPARK-16513) Spark executor deadlocks itself in memory management

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16513. --- Resolution: Duplicate I am pretty sure it's a duplicate of https://issues.apache.org/jira/browse/SPA

[jira] [Commented] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

2016-07-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375574#comment-15375574 ] Sean Owen commented on SPARK-14816: --- I am still not quite sure what it means to have up

[jira] [Commented] (SPARK-14808) Spark MLlib, GraphX, SparkR 2.0 QA umbrella

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376514#comment-15376514 ] Sean Owen commented on SPARK-14808: --- [~josephkb] I'd like to discuss this on dev@. I do

[jira] [Commented] (SPARK-16365) Ideas for moving "mllib-local" forward

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376522#comment-15376522 ] Sean Owen commented on SPARK-16365: --- This might be better on user@ or dev@. I don't kno

[jira] [Commented] (SPARK-16541) SparkTC application could not shutdown successfully

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376537#comment-15376537 ] Sean Owen commented on SPARK-16541: --- This really could be anything. Do you have more in

[jira] [Updated] (SPARK-16500) Add LBFG training not convergence warning for all ML algorithm

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16500: -- Assignee: Weichen Xu Priority: Trivial (was: Major) > Add LBFG training not convergence warning fo

[jira] [Resolved] (SPARK-16500) Add LBFG training not convergence warning for all ML algorithm

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16500. --- Resolution: Fixed Fix Version/s: 2.0.1 Issue resolved by pull request 14157 [https://github.co

[jira] [Resolved] (SPARK-16403) Example cleanup and fix minor issues

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16403. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14081 [https://github.co

[jira] [Updated] (SPARK-16403) Example cleanup and fix minor issues

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16403: -- Assignee: Bryan Cutler Priority: Minor (was: Trivial) > Example cleanup and fix minor issues > ---

[jira] [Updated] (SPARK-16534) Kafka 0.10 Python support

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16534: -- Fix Version/s: (was: 2.0.0) > Kafka 0.10 Python support > - > >

[jira] [Updated] (SPARK-16531) Remove TimeZone from DataFrameTimeWindowingSuite

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16531: -- Assignee: Burak Yavuz > Remove TimeZone from DataFrameTimeWindowingSuite >

[jira] [Updated] (SPARK-16532) Provide a REST API for submitting and tracking status of jobs

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16532: -- External issue URL: (was: https://issues.apache.org/jira/browse/SPARK-5388) Target Version/s:

[jira] [Commented] (SPARK-16541) SparkTC application could not shutdown successfully

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376552#comment-15376552 ] Sean Owen commented on SPARK-16541: --- This doesn't help, just shows things have shut dow

[jira] [Commented] (SPARK-16548) java.io.CharConversionException: Invalid UTF-32 character prevents me from querying my data

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15377352#comment-15377352 ] Sean Owen commented on SPARK-16548: --- Tough call, just because returning 'null' is also

[jira] [Commented] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15377361#comment-15377361 ] Sean Owen commented on SPARK-16522: --- I'm not the release guy, but I also saw that RC3 l

[jira] [Commented] (SPARK-16480) Streaming checkpointing does not work well with SIGTERM

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15377616#comment-15377616 ] Sean Owen commented on SPARK-16480: --- It's just reading serialized data, here apparently

[jira] [Resolved] (SPARK-16549) GC Overhead Limit Reached and Core Dump

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16549. --- Resolution: Fixed "Out of memory" isn't a Spark bug. From your description, it's not clear that you

[jira] [Commented] (SPARK-16528) HiveClientImpl throws NPE when reading database from a custom metastore

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378078#comment-15378078 ] Sean Owen commented on SPARK-16528: --- No, but the latest RC could theoretically be the r

[jira] [Reopened] (SPARK-16549) GC Overhead Limit Reached and Core Dump

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-16549: --- (Oops, didn't mean to resolve yet) > GC Overhead Limit Reached and Core Dump > -

[jira] [Commented] (SPARK-16548) java.io.CharConversionException: Invalid UTF-32 character prevents me from querying my data

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378123#comment-15378123 ] Sean Owen commented on SPARK-16548: --- Yeah sure, but is that much better? the job contin

[jira] [Updated] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16545: -- Fix Version/s: (was: 2.0.0) > Structured Streaming : foreachSink creates the Physical Plan multiple

[jira] [Commented] (SPARK-16550) Caching data with replication doesn't replicate data

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378167#comment-15378167 ] Sean Owen commented on SPARK-16550: --- You have a shell-related problem here that caused

[jira] [Updated] (SPARK-16428) Spark file system watcher not working on Windows

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16428: -- Priority: Major (was: Blocker) > Spark file system watcher not working on Windows > --

[jira] [Updated] (SPARK-16533) Spark application not handling preemption messages

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16533: -- Priority: Major (was: Blocker) > Spark application not handling preemption messages >

[jira] [Updated] (SPARK-16550) Caching data with replication doesn't replicate data

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16550: -- Priority: Major (was: Blocker) > Caching data with replication doesn't replicate data > --

[jira] [Updated] (SPARK-16011) SQL metrics include duplicated attempts

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16011: -- Target Version/s: 2.0.0 I assume you mean this blocks 2.0? > SQL metrics include duplicated attempts >

[jira] [Resolved] (SPARK-15578) Run Spark jobs in Mesos sequentially

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15578. --- Resolution: Invalid > Run Spark jobs in Mesos sequentially > > >

[jira] [Updated] (SPARK-15578) Run Spark jobs in Mesos sequentially

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15578: -- Priority: Major (was: Blocker) Don't set Blocker. I don't think this is true; most likely a problem in

[jira] [Updated] (SPARK-14818) Move sketch, mllibLocal, and hivecontext-compatibility out from mima exclusion

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14818: -- Target Version/s: 2.0.0 (was: ) [~yhuai] what is this a blocker for -- 2.0? > Move sketch, mllibLocal

[jira] [Updated] (SPARK-15118) spark couldn't get hive properyties in hive-site.xml

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15118: -- Priority: Major (was: Blocker) Don't set Blocker. I don't even see a valid issue description here. >

[jira] [Resolved] (SPARK-15118) spark couldn't get hive properyties in hive-site.xml

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15118. --- Resolution: Invalid > spark couldn't get hive properyties in hive-site.xml > ---

[jira] [Updated] (SPARK-16112) R programming guide update for gapply and gapplyCollect

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16112: -- Priority: Major (was: Blocker) > R programming guide update for gapply and gapplyCollect > ---

[jira] [Updated] (SPARK-15666) Join on two tables generated from a same table throwing query analyzer issue

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15666: -- Priority: Major (was: Blocker) > Join on two tables generated from a same table throwing query analyze

[jira] [Updated] (SPARK-16551) Accumulator Examples should demonstrate different use case from UDAFs

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16551: -- Priority: Minor (was: Major) > Accumulator Examples should demonstrate different use case from UDAFs >

[jira] [Commented] (SPARK-14808) Spark MLlib, GraphX, SparkR 2.0 QA umbrella

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378335#comment-15378335 ] Sean Owen commented on SPARK-14808: --- What's the counter-argument though? either some pi

[jira] [Updated] (SPARK-16550) Caching data with replication doesn't replicate data

2016-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16550: -- Priority: Major (was: Blocker) [~shubhamc] don't set Blocker. Even if this is a real issue, it would n

[jira] [Updated] (SPARK-16560) Spark-submit fails without an error

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16560: -- Target Version/s: (was: 1.6.2) Labels: (was: example java newbie spark-submit)

[jira] [Reopened] (SPARK-16564) DeadLock happens when ‘StaticMemoryManager‘ release in-memory block

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-16564: --- Please search JIRA first. It's a duplicate > DeadLock happens when ‘StaticMemoryManager‘ release in-memo

[jira] [Resolved] (SPARK-16564) DeadLock happens when ‘StaticMemoryManager‘ release in-memory block

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16564. --- Resolution: Duplicate > DeadLock happens when ‘StaticMemoryManager‘ release in-memory block > ---

[jira] [Commented] (SPARK-16566) Bug in SparseMatrix multiplication with SparseVector

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379026#comment-15379026 ] Sean Owen commented on SPARK-16566: --- This has been discussed before. I think the issue

[jira] [Resolved] (SPARK-16565) Implementation for processing 50-70 GB data using java......

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16565. --- Resolution: Invalid user@ is the right place for questions, but, this is pretty broad. You might nee

[jira] [Resolved] (SPARK-16567) how to increase performance of rdbms dataframe.

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16567. --- Resolution: Invalid Please don't keep opening these types of issues here. As I say user@ is the plac

[jira] [Commented] (SPARK-16566) Bug in SparseMatrix multiplication with SparseVector

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379087#comment-15379087 ] Sean Owen commented on SPARK-16566: --- Adding it where you did could be OK; I think previ

[jira] [Resolved] (SPARK-16426) IsotonicRegression produces NaNs with certain data

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16426. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14140 [https://github.co

[jira] [Updated] (SPARK-16426) IsotonicRegression produces NaNs with certain data

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16426: -- Assignee: Nic Eggert Priority: Minor (was: Major) > IsotonicRegression produces NaNs with certain

[jira] [Resolved] (SPARK-16357) After enabling Spark shuffle RPC encryption using 3DES, Sparksql query has poor performance when running in parallel.

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16357. --- Resolution: Not A Problem > After enabling Spark shuffle RPC encryption using 3DES, Sparksql query ha

[jira] [Resolved] (SPARK-16325) reduceByKey requires an implicit ordering which it never uses

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16325. --- Resolution: Cannot Reproduce > reduceByKey requires an implicit ordering which it never uses > --

[jira] [Resolved] (SPARK-15213) Unify 'range' usages in PySpark

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15213. --- Resolution: Won't Fix > Unify 'range' usages in PySpark > --- > >

[jira] [Resolved] (SPARK-16105) PCA Reverse Transformer

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16105. --- Resolution: Won't Fix > PCA Reverse Transformer > --- > > Key: SP

[jira] [Resolved] (SPARK-15999) Wrong/Missing information for Spark UI/REST interface

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15999. --- Resolution: Not A Problem This has veered a bit from the original assertion, which I don't think is a

[jira] [Commented] (SPARK-16319) Non-linear (DAG) pipelines need better explanation

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379314#comment-15379314 ] Sean Owen commented on SPARK-16319: --- [~mmoroz] are you going to propose a text change o

[jira] [Resolved] (SPARK-16479) Add Asynchronous Actions Example

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16479. --- Resolution: Won't Fix > Add Asynchronous Actions Example > > >

[jira] [Resolved] (SPARK-13645) DAG Diagram not shown properly in Chrome

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13645. --- Resolution: Duplicate Worth a search of JIRA first > DAG Diagram not shown properly in Chrome >

[jira] [Resolved] (SPARK-16497) Don't throw an exception if drop non-existent TABLE/VIEW/Function/Partitions

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16497. --- Resolution: Won't Fix > Don't throw an exception if drop non-existent TABLE/VIEW/Function/Partitions

[jira] [Resolved] (SPARK-16265) Add option to SparkSubmit to ship driver JRE to YARN

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16265. --- Resolution: Won't Fix > Add option to SparkSubmit to ship driver JRE to YARN > --

[jira] [Updated] (SPARK-15829) spark master webpage links to application UI broke when running in cluster mode

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15829: -- Priority: Major (was: Critical) > spark master webpage links to application UI broke when running in c

[jira] [Commented] (SPARK-16573) executor stderr processing tools

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379862#comment-15379862 ] Sean Owen commented on SPARK-16573: --- I am not so sure this is a Spark issue per se. You

[jira] [Resolved] (SPARK-16572) DStream Kinesis Connector Doc formatting

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16572. --- Resolution: Not A Problem > DStream Kinesis Connector Doc formatting > --

[jira] [Reopened] (SPARK-16572) DStream Kinesis Connector Doc formatting

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-16572: --- > DStream Kinesis Connector Doc formatting > > >

[jira] [Commented] (SPARK-16574) Distribute computing to each node based on certain hints

2016-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379890#comment-15379890 ] Sean Owen commented on SPARK-16574: --- You can target which machines to choose with somet

<    2   3   4   5   6   7   8   9   10   11   >