[jira] [Updated] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-08-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-10316: Description: We did a lot of special handling for non-deterministic expressions in Optimizer.

[jira] [Created] (SPARK-10316) respect nondeterministic expressions in PhysicalOperation

2015-08-27 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-10316: --- Summary: respect nondeterministic expressions in PhysicalOperation Key: SPARK-10316 URL: https://issues.apache.org/jira/browse/SPARK-10316 Project: Spark

[jira] [Assigned] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10316: Assignee: Apache Spark respect non-deterministic expressions in PhysicalOperation

[jira] [Commented] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716786#comment-14716786 ] Apache Spark commented on SPARK-10316: -- User 'cloud-fan' has created a pull request

[jira] [Updated] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10295: -- I believe that YARN currently will release executors even if they have cached data. I also recall that

[jira] [Updated] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10295: -- Component/s: Mesos Dynamic allocation in Mesos does not release when RDDs are cached

[jira] [Updated] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-08-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-10316: Description: We did a lot of special handling for non-deterministic expressions in Optimizer.

[jira] [Commented] (SPARK-6906) Improve Hive integration support

2015-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716616#comment-14716616 ] Thomas Graves commented on SPARK-6906: -- Thanks for the information. I'm trying to

[jira] [Commented] (SPARK-10002) SSH problem during Setup of Spark(1.3.0) cluster on EC2

2015-08-27 Thread Zero tolerance (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716767#comment-14716767 ] Zero tolerance commented on SPARK-10002: I met the same problem. Adding the

[jira] [Updated] (SPARK-10316) respect nondeterministic expressions in PhysicalOperation

2015-08-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-10316: Description: We did a lot of special handling for non-deterministic expressions in (was: We did

[jira] [Updated] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-08-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-10316: Summary: respect non-deterministic expressions in PhysicalOperation (was: respect

[jira] [Assigned] (SPARK-8472) Python API for DCT

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8472: --- Assignee: Apache Spark Python API for DCT -- Key:

[jira] [Assigned] (SPARK-8472) Python API for DCT

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8472: --- Assignee: (was: Apache Spark) Python API for DCT --

[jira] [Commented] (SPARK-8472) Python API for DCT

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716677#comment-14716677 ] Apache Spark commented on SPARK-8472: - User 'yanboliang' has created a pull request

[jira] [Assigned] (SPARK-10316) respect non-deterministic expressions in PhysicalOperation

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10316: Assignee: (was: Apache Spark) respect non-deterministic expressions in

[jira] [Updated] (SPARK-10314) [CORE]RDD persist to OFF_HEAP tachyon got block rdd_x_x not found exception when parallelism is big than data split size

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10314: -- Priority: Minor (was: Major) [CORE]RDD persist to OFF_HEAP tachyon got block rdd_x_x not found

[jira] [Updated] (SPARK-10315) remove document on spark.akka.failure-detector.threshold

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10315: -- Priority: Minor (was: Major) remove document on spark.akka.failure-detector.threshold

[jira] [Updated] (SPARK-10316) respect nondeterministic expressions in PhysicalOperation

2015-08-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-10316: Description: We did a lot of special handling for respect nondeterministic expressions in

[jira] [Created] (SPARK-10319) ALS training using PySpark throws a StackOverflowError

2015-08-27 Thread Velu nambi (JIRA)
Velu nambi created SPARK-10319: -- Summary: ALS training using PySpark throws a StackOverflowError Key: SPARK-10319 URL: https://issues.apache.org/jira/browse/SPARK-10319 Project: Spark Issue

[jira] [Commented] (SPARK-10318) Getting issue in spark connectivity with cassandra

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717139#comment-14717139 ] Sean Owen commented on SPARK-10318: --- I personally don't know, but if this is a question

[jira] [Commented] (SPARK-10320) Support new topic subscriptions without requiring restart of the streaming context

2015-08-27 Thread Sudarshan Kadambi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717220#comment-14717220 ] Sudarshan Kadambi commented on SPARK-10320: --- There is ingest-time analytics

[jira] [Commented] (SPARK-5741) Support the path contains comma in HiveContext

2015-08-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717235#comment-14717235 ] koert kuipers commented on SPARK-5741: -- i am reading avro and csv mostly. but we try

[jira] [Commented] (SPARK-10317) start-history-server.sh CLI parsing incompatible with HistoryServer's arg parsing

2015-08-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716819#comment-14716819 ] Steve Loughran commented on SPARK-10317: There's various possible fixes here #

[jira] [Commented] (SPARK-10318) Getting issue in spark connectivity with cassandra

2015-08-27 Thread Poorvi Lashkary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717049#comment-14717049 ] Poorvi Lashkary commented on SPARK-10318: - I have done the following: private

[jira] [Commented] (SPARK-10318) Getting issue in spark connectivity with cassandra

2015-08-27 Thread Poorvi Lashkary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717093#comment-14717093 ] Poorvi Lashkary commented on SPARK-10318: - can you provide the way to establish

[jira] [Commented] (SPARK-10319) ALS training using PySpark throws a StackOverflowError

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717134#comment-14717134 ] Sean Owen commented on SPARK-10319: --- Definitely sounds like

[jira] [Updated] (SPARK-10320) Support new topic subscriptions without requiring restart of the streaming context

2015-08-27 Thread Sudarshan Kadambi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudarshan Kadambi updated SPARK-10320: -- Description: Spark Streaming lacks the ability to subscribe to newer topics or

[jira] [Resolved] (SPARK-10182) GeneralizedLinearModel doesn't unpersist cached data

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10182. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8395

[jira] [Commented] (SPARK-10320) Support new topic subscriptions without requiring restart of the streaming context

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717184#comment-14717184 ] Sean Owen commented on SPARK-10320: --- It sounds like you listen to topics and processing

[jira] [Created] (SPARK-10321) OrcRelation doesn't override sizeInBytes

2015-08-27 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-10321: -- Summary: OrcRelation doesn't override sizeInBytes Key: SPARK-10321 URL: https://issues.apache.org/jira/browse/SPARK-10321 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4240) Refine Tree Predictions in Gradient Boosting to Improve Prediction Accuracy.

2015-08-27 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717209#comment-14717209 ] Seth Hendrickson commented on SPARK-4240: - [~josephkb] I think there needs to be

[jira] [Created] (SPARK-10320) Support new topic subscriptions without requiring restart of the streaming context

2015-08-27 Thread Sudarshan Kadambi (JIRA)
Sudarshan Kadambi created SPARK-10320: - Summary: Support new topic subscriptions without requiring restart of the streaming context Key: SPARK-10320 URL: https://issues.apache.org/jira/browse/SPARK-10320

[jira] [Commented] (SPARK-5741) Support the path contains comma in HiveContext

2015-08-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717053#comment-14717053 ] koert kuipers commented on SPARK-5741: -- i realize i am late to the party but... by

[jira] [Commented] (SPARK-6817) DataFrame UDFs in R

2015-08-27 Thread Indrajit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717079#comment-14717079 ] Indrajit commented on SPARK-6817: -- Here are some suggestions on the proposed API. If the

[jira] [Commented] (SPARK-5741) Support the path contains comma in HiveContext

2015-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717165#comment-14717165 ] Michael Armbrust commented on SPARK-5741: - What format are you trying to read?

[jira] [Commented] (SPARK-10318) Getting issue in spark connectivity with cassandra

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717078#comment-14717078 ] Sean Owen commented on SPARK-10318: --- This is a Cassandra exception. I don't see that

[jira] [Commented] (SPARK-8292) ShortestPaths run with error result

2015-08-27 Thread Anita Tailor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717115#comment-14717115 ] Anita Tailor commented on SPARK-8292: - No an issue, It's directed graph and there is

[jira] [Resolved] (SPARK-10253) Remove Guava dependencies in MLlib java tests

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10253. --- Resolution: Fixed Fix Version/s: 1.6.0 Remove Guava dependencies in MLlib java tests

[jira] [Resolved] (SPARK-10257) Remove Guava dependencies in spark.mllib JavaTests

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10257. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8451

[jira] [Assigned] (SPARK-9890) User guide for CountVectorizer

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9890: --- Assignee: Apache Spark User guide for CountVectorizer --

[jira] [Assigned] (SPARK-9890) User guide for CountVectorizer

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9890: --- Assignee: (was: Apache Spark) User guide for CountVectorizer

[jira] [Commented] (SPARK-6918) Secure HBase with Kerberos does not work over YARN

2015-08-27 Thread LINTE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716806#comment-14716806 ] LINTE commented on SPARK-6918: -- Is this issue really fixed ? I work with secure hadoop 2.7.1

[jira] [Commented] (SPARK-9890) User guide for CountVectorizer

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716836#comment-14716836 ] Apache Spark commented on SPARK-9890: - User 'hhbyyh' has created a pull request for

[jira] [Commented] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716992#comment-14716992 ] Marcelo Vanzin commented on SPARK-10295: In 1.5 executors with cached data are

[jira] [Created] (SPARK-10317) start-history-server.sh CLI parsing incompatible with HistoryServer's arg parsing

2015-08-27 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-10317: -- Summary: start-history-server.sh CLI parsing incompatible with HistoryServer's arg parsing Key: SPARK-10317 URL: https://issues.apache.org/jira/browse/SPARK-10317

[jira] [Created] (SPARK-10318) Getting issue in spark connectivity with cassandra

2015-08-27 Thread Poorvi Lashkary (JIRA)
Poorvi Lashkary created SPARK-10318: --- Summary: Getting issue in spark connectivity with cassandra Key: SPARK-10318 URL: https://issues.apache.org/jira/browse/SPARK-10318 Project: Spark

[jira] [Commented] (SPARK-5741) Support the path contains comma in HiveContext

2015-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717258#comment-14717258 ] Michael Armbrust commented on SPARK-5741: - It was originally just parquet that

[jira] [Commented] (SPARK-10319) ALS training using PySpark throws a StackOverflowError

2015-08-27 Thread Velu nambi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717270#comment-14717270 ] Velu nambi commented on SPARK-10319: bq. do you see evidence of checkpointing in the

[jira] [Resolved] (SPARK-9148) User-facing documentation for NaN handling semantics

2015-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9148. - Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8441

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717297#comment-14717297 ] Felix Cheung commented on SPARK-9316: -

[jira] [Resolved] (SPARK-10252) Update Spark SQL Programming Guide for Spark 1.5

2015-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10252. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8441

[jira] [Created] (SPARK-10322) Column %in% function is not working

2015-08-27 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-10322: Summary: Column %in% function is not working Key: SPARK-10322 URL: https://issues.apache.org/jira/browse/SPARK-10322 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10320) Support new topic subscriptions without requiring restart of the streaming context

2015-08-27 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717342#comment-14717342 ] Cody Koeninger commented on SPARK-10320: As I said on the list, the best way to

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2015-08-27 Thread Marcel Mitsuto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717374#comment-14717374 ] Marcel Mitsuto commented on SPARK-4105: --- mapPartitions at Exchange.scala:60 +details

[jira] [Comment Edited] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717297#comment-14717297 ] Felix Cheung edited comment on SPARK-9316 at 8/27/15 6:58 PM: --

[jira] [Resolved] (SPARK-10315) remove document on spark.akka.failure-detector.threshold

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10315. --- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull

[jira] [Updated] (SPARK-10322) Column %in% function is not exported

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-10322: - Summary: Column %in% function is not exported (was: Column %in% function is not working)

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717328#comment-14717328 ] Felix Cheung commented on SPARK-9316: - As for this, subsetdf - df[age in (19,

[jira] [Comment Edited] (SPARK-10319) ALS training using PySpark throws a StackOverflowError

2015-08-27 Thread Velu nambi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717263#comment-14717263 ] Velu nambi edited comment on SPARK-10319 at 8/27/15 6:42 PM: -

[jira] [Commented] (SPARK-10319) ALS training using PySpark throws a StackOverflowError

2015-08-27 Thread Velu nambi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717263#comment-14717263 ] Velu nambi commented on SPARK-10319: Yes it does seem similar to SPARK-5955, it works

[jira] [Assigned] (SPARK-9148) User-facing documentation for NaN handling semantics

2015-08-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-9148: --- Assignee: Michael Armbrust User-facing documentation for NaN handling semantics

[jira] [Commented] (SPARK-10322) Column %in% function is not exported

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717305#comment-14717305 ] Felix Cheung commented on SPARK-10322: --

[jira] [Resolved] (SPARK-10322) Column %in% function is not exported

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-10322. -- Resolution: Duplicate Looks like this was fixed last night. Column %in% function is not

[jira] [Commented] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717444#comment-14717444 ] Apache Spark commented on SPARK-10295: -- User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10295: -- Component/s: (was: Mesos) Spark Core Documentation Issue

[jira] [Updated] (SPARK-10304) Partition discovery does not throw an exception if the dir structure is valid

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10304: - Summary: Partition discovery does not throw an exception if the dir structure is valid (was: Need to

[jira] [Assigned] (SPARK-10321) OrcRelation doesn't override sizeInBytes

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10321: Assignee: Apache Spark OrcRelation doesn't override sizeInBytes

[jira] [Assigned] (SPARK-10321) OrcRelation doesn't override sizeInBytes

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10321: Assignee: (was: Apache Spark) OrcRelation doesn't override sizeInBytes

[jira] [Comment Edited] (SPARK-5741) Support the path contains comma in HiveContext

2015-08-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717450#comment-14717450 ] koert kuipers edited comment on SPARK-5741 at 8/27/15 8:22 PM:

[jira] [Commented] (SPARK-10321) OrcRelation doesn't override sizeInBytes

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717453#comment-14717453 ] Apache Spark commented on SPARK-10321: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-10321) OrcRelation doesn't override sizeInBytes

2015-08-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-10321: -- Assignee: Davies Liu OrcRelation doesn't override sizeInBytes

[jira] [Assigned] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10295: Assignee: (was: Apache Spark) Dynamic allocation in Mesos does not release when RDDs

[jira] [Assigned] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached

2015-08-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10295: Assignee: Apache Spark Dynamic allocation in Mesos does not release when RDDs are cached

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Deborah Siegel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717473#comment-14717473 ] Deborah Siegel commented on SPARK-9316: --- Now that %in% is exported in namespace,

[jira] [Commented] (SPARK-10304) Need to add a null check in unwrapperFor in HiveInspectors

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717475#comment-14717475 ] Yin Huai commented on SPARK-10304: -- Will field be null? I will try to get more info.

[jira] [Updated] (SPARK-10304) Partition discovery does not throw an exception if the dir structure is valid

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10304: - Description: I have a dir structure like {{/path/table1/partition_column=1/}}. When I try to use

[jira] [Commented] (SPARK-5741) Support the path contains comma in HiveContext

2015-08-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717450#comment-14717450 ] koert kuipers commented on SPARK-5741: -- given the requirement of source/binary

[jira] [Resolved] (SPARK-9901) User guide for RowMatrix Tall-and-skinny QR

2015-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9901. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8462

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717531#comment-14717531 ] Shivaram Venkataraman commented on SPARK-9316: -- I don't think supporting the

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated into ONE line during the script transform due to missing the correct line/filed delimeter

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10310: - Priority: Critical (was: Blocker) [Spark SQL] All result records will be popluated into ONE line

[jira] [Commented] (SPARK-10304) Partition discovery does not throw an exception if the dir structure is valid

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717575#comment-14717575 ] Yin Huai commented on SPARK-10304: -- [~zhazhan] just took a look, it is not an ORC issue.

[jira] [Commented] (SPARK-10296) add preservesParitioning parameter to RDD.map

2015-08-27 Thread Esteban Donato (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717713#comment-14717713 ] Esteban Donato commented on SPARK-10296: Sean, thanks your your response. As per

[jira] [Updated] (SPARK-10323) NPE in code-gened In expression

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10323: - Assignee: Davies Liu NPE in code-gened In expression ---

[jira] [Comment Edited] (SPARK-8514) LU factorization on BlockMatrix

2015-08-27 Thread Jerome (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717696#comment-14717696 ] Jerome edited comment on SPARK-8514 at 8/27/15 11:03 PM: - I have a

[jira] [Updated] (SPARK-10287) After processing a query using JSON data, Spark SQL continuously refreshes metadata of the table

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10287: - Labels: releasenotes (was: ) After processing a query using JSON data, Spark SQL continuously

[jira] [Updated] (SPARK-10287) After processing a query using JSON data, Spark SQL continuously refreshes metadata of the table

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10287: - Target Version/s: (was: 1.5.0) After processing a query using JSON data, Spark SQL continuously

[jira] [Resolved] (SPARK-10287) After processing a query using JSON data, Spark SQL continuously refreshes metadata of the table

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10287. -- Resolution: Fixed Fix Version/s: 1.5.1 Issue resolved by pull request 8469

[jira] [Commented] (SPARK-10287) After processing a query using JSON data, Spark SQL continuously refreshes metadata of the table

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717737#comment-14717737 ] Yin Huai commented on SPARK-10287: -- We need to put the following release note JSON data

[jira] [Commented] (SPARK-8514) LU factorization on BlockMatrix

2015-08-27 Thread Jerome (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717696#comment-14717696 ] Jerome commented on SPARK-8514: --- I have a draft of the LU Decomposition in BlockMatrix.scala

[jira] [Commented] (SPARK-10307) Fix regression in block matrix multiply (1.4-1.5 regression)

2015-08-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717704#comment-14717704 ] Joseph K. Bradley commented on SPARK-10307: --- I tested this a number of times to

[jira] [Resolved] (SPARK-9680) Update programming guide section for ml.feature.StopWordsRemover

2015-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9680. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8436

[jira] [Resolved] (SPARK-9906) User guide for LogisticRegressionSummary

2015-08-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-9906. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8197

[jira] [Resolved] (SPARK-10307) Fix regression in block matrix multiply (1.4-1.5 regression)

2015-08-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-10307. --- Resolution: Cannot Reproduce Fix regression in block matrix multiply (1.4-1.5

[jira] [Created] (SPARK-10323) NPE in code-gened In expression

2015-08-27 Thread Yin Huai (JIRA)
Yin Huai created SPARK-10323: Summary: NPE in code-gened In expression Key: SPARK-10323 URL: https://issues.apache.org/jira/browse/SPARK-10323 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2015-08-27 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-4066: -- Description: Here is the thread Koert started:

[jira] [Commented] (SPARK-10323) NPE in code-gened In expression

2015-08-27 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717762#comment-14717762 ] Yin Huai commented on SPARK-10323: -- Seems {{array_contains}} does not have this NPE

[jira] [Created] (SPARK-10324) MLlib 1.6 Roadmap

2015-08-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-10324: - Summary: MLlib 1.6 Roadmap Key: SPARK-10324 URL: https://issues.apache.org/jira/browse/SPARK-10324 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-10326) Cannot launch YARN job on Windows

2015-08-27 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-10326: -- Summary: Cannot launch YARN job on Windows Key: SPARK-10326 URL: https://issues.apache.org/jira/browse/SPARK-10326 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10329) Cost RDD in k-means initialization is not storage-efficient

2015-08-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10329: -- Description: Currently we use `RDD[Vector]` to store point cost during k-means||

[jira] [Created] (SPARK-10329) Cost RDD in k-means initialization is not storage-efficient

2015-08-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-10329: - Summary: Cost RDD in k-means initialization is not storage-efficient Key: SPARK-10329 URL: https://issues.apache.org/jira/browse/SPARK-10329 Project: Spark

  1   2   >