[jira] [Commented] (SPARK-16733) use antlr to parse column name instead of implementing it ourselves

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395163#comment-15395163 ] Reynold Xin commented on SPARK-16733: - Putting the comment here as well: So I think the proper fix

[jira] [Created] (SPARK-16749) Clean-up OffsetWindowFrame

2016-07-27 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-16749: - Summary: Clean-up OffsetWindowFrame Key: SPARK-16749 URL: https://issues.apache.org/jira/browse/SPARK-16749 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-16729) Spark should throw analysis exception for invalid casts to date type

2016-07-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16729. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14358

[jira] [Comment Edited] (SPARK-14489) RegressionEvaluator returns NaN for ALS in Spark ml

2016-07-27 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395275#comment-15395275 ] Nick Pentreath edited comment on SPARK-14489 at 7/27/16 9:03 AM: - Thanks

[jira] [Updated] (SPARK-16743) converter and access code out of sync: createDataFrame on RDD[Option[C]] fails with MatchError

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16743: -- Affects Version/s: (was: 1.6.1) Priority: Minor (was: Major) Issue Type:

[jira] [Commented] (SPARK-15817) Spark client picking hive 1.2.1 by default which failed to alter a table name

2016-07-27 Thread fengchaoge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395261#comment-15395261 ] fengchaoge commented on SPARK-15817: spark.sql.hive.metastore.jars

[jira] [Resolved] (SPARK-16733) use antlr to parse column name instead of implementing it ourselves

2016-07-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16733. - Resolution: Won't Fix > use antlr to parse column name instead of implementing it ourselves >

[jira] [Updated] (SPARK-16729) Spark should throw analysis exception for invalid casts to date type

2016-07-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16729: Assignee: Peter Lee > Spark should throw analysis exception for invalid casts to date type >

[jira] [Assigned] (SPARK-16749) Clean-up OffsetWindowFrame

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16749: Assignee: Herman van Hovell (was: Apache Spark) > Clean-up OffsetWindowFrame >

[jira] [Assigned] (SPARK-16749) Clean-up OffsetWindowFrame

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16749: Assignee: Apache Spark (was: Herman van Hovell) > Clean-up OffsetWindowFrame >

[jira] [Updated] (SPARK-16621) Generate stable SQLs in SQLBuilder

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16621: Fix Version/s: 2.0.1 > Generate stable SQLs in SQLBuilder > -- > >

[jira] [Commented] (SPARK-16745) Spark job completed however have to wait for 13 mins (data size is small)

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395194#comment-15395194 ] Sean Owen commented on SPARK-16745: --- I think the answer is reasonably clear here -- it's delayed in

[jira] [Commented] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395196#comment-15395196 ] Sean Owen commented on SPARK-16746: --- If you're saying that the error is in timing out or being unable

[jira] [Commented] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395218#comment-15395218 ] Herman van Hovell commented on SPARK-15232: --- [~dongjoon] off course you can! > Add subquery

[jira] [Comment Edited] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395218#comment-15395218 ] Herman van Hovell edited comment on SPARK-15232 at 7/27/16 8:24 AM:

[jira] [Commented] (SPARK-16749) Clean-up OffsetWindowFrame

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395273#comment-15395273 ] Apache Spark commented on SPARK-16749: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Commented] (SPARK-14489) RegressionEvaluator returns NaN for ALS in Spark ml

2016-07-27 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395275#comment-15395275 ] Nick Pentreath commented on SPARK-14489: Thanks for the thoughts Krishna. # Initially I also

[jira] [Updated] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-16750: Description: ML GaussianMixture training failed due to feature column type mistake. The feature

[jira] [Updated] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-16750: Description: ML GaussianMixture training failed due to feature column type mistake. The feature

[jira] [Updated] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-16750: Description: ML GaussianMixture training failed due to feature column type mistake. The feature

[jira] [Assigned] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16750: Assignee: Yanbo Liang (was: Apache Spark) > ML GaussianMixture training failed due to

[jira] [Assigned] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16750: Assignee: Apache Spark (was: Yanbo Liang) > ML GaussianMixture training failed due to

[jira] [Commented] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395338#comment-15395338 ] Apache Spark commented on SPARK-16750: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Commented] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395354#comment-15395354 ] Hongyao Zhao commented on SPARK-16746: -- I think if spark.streaming.receiver.writeAheadLog.enable is

[jira] [Resolved] (SPARK-15817) Spark client picking hive 1.2.1 by default which failed to alter a table name

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15817. --- Resolution: Not A Problem > Spark client picking hive 1.2.1 by default which failed to alter a table

[jira] [Commented] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395434#comment-15395434 ] Alexander Ulanov commented on SPARK-9120: - Thanks for the comment, RegressionModel does not extend

[jira] [Assigned] (SPARK-14489) RegressionEvaluator returns NaN for ALS in Spark ml

2016-07-27 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-14489: -- Assignee: Nick Pentreath > RegressionEvaluator returns NaN for ALS in Spark ml >

[jira] [Comment Edited] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395354#comment-15395354 ] Hongyao Zhao edited comment on SPARK-16746 at 7/27/16 10:06 AM: I think

[jira] [Commented] (SPARK-16625) Oracle JDBC table creation fails with ORA-00902: invalid datatype

2016-07-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395307#comment-15395307 ] Yuming Wang commented on SPARK-16625: - {code:java} val jdbcUrl =

[jira] [Commented] (SPARK-16625) Oracle JDBC table creation fails with ORA-00902: invalid datatype

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395311#comment-15395311 ] Apache Spark commented on SPARK-16625: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16625) Oracle JDBC table creation fails with ORA-00902: invalid datatype

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16625: Assignee: Apache Spark > Oracle JDBC table creation fails with ORA-00902: invalid

[jira] [Commented] (SPARK-16606) Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395378#comment-15395378 ] Sean Owen commented on SPARK-16606: --- [~ja...@japila.pl] are you working on this or should I zap it? >

[jira] [Commented] (SPARK-16606) Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

2016-07-27 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395420#comment-15395420 ] Jacek Laskowski commented on SPARK-16606: - Haven't really started to be honest. I'd like to work

[jira] [Assigned] (SPARK-16625) Oracle JDBC table creation fails with ORA-00902: invalid datatype

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16625: Assignee: (was: Apache Spark) > Oracle JDBC table creation fails with ORA-00902:

[jira] [Created] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-16750: --- Summary: ML GaussianMixture training failed due to feature column type mistake Key: SPARK-16750 URL: https://issues.apache.org/jira/browse/SPARK-16750 Project: Spark

[jira] [Commented] (SPARK-16085) Spark stand-alone ui redirects to RM application master UI for yarn-client mode

2016-07-27 Thread Lve Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395334#comment-15395334 ] Lve Chen commented on SPARK-16085: -- [~srowen] [~jerryshao] I'm in a similar circumstance where we are

[jira] [Commented] (SPARK-10627) Regularization for artificial neural networks

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395379#comment-15395379 ] Alexander Ulanov commented on SPARK-10627: -- [~RubenJanssen] These are major features. Could you

[jira] [Updated] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-9120: Description: org.apache.spark.ml.regression.RegressionModel supports prediction only for a

[jira] [Updated] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Pran updated SPARK-16752: - Description: We are having a strange issue with Spark Job Server (SJS) We are using SJS 0.6.1 and Spark

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-07-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395815#comment-15395815 ] Maciej BryƄski commented on SPARK-15541: [~srowen] I have patch that resolve this issue. Do you

[jira] [Created] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
Ash Pran created SPARK-16752: Summary: Spark Job Server not releasing jobs from "running list" even after yarn completes the job Key: SPARK-16752 URL: https://issues.apache.org/jira/browse/SPARK-16752

[jira] [Resolved] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16752. Resolution: Invalid This is not the place to report bugs about the SJS, which is unrelated

[jira] [Updated] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Pran updated SPARK-16752: - Attachment: SJS_JOBS_RUNNING SJS_JOB_LOG_CONSOLE SJS_JOB_COMP_YARN

[jira] [Comment Edited] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395907#comment-15395907 ] Ash Pran edited comment on SPARK-16752 at 7/27/16 4:11 PM: --- Please see the

[jira] [Updated] (SPARK-16762) spark hanging when action method print

2016-07-27 Thread Anh Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anh Nguyen updated SPARK-16762: --- Attachment: Screen Shot 2016-07-28 at 12.34.22 PM.png > spark hanging when action method print >

[jira] [Updated] (SPARK-16762) spark hanging when action method print

2016-07-27 Thread Anh Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anh Nguyen updated SPARK-16762: --- Attachment: Screen Shot 2016-07-28 at 12.33.35 PM.png > spark hanging when action method print >

[jira] [Resolved] (SPARK-5847) Allow for configuring MetricsSystem's use of app ID to namespace all metrics

2016-07-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5847. --- Resolution: Fixed Assignee: Mark Grover Fix Version/s: 2.1.0 > Allow for

[jira] [Commented] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2016-07-27 Thread Matthew Gwynne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395938#comment-15395938 ] Matthew Gwynne commented on SPARK-15872: I have had what appears to be exactly the same issue in

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396053#comment-15396053 ] Jurriaan Pruis commented on SPARK-16753: [~rxin] Do you know something about this? I've seen a

[jira] [Created] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
Jurriaan Pruis created SPARK-16753: -- Summary: Spark SQL doesn't handle skewed dataset joins properly Key: SPARK-16753 URL: https://issues.apache.org/jira/browse/SPARK-16753 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2016-07-27 Thread Matthew Gwynne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Gwynne updated SPARK-15872: --- Comment: was deleted (was: I have had what appears to be exactly the same issue in 1.6.2.

[jira] [Commented] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396082#comment-15396082 ] Shixiong Zhu commented on SPARK-16754: -- The issue is because OuterScopes.getOuterScope assumes

[jira] [Assigned] (SPARK-16673) New Executor Page displays columns that used to be conditionally hidden

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16673: Assignee: Apache Spark > New Executor Page displays columns that used to be conditionally

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396078#comment-15396078 ] Jurriaan Pruis commented on SPARK-16753: This is looks like a skew problem to me since the tasks

[jira] [Commented] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-27 Thread Niels Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396032#comment-15396032 ] Niels Becker commented on SPARK-16450: -- Its out > Build failes for Mesos 0.28.x >

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396067#comment-15396067 ] Reynold Xin commented on SPARK-16753: - Is this a problem with skew, or a problem with GC? Have you

[jira] [Created] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-16754: Summary: NPE when defining case class and searching Encoder in the same line Key: SPARK-16754 URL: https://issues.apache.org/jira/browse/SPARK-16754 Project: Spark

[jira] [Commented] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396085#comment-15396085 ] Shixiong Zhu commented on SPARK-16754: -- Perhaps we should inject

[jira] [Commented] (SPARK-16673) New Executor Page displays columns that used to be conditionally hidden

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396099#comment-15396099 ] Apache Spark commented on SPARK-16673: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16673) New Executor Page displays columns that used to be conditionally hidden

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16673: Assignee: (was: Apache Spark) > New Executor Page displays columns that used to be

[jira] [Updated] (SPARK-15675) Implicit resolution doesn't work in multiple Statements in Spark Repl

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15675: - Priority: Major (was: Minor) > Implicit resolution doesn't work in multiple Statements in Spark

[jira] [Updated] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16754: - Priority: Minor (was: Major) > NPE when defining case class and searching Encoder in the same

[jira] [Updated] (SPARK-15675) Implicit resolution doesn't work in multiple Statements in Spark Repl

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15675: - Priority: Minor (was: Major) > Implicit resolution doesn't work in multiple Statements in Spark

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396209#comment-15396209 ] Jurriaan Pruis commented on SPARK-16753: It's not only more memory, they also take up a lot more

[jira] [Resolved] (SPARK-16755) Enable external shuffle service with Mesos Docker Containers

2016-07-27 Thread Niels Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Becker resolved SPARK-16755. -- Resolution: Not A Problem > Enable external shuffle service with Mesos Docker Containers >

[jira] [Created] (SPARK-16755) Enable external shuffle service with Mesos Docker Containers

2016-07-27 Thread Niels Becker (JIRA)
Niels Becker created SPARK-16755: Summary: Enable external shuffle service with Mesos Docker Containers Key: SPARK-16755 URL: https://issues.apache.org/jira/browse/SPARK-16755 Project: Spark

[jira] [Assigned] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15232: Assignee: (was: Apache Spark) > Add subquery SQL building tests to

[jira] [Commented] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396283#comment-15396283 ] Apache Spark commented on SPARK-15232: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15232: Assignee: Apache Spark > Add subquery SQL building tests to LogicalPlanToSQLSuite >

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396192#comment-15396192 ] Reynold Xin commented on SPARK-16753: - Yea I understand some tasks are using more memory due to data

[jira] [Commented] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-27 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396236#comment-15396236 ] Michael Gummelt commented on SPARK-16450: - I'll update soon. Though this pending PR updates to

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396340#comment-15396340 ] Dongjoon Hyun commented on SPARK-16756: --- If there is anything you have in mind, please give me

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396341#comment-15396341 ] Reynold Xin commented on SPARK-16756: - That makes sense. > Add `sql` function to LogicalPlan and

[jira] [Updated] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16756: -- Description: Like `Expression`, this issue adds `sql` function for `LogicalPlan` and

[jira] [Resolved] (SPARK-16110) Can't set Python via spark-submit for YARN cluster mode when PYSPARK_PYTHON & PYSPARK_DRIVER_PYTHON are set

2016-07-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16110. Resolution: Fixed Assignee: Kevin Grealish Fix Version/s: 2.1.0 > Can't

[jira] [Created] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-16756: - Summary: Add `sql` function to LogicalPlan and `NonSQLPlan` trait Key: SPARK-16756 URL: https://issues.apache.org/jira/browse/SPARK-16756 Project: Spark

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396331#comment-15396331 ] Reynold Xin commented on SPARK-16756: - This makes sense. Are there flags we might want to pass into

[jira] [Comment Edited] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396337#comment-15396337 ] Dongjoon Hyun edited comment on SPARK-16756 at 7/27/16 8:45 PM: Until

[jira] [Commented] (SPARK-16365) Ideas for moving "mllib-local" forward

2016-07-27 Thread Jay White Bear (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396480#comment-15396480 ] Jay White Bear commented on SPARK-16365: The discussion is about adding a full blown linear

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396321#comment-15396321 ] Dongjoon Hyun commented on SPARK-16756: --- Hi, [~rxin]. So far, I've been working in this direction.

[jira] [Updated] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Roberts updated SPARK-16751: - Description: This JIRA is to upgrade the derby version from 10.11.1.1 to 10.12.1.1 Sean and I

[jira] [Updated] (SPARK-16729) Spark should throw analysis exception for invalid casts to date type

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16729: Fix Version/s: 2.0.1 > Spark should throw analysis exception for invalid casts to date type >

[jira] [Resolved] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16450. --- Resolution: Duplicate This is subsumed by the containerizer change, which updates for Mesos 1.x >

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396651#comment-15396651 ] Sean Owen commented on SPARK-15541: --- What's your fix -- avoiding directly calling this method? >

[jira] [Commented] (SPARK-15857) Add Caller Context in Spark

2016-07-27 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396677#comment-15396677 ] Weiqing Yang commented on SPARK-15857: -- That single PR is for subtasks 1, 2 and 4. Based on the

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396337#comment-15396337 ] Dongjoon Hyun commented on SPARK-16756: --- Until now, there are no flags. I'm trying to not use any

[jira] [Updated] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16751: -- Priority: Minor (was: Critical) We can do this, sure, but why is it Critical? tests have zero surface

[jira] [Created] (SPARK-16759) Spark expose an API to pass in Caller Context into it

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16759: Summary: Spark expose an API to pass in Caller Context into it Key: SPARK-16759 URL: https://issues.apache.org/jira/browse/SPARK-16759 Project: Spark Issue

[jira] [Created] (SPARK-16760) Pass 'jobId' to Task

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16760: Summary: Pass 'jobId' to Task Key: SPARK-16760 URL: https://issues.apache.org/jira/browse/SPARK-16760 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-15857) Add Caller Context in Spark

2016-07-27 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396583#comment-15396583 ] Weiqing Yang commented on SPARK-15857: -- To make review easier, subtasks have been created. > Add

[jira] [Commented] (SPARK-15857) Add Caller Context in Spark

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396638#comment-15396638 ] Sean Owen commented on SPARK-15857: --- How about elaborating with a design doc before making all these

[jira] [Created] (SPARK-16758) Set up caller context to YARN

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16758: Summary: Set up caller context to YARN Key: SPARK-16758 URL: https://issues.apache.org/jira/browse/SPARK-16758 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-16443) ALS wrapper in SparkR

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16443: Assignee: Apache Spark (was: Junyang Qian) > ALS wrapper in SparkR >

[jira] [Commented] (SPARK-16443) ALS wrapper in SparkR

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396580#comment-15396580 ] Apache Spark commented on SPARK-16443: -- User 'junyangq' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16443) ALS wrapper in SparkR

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16443: Assignee: Junyang Qian (was: Apache Spark) > ALS wrapper in SparkR >

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Commented] (SPARK-16745) Spark job completed however have to wait for 13 mins (data size is small)

2016-07-27 Thread Joe Chong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395567#comment-15395567 ] Joe Chong commented on SPARK-16745: --- Thanks for the reply. This is a standalone Spark instantiation.

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

  1   2   >