[jira] [Assigned] (SPARK-7338) Survival Modelling - Cox proportional hazards

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7338: --- Assignee: (was: Apache Spark) > Survival Modelling - Cox proportional hazards > -

[jira] [Created] (SPARK-7738) DataFramer reader/writer API in Python

2015-05-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-7738: - Summary: DataFramer reader/writer API in Python Key: SPARK-7738 URL: https://issues.apache.org/jira/browse/SPARK-7738 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-3644) REST API for Spark application info (jobs / stages / tasks / storage info)

2015-05-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550981#comment-14550981 ] Josh Rosen commented on SPARK-3644: --- I'm going to close this as fixed in 1.4.0, since SP

[jira] [Resolved] (SPARK-3644) REST API for Spark application info (jobs / stages / tasks / storage info)

2015-05-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3644. --- Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Imran Rashid > REST API for Spark app

[jira] [Resolved] (SPARK-7678) Scala ML seed Param should be fixed but vary per class

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7678. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6251 [https://githu

[jira] [Closed] (SPARK-7696) Aggregate function's result should be nullable only if the input expression is nullable

2015-05-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7696. -- Resolution: Not A Problem > Aggregate function's result should be nullable only if the input expression

[jira] [Resolved] (SPARK-7047) Model parent should be Optional

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7047. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5914 [https://githu

[jira] [Updated] (SPARK-7382) Python API for ml.classification

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7382: - Fix Version/s: 1.4.0 > Python API for ml.classification > > >

[jira] [Closed] (SPARK-7381) Missing Python API for o.a.s.ml

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7381. Resolution: Fixed Fix Version/s: 1.4.0 I'm closing this JIRA. Thanks [~brkyvz] for boosting t

[jira] [Updated] (SPARK-7381) Missing Python API for o.a.s.ml in 1.4

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7381: - Summary: Missing Python API for o.a.s.ml in 1.4 (was: Missing Python API for o.a.s.ml) > Missing

[jira] [Commented] (SPARK-6907) Create an isolated classloader for the Hive Client.

2015-05-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550869#comment-14550869 ] Michael Armbrust commented on SPARK-6907: - Steve, thanks for the feedback. The iv

[jira] [Updated] (SPARK-7443) MLlib 1.4 QA plan

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7443: - Description: TODO: create JIRAs for each task and assign them accordingly. h2. API * Check API c

[jira] [Assigned] (SPARK-7537) Audit new public Scala APIs for MLlib 1.4

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-7537: Assignee: Xiangrui Meng > Audit new public Scala APIs for MLlib 1.4 > -

[jira] [Commented] (SPARK-7720) MLlib 1.3 -> 1.4 migration guide

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550865#comment-14550865 ] Xiangrui Meng commented on SPARK-7720: -- No, I didn't realize that SPARK-7715 covered

[jira] [Closed] (SPARK-7720) MLlib 1.3 -> 1.4 migration guide

2015-05-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7720. Resolution: Duplicate > MLlib 1.3 -> 1.4 migration guide > > >

[jira] [Commented] (SPARK-6907) Create an isolated classloader for the Hive Client.

2015-05-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550808#comment-14550808 ] Steve Loughran commented on SPARK-6907: --- Having just looked at this code, I'm a bit

[jira] [Updated] (SPARK-7704) Updating Programming Guides per SPARK-4397

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7704: - Assignee: Daisuke Kobayashi > Updating Programming Guides per SPARK-4397 > ---

[jira] [Updated] (SPARK-7704) Updating Programming Guides per SPARK-4397

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7704: - Priority: Trivial (was: Major) > Updating Programming Guides per SPARK-4397 > ---

[jira] [Resolved] (SPARK-7704) Updating Programming Guides per SPARK-4397

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7704. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6234 [https://github.com/ap

[jira] [Updated] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7726: - Target Version/s: 1.4.0 Hoping to get this back in for 1.4.0. > Maven Install Breaks When Upgrading Scala

[jira] [Assigned] (SPARK-7734) make explode support struct type

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7734: --- Assignee: (was: Apache Spark) > make explode support struct type > --

[jira] [Commented] (SPARK-7734) make explode support struct type

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550761#comment-14550761 ] Apache Spark commented on SPARK-7734: - User 'cloud-fan' has created a pull request for

[jira] [Assigned] (SPARK-7734) make explode support struct type

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7734: --- Assignee: Apache Spark > make explode support struct type >

[jira] [Created] (SPARK-7737) parquet schema discovery should not fail because of empty _temporary dir

2015-05-19 Thread Yin Huai (JIRA)
Yin Huai created SPARK-7737: --- Summary: parquet schema discovery should not fail because of empty _temporary dir Key: SPARK-7737 URL: https://issues.apache.org/jira/browse/SPARK-7737 Project: Spark

[jira] [Resolved] (SPARK-7732) org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree:

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7732. -- Resolution: Invalid This is more of a question at this point, not a clear issue report. I'd ask on user

[jira] [Created] (SPARK-7736) Exception not failing Python applications (in yarn cluster mode)

2015-05-19 Thread Shay Rojansky (JIRA)
Shay Rojansky created SPARK-7736: Summary: Exception not failing Python applications (in yarn cluster mode) Key: SPARK-7736 URL: https://issues.apache.org/jira/browse/SPARK-7736 Project: Spark

[jira] [Resolved] (SPARK-4808) Spark fails to spill with small number of large objects

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4808. -- Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Mingyu Kim > Spark fails to spill with

[jira] [Updated] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7726: --- Assignee: Iulian Dragos > Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher

[jira] [Commented] (SPARK-6246) spark-ec2 can't handle clusters with > 100 nodes

2015-05-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550652#comment-14550652 ] Shivaram Venkataraman commented on SPARK-6246: -- Could you send a PR for this

[jira] [Commented] (SPARK-6246) spark-ec2 can't handle clusters with > 100 nodes

2015-05-19 Thread Alex (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550637#comment-14550637 ] Alex commented on SPARK-6246: - This can be fixed by replacing the line in file ec2/spark_ec2.p

[jira] [Assigned] (SPARK-7735) Raise Exception on non-zero exit from pyspark pipe commands

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7735: --- Assignee: (was: Apache Spark) > Raise Exception on non-zero exit from pyspark pipe comman

[jira] [Assigned] (SPARK-7735) Raise Exception on non-zero exit from pyspark pipe commands

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7735: --- Assignee: Apache Spark > Raise Exception on non-zero exit from pyspark pipe commands > --

[jira] [Commented] (SPARK-7735) Raise Exception on non-zero exit from pyspark pipe commands

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550611#comment-14550611 ] Apache Spark commented on SPARK-7735: - User 'megatron-me-uk' has created a pull reques

[jira] [Commented] (SPARK-7733) Update build, code to use Java 7 for 1.5.0+

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550610#comment-14550610 ] Apache Spark commented on SPARK-7733: - User 'srowen' has created a pull request for th

[jira] [Assigned] (SPARK-7733) Update build, code to use Java 7 for 1.5.0+

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7733: --- Assignee: Sean Owen (was: Apache Spark) > Update build, code to use Java 7 for 1.5.0+ >

[jira] [Assigned] (SPARK-7733) Update build, code to use Java 7 for 1.5.0+

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7733: --- Assignee: Apache Spark (was: Sean Owen) > Update build, code to use Java 7 for 1.5.0+ >

[jira] [Created] (SPARK-7735) Raise Exception on non-zero exit from pyspark pipe commands

2015-05-19 Thread Scott Taylor (JIRA)
Scott Taylor created SPARK-7735: --- Summary: Raise Exception on non-zero exit from pyspark pipe commands Key: SPARK-7735 URL: https://issues.apache.org/jira/browse/SPARK-7735 Project: Spark Issu

[jira] [Created] (SPARK-7733) Update build, code to use Java 7 for 1.5.0+

2015-05-19 Thread Sean Owen (JIRA)
Sean Owen created SPARK-7733: Summary: Update build, code to use Java 7 for 1.5.0+ Key: SPARK-7733 URL: https://issues.apache.org/jira/browse/SPARK-7733 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-7734) make explode support struct type

2015-05-19 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-7734: -- Summary: make explode support struct type Key: SPARK-7734 URL: https://issues.apache.org/jira/browse/SPARK-7734 Project: Spark Issue Type: Sub-task Com

[jira] [Comment Edited] (SPARK-7700) Spark 1.3.0 on YARN: Application failed 2 times due to AM Container

2015-05-19 Thread Kaveen Raajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550436#comment-14550436 ] Kaveen Raajan edited comment on SPARK-7700 at 5/19/15 2:13 PM: -

[jira] [Resolved] (SPARK-872) Should revive offer after tasks finish in Mesos fine-grained mode

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-872. - Resolution: Won't Fix > Should revive offer after tasks finish in Mesos fine-grained mode > --

[jira] [Commented] (SPARK-872) Should revive offer after tasks finish in Mesos fine-grained mode

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550473#comment-14550473 ] Iulian Dragos commented on SPARK-872: - I think we should close this due to inactivity.

[jira] [Resolved] (SPARK-2555) Support configuration spark.scheduler.minRegisteredExecutorsRatio in Mesos mode.

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos resolved SPARK-2555. -- Resolution: Invalid As [commented here|https://github.com/apache/spark/pull/1462#issuecomment-7

[jira] [Issue Comment Deleted] (SPARK-4899) Support Mesos features: roles and checkpoints

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-4899: - Comment: was deleted (was: Seems so, at least.) > Support Mesos features: roles and checkpoints >

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550442#comment-14550442 ] Saisai Shao commented on SPARK-7699: Yes, you are right after checking the code of {{a

[jira] [Commented] (SPARK-5488) SPARK_LOCAL_IP not read by mesos scheduler

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550434#comment-14550434 ] Iulian Dragos commented on SPARK-5488: -- I've noticed something fishy as well, however

[jira] [Commented] (SPARK-7700) Spark 1.3.0 on YARN: Application failed 2 times due to AM Container

2015-05-19 Thread Kaveen Raajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550436#comment-14550436 ] Kaveen Raajan commented on SPARK-7700: -- Hi [~srowen], Thanks for the update. Now I ab

[jira] [Created] (SPARK-7732) org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree:

2015-05-19 Thread Prem singh Bist (JIRA)
Prem singh Bist created SPARK-7732: -- Summary: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Key: SPARK-7732 URL: https://issues.apache.org/jira/browse/SPARK-7732 Proj

[jira] [Updated] (SPARK-6679) java.lang.ClassNotFoundException on Mesos fine grained mode and input replication

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iulian Dragos updated SPARK-6679: - Description: Spark Streaming 1.3.0, Mesos 0.21.1 - Only when using fine grained mode and receiver

[jira] [Updated] (SPARK-7730) Complex Teradata queries throwing Analysis Exception when running on spark

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7730: - Component/s: (was: Spark Shell) SQL > Complex Teradata queries throwing Analysis Exce

[jira] [Resolved] (SPARK-7731) Complex Teradata queries throwing Analysis Exception when running on spark

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7731. -- Resolution: Duplicate > Complex Teradata queries throwing Analysis Exception when running on spark > ---

[jira] [Created] (SPARK-7731) Complex Teradata queries throwing Analysis Exception when running on spark

2015-05-19 Thread pankhuri (JIRA)
pankhuri created SPARK-7731: --- Summary: Complex Teradata queries throwing Analysis Exception when running on spark Key: SPARK-7731 URL: https://issues.apache.org/jira/browse/SPARK-7731 Project: Spark

[jira] [Created] (SPARK-7730) Complex Teradata queries throwing Analysis Exception when running on spark

2015-05-19 Thread pankhuri (JIRA)
pankhuri created SPARK-7730: --- Summary: Complex Teradata queries throwing Analysis Exception when running on spark Key: SPARK-7730 URL: https://issues.apache.org/jira/browse/SPARK-7730 Project: Spark

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550401#comment-14550401 ] Sean Owen commented on SPARK-7699: -- Backing up a bit here, after I re-read the code in mo

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550378#comment-14550378 ] Saisai Shao commented on SPARK-7699: Yes, {{maxNeeded}} is the correct # if we have lo

[jira] [Resolved] (SPARK-7702) Is it necessary to limit the precision of a Decimal type?

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7702. -- Resolution: Invalid I think you're right the first time that this is really a question to ask on user@

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550367#comment-14550367 ] Sean Owen commented on SPARK-7699: -- Isn't maxNeeded the correct number at that point then

[jira] [Updated] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Archit Thakur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Archit Thakur updated SPARK-7729: - Attachment: WebUI.png After this patch, UI looks something like this. > Executor which has been k

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550357#comment-14550357 ] Saisai Shao commented on SPARK-7699: I think if we have lots of pending tasks, the act

[jira] [Updated] (SPARK-7702) Is it necessary to limit the precision of a Decimal type?

2015-05-19 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7702: -- Issue Type: Bug (was: Question) > Is it necessary to limit the precision of a Decimal type? > -

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550344#comment-14550344 ] Sean Owen commented on SPARK-7699: -- Does your invocation use enough tasks to need more th

[jira] [Commented] (SPARK-7701) UDT not working

2015-05-19 Thread bogdan baraila (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550342#comment-14550342 ] bogdan baraila commented on SPARK-7701: --- Wrapping it into a Tuple1 worked indded. I

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550339#comment-14550339 ] meiyoula commented on SPARK-7699: - [~srowen]Sorry, I have tested it, run SparkPi with spar

[jira] [Assigned] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7729: --- Assignee: Apache Spark > Executor which has been killed should also be displayed on Executors

[jira] [Commented] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Archit Thakur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550327#comment-14550327 ] Archit Thakur commented on SPARK-7729: -- pull request https://github.com/apache/spark/

[jira] [Commented] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550325#comment-14550325 ] Apache Spark commented on SPARK-7729: - User 'archit279thakur' has created a pull reque

[jira] [Assigned] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7729: --- Assignee: (was: Apache Spark) > Executor which has been killed should also be displayed o

[jira] [Created] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

2015-05-19 Thread Archit Thakur (JIRA)
Archit Thakur created SPARK-7729: Summary: Executor which has been killed should also be displayed on Executors Tab. Key: SPARK-7729 URL: https://issues.apache.org/jira/browse/SPARK-7729 Project: Spar

[jira] [Updated] (SPARK-7728) SQL: cannot add timestamp partition as string

2015-05-19 Thread Kanwaljit Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kanwaljit Singh updated SPARK-7728: --- Description: create table spark_test(i int) partitioned by (p timestamp); alter table spark_te

[jira] [Created] (SPARK-7728) SQL: cannot add timestamp partition as string

2015-05-19 Thread Kanwaljit Singh (JIRA)
Kanwaljit Singh created SPARK-7728: -- Summary: SQL: cannot add timestamp partition as string Key: SPARK-7728 URL: https://issues.apache.org/jira/browse/SPARK-7728 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-4412) Parquet logger cannot be configured

2015-05-19 Thread Yana Kadiyska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yana Kadiyska updated SPARK-4412: - Affects Version/s: 1.3.1 > Parquet logger cannot be configured > -

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550249#comment-14550249 ] Sean Owen commented on SPARK-7699: -- Isn't this the normal state of a Spark app, run with

[jira] [Closed] (SPARK-7709) spark-submit option to quit after submitting in cluster mode

2015-05-19 Thread Shay Rojansky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Rojansky closed SPARK-7709. Resolution: Duplicate Oops, it seems this was already implemented... > spark-submit option to quit

[jira] [Comment Edited] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550223#comment-14550223 ] meiyoula edited comment on SPARK-7699 at 5/19/15 10:58 AM: --- [~sr

[jira] [Comment Edited] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550223#comment-14550223 ] meiyoula edited comment on SPARK-7699 at 5/19/15 10:57 AM: --- [~sr

[jira] [Comment Edited] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550223#comment-14550223 ] meiyoula edited comment on SPARK-7699 at 5/19/15 10:55 AM: --- [~sr

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550223#comment-14550223 ] meiyoula commented on SPARK-7699: - [~srowen] you said "I'm just talking about the case whe

[jira] [Assigned] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7726: --- Assignee: (was: Apache Spark) > Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550193#comment-14550193 ] Apache Spark commented on SPARK-7726: - User 'dragos' has created a pull request for th

[jira] [Assigned] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7726: --- Assignee: Apache Spark > Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher

[jira] [Created] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-19 Thread Santiago M. Mola (JIRA)
Santiago M. Mola created SPARK-7727: --- Summary: Avoid inner classes in RuleExecutor Key: SPARK-7727 URL: https://issues.apache.org/jira/browse/SPARK-7727 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550185#comment-14550185 ] Sean Owen commented on SPARK-7699: -- The parameter has meaning if the system isn't idle, r

[jira] [Comment Edited] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550170#comment-14550170 ] Iulian Dragos edited comment on SPARK-7726 at 5/19/15 10:11 AM:

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550170#comment-14550170 ] Iulian Dragos commented on SPARK-7726: -- The problem is different visibility rules in

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550163#comment-14550163 ] Saisai Shao commented on SPARK-7699: The problem is as you mentioned here: {code} val

[jira] [Commented] (SPARK-7699) Config "spark.dynamicAllocation.initialExecutors" has no effect

2015-05-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550157#comment-14550157 ] Saisai Shao commented on SPARK-7699: I think for the first time called {{updateAndSync

[jira] [Commented] (SPARK-7701) UDT not working

2015-05-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550143#comment-14550143 ] Liang-Chi Hsieh commented on SPARK-7701: I meant wrapping ExamplePoint in Products

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550142#comment-14550142 ] Iulian Dragos commented on SPARK-7726: -- This fix in 2.11.3 looks like the culprit (me

[jira] [Commented] (SPARK-7684) TestHive.reset complains Database does not exist: default

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550132#comment-14550132 ] Cheng Lian commented on SPARK-7684: --- This looks quite similar to this one https://githu

[jira] [Resolved] (SPARK-7723) Fix string interpolation in pipeline examples

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7723. -- Resolution: Fixed Fix Version/s: 1.4.0 Resolved by https://github.com/apache/spark/pull/6258 > F

[jira] [Commented] (SPARK-7092) Update spark scala version to 2.11.6

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550124#comment-14550124 ] Patrick Wendell commented on SPARK-7092: This is reopened because it caused SPARK-

[jira] [Resolved] (SPARK-5777) Completes data source filter types and remove CatalystScan

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5777. --- Resolution: Won't Fix > Completes data source filter types and remove CatalystScan > -

[jira] [Reopened] (SPARK-7092) Update spark scala version to 2.11.6

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-7092: > Update spark scala version to 2.11.6 > > >

[jira] [Commented] (SPARK-5777) Completes data source filter types and remove CatalystScan

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550123#comment-14550123 ] Cheng Lian commented on SPARK-5777: --- I meant, leaving it for experimentation LGTM. > Co

[jira] [Commented] (SPARK-5777) Completes data source filter types and remove CatalystScan

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550118#comment-14550118 ] Cheng Lian commented on SPARK-5777: --- Yeah, leaving it in {{experimental}} LGTM. > Compl

[jira] [Commented] (SPARK-6533) Allow using wildcard and other file pattern in Parquet DataSource

2015-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550114#comment-14550114 ] Cheng Lian commented on SPARK-6533: --- {{parquetFile(...)}} should also work as it delegat

[jira] [Created] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-05-19 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-7726: -- Summary: Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher] Key: SPARK-7726 URL: https://issues.apache.org/jira/browse/SPARK-7726 Project: Spar

[jira] [Resolved] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7670. Resolution: Duplicate Fix Version/s: SPARK-7726 > Failure when building with scala 2.

[jira] [Commented] (SPARK-7670) Failure when building with scala 2.11 (after 1.3.1

2015-05-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550102#comment-14550102 ] Sean Owen commented on SPARK-7670: -- So [~pwendell] tracked this down to a failure in the

[jira] [Commented] (SPARK-7701) UDT not working

2015-05-19 Thread bogdan baraila (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550098#comment-14550098 ] bogdan baraila commented on SPARK-7701: --- I've already made my ExamplePoint a case cl

<    1   2   3   >