[jira] [Created] (SPARK-12516) Properly handle NM failure situation for Spark on Yarn

2015-12-23 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-12516: --- Summary: Properly handle NM failure situation for Spark on Yarn Key: SPARK-12516 URL: https://issues.apache.org/jira/browse/SPARK-12516 Project: Spark Issue Ty

[jira] [Commented] (SPARK-12514) Spark MetricsSystem can fill disks/cause OOMs when using GangliaSink

2015-12-23 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070717#comment-15070717 ] Saisai Shao commented on SPARK-12514: - Since we need to differentiate the metrics bet

[jira] [Commented] (SPARK-12387) JDBC IN operator push down

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070698#comment-15070698 ] Apache Spark commented on SPARK-12387: -- User 'maropu' has created a pull request for

[jira] [Comment Edited] (SPARK-5885) Add VectorAssembler

2015-12-23 Thread Rahul Tanwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070657#comment-15070657 ] Rahul Tanwani edited comment on SPARK-5885 at 12/24/15 7:02 AM:

[jira] [Commented] (SPARK-5885) Add VectorAssembler

2015-12-23 Thread Rahul Tanwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070657#comment-15070657 ] Rahul Tanwani commented on SPARK-5885: -- VectorAssembler does not handle null values c

[jira] [Comment Edited] (SPARK-11054) WARN ReliableDeliverySupervisor: Association with remote system

2015-12-23 Thread Nitin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070612#comment-15070612 ] Nitin edited comment on SPARK-11054 at 12/24/15 5:59 AM: - Facing

[jira] [Commented] (SPARK-11054) WARN ReliableDeliverySupervisor: Association with remote system

2015-12-23 Thread Nitin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070612#comment-15070612 ] Nitin commented on SPARK-11054: --- Facing similar issue when moved to spark 1.5.1 > WARN Re

[jira] [Assigned] (SPARK-12375) VectorIndexer: allow unknown categories

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12375: Assignee: Apache Spark > VectorIndexer: allow unknown categories > ---

[jira] [Assigned] (SPARK-12515) Minor clarification on DataFrameReader.jdbc doc

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12515: Assignee: Apache Spark > Minor clarification on DataFrameReader.jdbc doc > ---

[jira] [Assigned] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12513: Assignee: (was: Apache Spark) > SocketReceiver hang in Netcat example > --

[jira] [Assigned] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12513: Assignee: Apache Spark > SocketReceiver hang in Netcat example > -

[jira] [Created] (SPARK-12515) Minor clarification on DataFrameReader.jdbc doc

2015-12-23 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-12515: Summary: Minor clarification on DataFrameReader.jdbc doc Key: SPARK-12515 URL: https://issues.apache.org/jira/browse/SPARK-12515 Project: Spark Issue Type: B

[jira] [Created] (SPARK-12514) Spark MetricsSystem can fill disks/cause OOMs when using GangliaSink

2015-12-23 Thread Aaron Tokhy (JIRA)
Aaron Tokhy created SPARK-12514: --- Summary: Spark MetricsSystem can fill disks/cause OOMs when using GangliaSink Key: SPARK-12514 URL: https://issues.apache.org/jira/browse/SPARK-12514 Project: Spark

[jira] [Updated] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Shawn Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Guo updated SPARK-12513: -- Description: I add a SocketReceiver test based on the NetworkWordCount. Using pipeline and tail the co

[jira] [Updated] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Shawn Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Guo updated SPARK-12513: -- Description: I add a SocketReceiver test based on the NetworkWordCount. Using pipeline and tail the co

[jira] [Commented] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070550#comment-15070550 ] Apache Spark commented on SPARK-12513: -- User 'guoxu1231' has created a pull request

[jira] [Commented] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Shawn Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070548#comment-15070548 ] Shawn Guo commented on SPARK-12513: --- Close client side socket connection before restart

[jira] [Commented] (SPARK-12502) Script /dev/run-tests fails when IBM Java is used

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070543#comment-15070543 ] Apache Spark commented on SPARK-12502: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-12502) Script /dev/run-tests fails when IBM Java is used

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12502: Assignee: Apache Spark > Script /dev/run-tests fails when IBM Java is used > -

[jira] [Assigned] (SPARK-12502) Script /dev/run-tests fails when IBM Java is used

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12502: Assignee: (was: Apache Spark) > Script /dev/run-tests fails when IBM Java is used > --

[jira] [Created] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Shawn Guo (JIRA)
Shawn Guo created SPARK-12513: - Summary: SocketReceiver hang in Netcat example Key: SPARK-12513 URL: https://issues.apache.org/jira/browse/SPARK-12513 Project: Spark Issue Type: Bug Affects V

[jira] [Updated] (SPARK-12513) SocketReceiver hang in Netcat example

2015-12-23 Thread Shawn Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Guo updated SPARK-12513: -- Component/s: Streaming > SocketReceiver hang in Netcat example > -

[jira] [Commented] (SPARK-12262) describe extended doesn't return table on detail info tabled stored as PARQUET format

2015-12-23 Thread Xiu (Joe) Guo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070503#comment-15070503 ] Xiu (Joe) Guo commented on SPARK-12262: --- The property {code}spark.sql.hive.convertM

[jira] [Commented] (SPARK-12247) Documentation for spark.ml's ALS and collaborative filtering in general

2015-12-23 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070500#comment-15070500 ] Timothy Hunter commented on SPARK-12247: Sorry for the delay. That sounds great!

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antony Mayi updated SPARK-12511: Description: Spark streaming application when configured with checkpointing is filling driver's he

[jira] [Updated] (SPARK-12512) WithColumn does not work on multiple column with special character

2015-12-23 Thread JO EE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JO EE updated SPARK-12512: -- Description: Just for simplicity I am using Scalaide scala-worksheet to show the problem the withColumn could

[jira] [Created] (SPARK-12512) WithColumn does not work on multiple column with special character

2015-12-23 Thread JO EE (JIRA)
JO EE created SPARK-12512: - Summary: WithColumn does not work on multiple column with special character Key: SPARK-12512 URL: https://issues.apache.org/jira/browse/SPARK-12512 Project: Spark Issue T

[jira] [Updated] (SPARK-12411) Reconsider executor heartbeats rpc timeout

2015-12-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12411: -- Fix Version/s: 1.6.1 > Reconsider executor heartbeats rpc timeout > ---

[jira] [Updated] (SPARK-12411) Reconsider executor heartbeats rpc timeout

2015-12-23 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12411: -- Target Version/s: 1.6.1, 2.0.0 (was: 2.0.0) > Reconsider executor heartbeats rpc timeout > ---

[jira] [Commented] (SPARK-12356) Implement unhandledFilter interface for ORC

2015-12-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070346#comment-15070346 ] Yin Huai commented on SPARK-12356: -- oh, yes. Let's close it. ORC does not apply filter t

[jira] [Resolved] (SPARK-12356) Implement unhandledFilter interface for ORC

2015-12-23 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-12356. -- Resolution: Invalid > Implement unhandledFilter interface for ORC > ---

[jira] [Commented] (SPARK-12356) Implement unhandledFilter interface for ORC

2015-12-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070344#comment-15070344 ] Hyukjin Kwon commented on SPARK-12356: -- [~yhuai] I think this should not be fixed as

[jira] [Resolved] (SPARK-12499) make_distribution should not override MAVEN_OPTS

2015-12-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12499. Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by pull reque

[jira] [Updated] (SPARK-12499) make_distribution should not override MAVEN_OPTS

2015-12-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12499: --- Assignee: Adrian Bridgett > make_distribution should not override MAVEN_OPTS > --

[jira] [Assigned] (SPARK-10359) Enumerate Spark's dependencies in a file and diff against it for new pull requests

2015-12-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-10359: -- Assignee: Josh Rosen (was: Patrick Wendell) > Enumerate Spark's dependencies in a file and di

[jira] [Commented] (SPARK-10359) Enumerate Spark's dependencies in a file and diff against it for new pull requests

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070325#comment-15070325 ] Apache Spark commented on SPARK-10359: -- User 'JoshRosen' has created a pull request

[jira] [Commented] (SPARK-12459) Add ExpressionDescription to string functions

2015-12-23 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070288#comment-15070288 ] Huaxin Gao commented on SPARK-12459: https://github.com/apache/spark/pull/10460 > Ad

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antony Mayi updated SPARK-12511: Description: Spark streaming application when configured with checkpointing is filling driver's he

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antony Mayi updated SPARK-12511: Description: Spark streaming application when configured with checkpointing is filling driver's he

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antony Mayi updated SPARK-12511: Attachment: bug.py > streaming driver with checkpointing unable to finalize leading to OOM > --

[jira] [Commented] (SPARK-7995) Remove AkkaRpcEnv and remove Akka from the dependencies of Core

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070259#comment-15070259 ] Apache Spark commented on SPARK-7995: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-6280) Remove Akka systemName from Spark

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070260#comment-15070260 ] Apache Spark commented on SPARK-6280: - User 'zsxwing' has created a pull request for t

[jira] [Assigned] (SPARK-6280) Remove Akka systemName from Spark

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6280: --- Assignee: Apache Spark > Remove Akka systemName from Spark >

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antony Mayi updated SPARK-12511: Description: Spark streaming application when configured with checkpointing is filling driver's he

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antony Mayi updated SPARK-12511: Attachment: finalizer-spark_assembly.png finalizer-pending.png final

[jira] [Assigned] (SPARK-7995) Remove AkkaRpcEnv and remove Akka from the dependencies of Core

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7995: --- Assignee: Apache Spark > Remove AkkaRpcEnv and remove Akka from the dependencies of Core > --

[jira] [Created] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-23 Thread Antony Mayi (JIRA)
Antony Mayi created SPARK-12511: --- Summary: streaming driver with checkpointing unable to finalize leading to OOM Key: SPARK-12511 URL: https://issues.apache.org/jira/browse/SPARK-12511 Project: Spark

[jira] [Updated] (SPARK-7997) Remove the developer api SparkEnv.actorSystem and AkkaUtils

2015-12-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-7997: Target Version/s: 2.0.0 (was: ) > Remove the developer api SparkEnv.actorSystem and AkkaUtils > ---

[jira] [Resolved] (SPARK-7797) Remove "actorSystem" from SparkEnv

2015-12-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-7797. - Resolution: Duplicate Target Version/s: (was: ) > Remove "actorSystem" from SparkEnv >

[jira] [Updated] (SPARK-7997) Remove the developer api SparkEnv.actorSystem and AkkaUtils

2015-12-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-7997: Summary: Remove the developer api SparkEnv.actorSystem and AkkaUtils (was: Remove the developer api

[jira] [Updated] (SPARK-7995) Remove AkkaRpcEnv and remove Akka from the dependencies of Core

2015-12-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-7995: Summary: Remove AkkaRpcEnv and remove Akka from the dependencies of Core (was: Move AkkaRpcEnv to a

[jira] [Resolved] (SPARK-12500) Fix Tachyon deprecations; pull Tachyon dependency into one class

2015-12-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12500. - Resolution: Fixed Assignee: Sean Owen (was: Apache Spark) Fix Version/s: 2.0.0 >

[jira] [Assigned] (SPARK-12500) Fix Tachyon deprecations; pull Tachyon dependency into one class

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12500: Assignee: Apache Spark (was: Sean Owen) > Fix Tachyon deprecations; pull Tachyon dependen

[jira] [Commented] (SPARK-12510) Refactor ActorReceiver to support Java

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070191#comment-15070191 ] Apache Spark commented on SPARK-12510: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-12509) Fix error messages for DataFrame correlation and covariance

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12509: Assignee: Apache Spark > Fix error messages for DataFrame correlation and covariance > ---

[jira] [Commented] (SPARK-12509) Fix error messages for DataFrame correlation and covariance

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070190#comment-15070190 ] Apache Spark commented on SPARK-12509: -- User 'NarineK' has created a pull request fo

[jira] [Assigned] (SPARK-12509) Fix error messages for DataFrame correlation and covariance

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12509: Assignee: (was: Apache Spark) > Fix error messages for DataFrame correlation and covar

[jira] [Assigned] (SPARK-12510) Refactor ActorReceiver to support Java

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12510: Assignee: Apache Spark > Refactor ActorReceiver to support Java >

[jira] [Updated] (SPARK-12509) Fix error messages for DataFrame correlation and covariance

2015-12-23 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-12509: -- Description: Currently, when we call corr or cov on dataframe with invalid input we see

[jira] [Updated] (SPARK-12510) Refactor ActorReceiver to support Java

2015-12-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12510: - Issue Type: Improvement (was: Documentation) > Refactor ActorReceiver to support Java >

[jira] [Created] (SPARK-12510) Refactor ActorReceiver to support Java

2015-12-23 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-12510: Summary: Refactor ActorReceiver to support Java Key: SPARK-12510 URL: https://issues.apache.org/jira/browse/SPARK-12510 Project: Spark Issue Type: Documentat

[jira] [Updated] (SPARK-12509) Fix error messages for DataFrame correlation and covariance

2015-12-23 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-12509: -- Description: Currently, when we call corr or cov on dataframe with invalid input we see

[jira] [Created] (SPARK-12509) Fix error messages for DataFrame correlation and covariance

2015-12-23 Thread Narine Kokhlikyan (JIRA)
Narine Kokhlikyan created SPARK-12509: - Summary: Fix error messages for DataFrame correlation and covariance Key: SPARK-12509 URL: https://issues.apache.org/jira/browse/SPARK-12509 Project: Spark

[jira] [Comment Edited] (SPARK-12420) Have a built-in CSV data source implementation

2015-12-23 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070147#comment-15070147 ] Hossein Falaki edited comment on SPARK-12420 at 12/23/15 8:28 PM: -

[jira] [Updated] (SPARK-12420) Have a built-in CSV data source implementation

2015-12-23 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-12420: --- Attachment: Built-in CSV datasource in Spark.pdf Design doc for submitting spark-csv as a bui

[jira] [Assigned] (SPARK-12505) Pushdown a Limit on top of an Outer-Join

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12505: Assignee: Apache Spark > Pushdown a Limit on top of an Outer-Join > --

[jira] [Commented] (SPARK-12508) Fix multiple bugs in pr_public_classes.sh script

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070143#comment-15070143 ] Apache Spark commented on SPARK-12508: -- User 'JoshRosen' has created a pull request

[jira] [Commented] (SPARK-12453) Spark Streaming Kinesis Example broken due to wrong AWS Java SDK version

2015-12-23 Thread Brian London (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070142#comment-15070142 ] Brian London commented on SPARK-12453: -- I don't know if removing the explicit depend

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2015-12-23 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070135#comment-15070135 ] Yan commented on SPARK-12449: - Conceivably, if only logical plan is used, the Spark SQL exec

[jira] [Assigned] (SPARK-12508) Fix multiple bugs in pr_public_classes.sh script

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12508: Assignee: Apache Spark (was: Josh Rosen) > Fix multiple bugs in pr_public_classes.sh scri

[jira] [Commented] (SPARK-12501) spark fails to build with mesos-0.26.0 (secret)

2015-12-23 Thread Adrian Bridgett (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070117#comment-15070117 ] Adrian Bridgett commented on SPARK-12501: - Indeed - I'm not a Scala (or Java) dev

[jira] [Commented] (SPARK-12505) Pushdown a Limit on top of an Outer-Join

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070115#comment-15070115 ] Apache Spark commented on SPARK-12505: -- User 'gatorsmile' has created a pull request

[jira] [Updated] (SPARK-12508) Fix multiple bugs in pr_public_classes.sh script

2015-12-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12508: --- Priority: Minor (was: Major) > Fix multiple bugs in pr_public_classes.sh script > --

[jira] [Updated] (SPARK-12508) Fix multiple bugs in pr_public_classes.sh script

2015-12-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12508: --- Priority: Trivial (was: Minor) > Fix multiple bugs in pr_public_classes.sh script >

[jira] [Created] (SPARK-12508) Fix multiple bugs in pr_public_classes.sh script

2015-12-23 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-12508: -- Summary: Fix multiple bugs in pr_public_classes.sh script Key: SPARK-12508 URL: https://issues.apache.org/jira/browse/SPARK-12508 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2015-12-23 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070100#comment-15070100 ] Santiago M. Mola commented on SPARK-12449: -- Well, at least with the implementati

[jira] [Assigned] (SPARK-12507) Update Streaming configurations for 1.6

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12507: Assignee: Apache Spark > Update Streaming configurations for 1.6 > ---

[jira] [Commented] (SPARK-12507) Update Streaming configurations for 1.6

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070097#comment-15070097 ] Apache Spark commented on SPARK-12507: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-12499) make_distribution should not override MAVEN_OPTS

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12499: Assignee: (was: Apache Spark) > make_distribution should not override MAVEN_OPTS > ---

[jira] [Assigned] (SPARK-12499) make_distribution should not override MAVEN_OPTS

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12499: Assignee: Apache Spark > make_distribution should not override MAVEN_OPTS > --

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2015-12-23 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070088#comment-15070088 ] Yan commented on SPARK-12449: - To push down map-side (partial) ops, Physical Plan would need

[jira] [Commented] (SPARK-12426) Docker JDBC integration tests are failing again

2015-12-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070064#comment-15070064 ] Josh Rosen commented on SPARK-12426: This is probably an environment / configuration

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2015-12-23 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070062#comment-15070062 ] Santiago M. Mola commented on SPARK-12449: -- The physical plan would not be consu

[jira] [Commented] (SPARK-12459) Add ExpressionDescription to string functions

2015-12-23 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070061#comment-15070061 ] Huaxin Gao commented on SPARK-12459: Thanks for the suggestion. I will change and ha

[jira] [Assigned] (SPARK-12507) Update Streaming configurations for 1.6

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12507: Assignee: Apache Spark > Update Streaming configurations for 1.6 > ---

[jira] [Assigned] (SPARK-12507) Update Streaming configurations for 1.6

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12507: Assignee: (was: Apache Spark) > Update Streaming configurations for 1.6 >

[jira] [Commented] (SPARK-12506) Push down WHERE clause arithmetic operator to JDBC layer

2015-12-23 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070058#comment-15070058 ] Huaxin Gao commented on SPARK-12506: I am working on this and will have a PR soon.

[jira] [Created] (SPARK-12507) Update Streaming configurations for 1.6

2015-12-23 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-12507: Summary: Update Streaming configurations for 1.6 Key: SPARK-12507 URL: https://issues.apache.org/jira/browse/SPARK-12507 Project: Spark Issue Type: Documenta

[jira] [Created] (SPARK-12506) Push down WHERE clause arithmetic operator to JDBC layer

2015-12-23 Thread Huaxin Gao (JIRA)
Huaxin Gao created SPARK-12506: -- Summary: Push down WHERE clause arithmetic operator to JDBC layer Key: SPARK-12506 URL: https://issues.apache.org/jira/browse/SPARK-12506 Project: Spark Issue Ty

[jira] [Updated] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2015-12-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12504: Description: Currently JDBC data source credentials are not masked in the explain output. This can

[jira] [Commented] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2015-12-23 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070054#comment-15070054 ] Suresh Thalamati commented on SPARK-12504: -- Pull Request: https://github.com/apa

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2015-12-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070027#comment-15070027 ] Reynold Xin commented on SPARK-12449: - This is great, but our internal logical/physic

[jira] [Updated] (SPARK-12505) Pushdown a Limit on top of an Outer-Join

2015-12-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12505: Summary: Pushdown a Limit on top of an Outer-Join (was: Pushdown a Limit on top of a Outer-Join) > Pushdo

[jira] [Created] (SPARK-12505) Pushdown a Limit on top of a Outer-Join

2015-12-23 Thread Xiao Li (JIRA)
Xiao Li created SPARK-12505: --- Summary: Pushdown a Limit on top of a Outer-Join Key: SPARK-12505 URL: https://issues.apache.org/jira/browse/SPARK-12505 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-12505) Pushdown a Limit on top of an Outer-Join

2015-12-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070012#comment-15070012 ] Xiao Li commented on SPARK-12505: - I am working on it. Thanks! > Pushdown a Limit on top

[jira] [Updated] (SPARK-12503) Pushdown a Limit on top of a Union

2015-12-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-12503: Description: "Rule that applies to a Limit on top of a Union. The original Limit won't go away after apply

[jira] [Commented] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2015-12-23 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070002#comment-15070002 ] Suresh Thalamati commented on SPARK-12504: -- I am testing the fix for this issue

[jira] [Created] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2015-12-23 Thread Suresh Thalamati (JIRA)
Suresh Thalamati created SPARK-12504: Summary: JDBC data source credentials are not masked in the data frame explain output. Key: SPARK-12504 URL: https://issues.apache.org/jira/browse/SPARK-12504

[jira] [Commented] (SPARK-12502) Script /dev/run-tests fails when IBM Java is used

2015-12-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15069991#comment-15069991 ] Kazuaki Ishizaki commented on SPARK-12502: -- Yes, I will do it. I will change a f

[jira] [Assigned] (SPARK-12503) Pushdown a Limit on top of a Union

2015-12-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12503: Assignee: Apache Spark > Pushdown a Limit on top of a Union >

  1   2   >