[jira] [Resolved] (SPARK-13408) Exception in resultHandler will shutdown SparkContext

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13408. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11280

[jira] [Comment Edited] (SPARK-13409) Log the stacktrace when stopping a SparkContext

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155441#comment-15155441 ] Reynold Xin edited comment on SPARK-13409 at 2/20/16 6:58 AM: -- OK that makes

[jira] [Resolved] (SPARK-13304) Broadcast join with two ints could be very slow

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13304. Resolution: Fixed Assignee: Davies Liu Fix Version/s: 2.0.0 Fixed by

[jira] [Commented] (SPARK-13409) Log the stacktrace when stopping a SparkContext

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155441#comment-15155441 ] Reynold Xin commented on SPARK-13409: - OK that makes sense. It wasn't clear to me what you meant by

[jira] [Commented] (SPARK-13409) Log the stacktrace when stopping a SparkContext

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155439#comment-15155439 ] Davies Liu commented on SPARK-13409: [~rxin] I think we should remember the stacktrace, put that in

[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155438#comment-15155438 ] Davies Liu commented on SPARK-13213: It depends, I'm open to any reasonable solution. >

[jira] [Assigned] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12567: Assignee: Kai Jiang (was: Apache Spark) > Add aes_encrypt and aes_decrypt UDFs >

[jira] [Reopened] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-12567: - > Add aes_encrypt and aes_decrypt UDFs > > >

[jira] [Assigned] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12567: Assignee: Apache Spark (was: Kai Jiang) > Add aes_encrypt and aes_decrypt UDFs >

[jira] [Updated] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12567: Summary: Add aes_encrypt and aes_decrypt UDFs (was: Add aes_{encrypt,decrypt} UDFs) > Add

[jira] [Resolved] (SPARK-12567) Add aes_{encrypt,decrypt} UDFs

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12567. Resolution: Fixed Assignee: Kai Jiang Fix Version/s: 2.0.0 > Add

[jira] [Resolved] (SPARK-12594) Outer Join Elimination by Filter Condition

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12594. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10567

[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155418#comment-15155418 ] Reynold Xin commented on SPARK-13213: - [~davies] what is this ticket about? Is it about making

[jira] [Commented] (SPARK-13382) Update PySpark testing notes

2016-02-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155415#comment-15155415 ] holdenk commented on SPARK-13382: - Note: while I've got a PR we should also update the wiki and I don't

[jira] [Commented] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155390#comment-15155390 ] Xiao Li commented on SPARK-12720: - Expression {{grouping_id()}} is needed for resolving this JIRA. Thus,

[jira] [Commented] (SPARK-13391) Use Apache Arrow as In-memory columnar store implementation

2016-02-19 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155268#comment-15155268 ] Wes McKinney commented on SPARK-13391: -- Indeed, one of the major motivations of Arrow (for Python

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-02-19 Thread Varadharajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155263#comment-15155263 ] Varadharajan commented on SPARK-13393: -- Hi, I just tried the same scenario with prebuilt 1.6.0 and

[jira] [Updated] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-02-19 Thread Varadharajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varadharajan updated SPARK-13393: - Description: Consider the below snippet: {code:title=test.scala|borderStyle=solid} case class

[jira] [Commented] (SPARK-10001) Allow Ctrl-C in spark-shell to kill running job

2016-02-19 Thread Jon Maurer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155232#comment-15155232 ] Jon Maurer commented on SPARK-10001: Thank you for your feedback and consideration. I opened a new

[jira] [Created] (SPARK-13412) Spark Shell Ctrl-C behaviour suggestion

2016-02-19 Thread Jon Maurer (JIRA)
Jon Maurer created SPARK-13412: -- Summary: Spark Shell Ctrl-C behaviour suggestion Key: SPARK-13412 URL: https://issues.apache.org/jira/browse/SPARK-13412 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155229#comment-15155229 ] Xiao Li commented on SPARK-1: - Thanks! > DataFrame filter + randn + unionAll has bad interaction >

[jira] [Commented] (SPARK-13048) EMLDAOptimizer deletes dependent checkpoint of DistributedLDAModel

2016-02-19 Thread Jeff Stein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155212#comment-15155212 ] Jeff Stein commented on SPARK-13048: As an aside, the code in the clustering namespace violates the

[jira] [Resolved] (SPARK-13407) TaskMetrics.fromAccumulatorUpdates can crash when trying to access garbage-collected accumulators

2016-02-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13407. Resolution: Fixed Fix Version/s: 2.0.0 Fixed by my patch. >

[jira] [Updated] (SPARK-13329) Considering output for statistics of logical plan

2016-02-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13329: -- Summary: Considering output for statistics of logical plan (was: Considering output for statistics of

[jira] [Updated] (SPARK-13409) Log the stacktrace when stopping a SparkContext

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13409: Description: Somethings we saw a stopped SparkContext, then have no idea it's stopped by what, we

[jira] [Updated] (SPARK-13409) Log the stacktrace when stopping a SparkContext

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13409: Summary: Log the stacktrace when stopping a SparkContext (was: Remember the stacktrace when stop

[jira] [Resolved] (SPARK-13091) Rewrite/Propagate constraints for Aliases

2016-02-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-13091. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11144

[jira] [Commented] (SPARK-13333) DataFrame filter + randn + unionAll has bad interaction

2016-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155047#comment-15155047 ] Reynold Xin commented on SPARK-1: - OK I think this is going to be really difficult to fix right

[jira] [Resolved] (SPARK-13261) Expose maxCharactersPerColumn as a user configurable option

2016-02-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-13261. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11147

[jira] [Updated] (SPARK-13411) change in null aggregation behavior from spark 1.5.2 and 1.6.0

2016-02-19 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barry Becker updated SPARK-13411: - Description: I don't know if the behavior in 1.5.3 or 1.6.0 is correct, but its definitely

[jira] [Resolved] (SPARK-12966) Postgres JDBC ArrayType(DecimalType) 'Unable to find server array type'

2016-02-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-12966. -- Resolution: Fixed Fix Version/s: 2.0.0 > Postgres JDBC ArrayType(DecimalType)

[jira] [Updated] (SPARK-13411) change in null aggregation behavior from spark 1.5.2 and 1.6.0

2016-02-19 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barry Becker updated SPARK-13411: - Description: I don't know if the behavior in 1.5.3 or 1.6.0 is correct, but its definitely

[jira] [Created] (SPARK-13411) change in null aggregation behavior from spark 1.5.2 and 1.6.0

2016-02-19 Thread Barry Becker (JIRA)
Barry Becker created SPARK-13411: Summary: change in null aggregation behavior from spark 1.5.2 and 1.6.0 Key: SPARK-13411 URL: https://issues.apache.org/jira/browse/SPARK-13411 Project: Spark

[jira] [Updated] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-13410: Description: Unioning two DataFrames that contain UDTs fails with {quote}

[jira] [Updated] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-13410: Description: Unioning two DataFrames that contain UDTs fails with {quote}

[jira] [Updated] (SPARK-13410) unionAll AnalysisException with DataFrames containing UDT columns.

2016-02-19 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-13410: Summary: unionAll AnalysisException with DataFrames containing UDT columns. (was:

[jira] [Commented] (SPARK-13408) Exception in resultHandler will shutdown SparkContext

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154962#comment-15154962 ] Apache Spark commented on SPARK-13408: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13408) Exception in resultHandler will shutdown SparkContext

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13408: Assignee: Apache Spark (was: Shixiong Zhu) > Exception in resultHandler will shutdown

[jira] [Commented] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154935#comment-15154935 ] Apache Spark commented on SPARK-13410: -- User 'damnMeddlingKid' has created a pull request for this

[jira] [Assigned] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13410: Assignee: Apache Spark > unionAll throws error with DataFrames containing UDT columns. >

[jira] [Updated] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-13410: Description: Unioning two DataFrames that contain UDTs fails with {quote}

[jira] [Assigned] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13410: Assignee: (was: Apache Spark) > unionAll throws error with DataFrames containing UDT

[jira] [Updated] (SPARK-12864) Fetch failure from AM restart

2016-02-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12864: -- Summary: Fetch failure from AM restart (was: initialize executorIdCounter after ApplicationMaster

[jira] [Created] (SPARK-13410) unionAll throws error with DataFrames containing UDT columns.

2016-02-19 Thread Franklyn Dsouza (JIRA)
Franklyn Dsouza created SPARK-13410: --- Summary: unionAll throws error with DataFrames containing UDT columns. Key: SPARK-13410 URL: https://issues.apache.org/jira/browse/SPARK-13410 Project: Spark

[jira] [Commented] (SPARK-13382) Update PySpark testing notes

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154926#comment-15154926 ] Apache Spark commented on SPARK-13382: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13382) Update PySpark testing notes

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13382: Assignee: Apache Spark > Update PySpark testing notes > > >

[jira] [Assigned] (SPARK-13382) Update PySpark testing notes

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13382: Assignee: (was: Apache Spark) > Update PySpark testing notes >

[jira] [Commented] (SPARK-10001) Allow Ctrl-C in spark-shell to kill running job

2016-02-19 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154885#comment-15154885 ] Ryan Blue commented on SPARK-10001: --- bq. I . . . am uneasy about adopting unusual semantics for a

[jira] [Updated] (SPARK-13409) Remember the stacktrace when stop a SparkContext

2016-02-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13409: --- Component/s: Spark Core > Remember the stacktrace when stop a SparkContext >

[jira] [Created] (SPARK-13409) Remember the stacktrace when stop a SparkContext

2016-02-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13409: -- Summary: Remember the stacktrace when stop a SparkContext Key: SPARK-13409 URL: https://issues.apache.org/jira/browse/SPARK-13409 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-13408) Exception in resultHandler will shutdown SparkContext

2016-02-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13408: -- Summary: Exception in resultHandler will shutdown SparkContext Key: SPARK-13408 URL: https://issues.apache.org/jira/browse/SPARK-13408 Project: Spark Issue

[jira] [Assigned] (SPARK-13387) Add support for SPARK_DAEMON_JAVA_OPTS with MesosClusterDispatcher.

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13387: Assignee: Apache Spark > Add support for SPARK_DAEMON_JAVA_OPTS with

[jira] [Assigned] (SPARK-13387) Add support for SPARK_DAEMON_JAVA_OPTS with MesosClusterDispatcher.

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13387: Assignee: (was: Apache Spark) > Add support for SPARK_DAEMON_JAVA_OPTS with

[jira] [Commented] (SPARK-13387) Add support for SPARK_DAEMON_JAVA_OPTS with MesosClusterDispatcher.

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154842#comment-15154842 ] Apache Spark commented on SPARK-13387: -- User 'tnachen' has created a pull request for this issue:

[jira] [Commented] (SPARK-13407) TaskMetrics.fromAccumulatorUpdates can crash when trying to access garbage-collected accumulators

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154826#comment-15154826 ] Apache Spark commented on SPARK-13407: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-13407) TaskMetrics.fromAccumulatorUpdates can crash when trying to access garbage-collected accumulators

2016-02-19 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-13407: -- Summary: TaskMetrics.fromAccumulatorUpdates can crash when trying to access garbage-collected accumulators Key: SPARK-13407 URL: https://issues.apache.org/jira/browse/SPARK-13407

[jira] [Created] (SPARK-13406) NPE in LazilyGeneratedOrdering

2016-02-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13406: -- Summary: NPE in LazilyGeneratedOrdering Key: SPARK-13406 URL: https://issues.apache.org/jira/browse/SPARK-13406 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-13384) Keep attribute qualifiers after dedup in Analyzer

2016-02-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13384: - Assignee: Liang-Chi Hsieh > Keep attribute qualifiers after dedup in Analyzer >

[jira] [Resolved] (SPARK-13384) Keep attribute qualifiers after dedup in Analyzer

2016-02-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-13384. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11261

[jira] [Comment Edited] (SPARK-13046) Partitioning looks broken in 1.6

2016-02-19 Thread Julien Baley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152718#comment-15152718 ] Julien Baley edited comment on SPARK-13046 at 2/19/16 7:56 PM: --- Sorry it

[jira] [Commented] (SPARK-13341) Casting Unix timestamp to SQL timestamp fails

2016-02-19 Thread Srinivasa Reddy Vundela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154694#comment-15154694 ] Srinivasa Reddy Vundela commented on SPARK-13341: - I guess the following commit is the

[jira] [Updated] (SPARK-13375) PySpark API Utils missing item: kFold

2016-02-19 Thread Bruno Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Wu updated SPARK-13375: - Affects Version/s: (was: 1.6.0) 1.5.0 > PySpark API Utils missing item: kFold

[jira] [Updated] (SPARK-13405) Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event ordering

2016-02-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-13405: - Labels: flaky-test (was: ) > Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event

[jira] [Assigned] (SPARK-13405) Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event ordering

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13405: Assignee: Apache Spark (was: Shixiong Zhu) > Flaky test:

[jira] [Commented] (SPARK-13405) Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event ordering

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154580#comment-15154580 ] Apache Spark commented on SPARK-13405: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13405) Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event ordering

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13405: Assignee: Shixiong Zhu (was: Apache Spark) > Flaky test:

[jira] [Updated] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2016-02-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1405: - Description: Latent Dirichlet Allocation (a.k.a. LDA) is a topic model which extracts topics

[jira] [Created] (SPARK-13405) Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event ordering

2016-02-19 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-13405: Summary: Flaky test: o.a.s.sql.util.ContinuousQueryListenerSuite.event ordering Key: SPARK-13405 URL: https://issues.apache.org/jira/browse/SPARK-13405 Project:

[jira] [Assigned] (SPARK-13304) Broadcast join with two ints could be very slow

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13304: Assignee: Apache Spark > Broadcast join with two ints could be very slow >

[jira] [Commented] (SPARK-13304) Broadcast join with two ints could be very slow

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154570#comment-15154570 ] Apache Spark commented on SPARK-13304: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13304) Broadcast join with two ints could be very slow

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13304: Assignee: (was: Apache Spark) > Broadcast join with two ints could be very slow >

[jira] [Assigned] (SPARK-13404) Create the variables for input when it's used

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13404: Assignee: Apache Spark (was: Davies Liu) > Create the variables for input when it's used

[jira] [Commented] (SPARK-13404) Create the variables for input when it's used

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154559#comment-15154559 ] Apache Spark commented on SPARK-13404: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13404) Create the variables for input when it's used

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13404: Assignee: Davies Liu (was: Apache Spark) > Create the variables for input when it's used

[jira] [Comment Edited] (SPARK-13349) adding a split and union to a streaming application cause big performance hit

2016-02-19 Thread krishna ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154558#comment-15154558 ] krishna ramachandran edited comment on SPARK-13349 at 2/19/16 6:01 PM:

[jira] [Reopened] (SPARK-13349) adding a split and union to a streaming application cause big performance hit

2016-02-19 Thread krishna ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] krishna ramachandran reopened SPARK-13349: -- enabling "cache" for a DStream causes the app to run out of memory > adding a

[jira] [Commented] (SPARK-13349) adding a split and union to a streaming application cause big performance hit

2016-02-19 Thread krishna ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154555#comment-15154555 ] krishna ramachandran commented on SPARK-13349: -- Hi Sean I posted to user@ 2 problems 1) not

[jira] [Created] (SPARK-13404) Create the variables for input when it's used

2016-02-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13404: -- Summary: Create the variables for input when it's used Key: SPARK-13404 URL: https://issues.apache.org/jira/browse/SPARK-13404 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-13373) Generate code for sort merge join

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13373: Assignee: Apache Spark (was: Davies Liu) > Generate code for sort merge join >

[jira] [Commented] (SPARK-13373) Generate code for sort merge join

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154549#comment-15154549 ] Apache Spark commented on SPARK-13373: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13373) Generate code for sort merge join

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13373: Assignee: Davies Liu (was: Apache Spark) > Generate code for sort merge join >

[jira] [Assigned] (SPARK-13403) HiveConf used for SparkSQL is not based on the Hadoop configuration

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13403: Assignee: Apache Spark > HiveConf used for SparkSQL is not based on the Hadoop

[jira] [Assigned] (SPARK-13403) HiveConf used for SparkSQL is not based on the Hadoop configuration

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13403: Assignee: (was: Apache Spark) > HiveConf used for SparkSQL is not based on the Hadoop

[jira] [Commented] (SPARK-13403) HiveConf used for SparkSQL is not based on the Hadoop configuration

2016-02-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154533#comment-15154533 ] Apache Spark commented on SPARK-13403: -- User 'rdblue' has created a pull request for this issue:

[jira] [Created] (SPARK-13403) HiveConf used for SparkSQL is not based on the Hadoop configuration

2016-02-19 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-13403: - Summary: HiveConf used for SparkSQL is not based on the Hadoop configuration Key: SPARK-13403 URL: https://issues.apache.org/jira/browse/SPARK-13403 Project: Spark

[jira] [Closed] (SPARK-13402) List Spark R dependencies

2016-02-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk closed SPARK-13402. --- Resolution: Not A Problem > List Spark R dependencies > - > > Key:

[jira] [Commented] (SPARK-13402) List Spark R dependencies

2016-02-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154507#comment-15154507 ] holdenk commented on SPARK-13402: - Oh let me close this, I was looking in the wrong place (inside of the

[jira] [Commented] (SPARK-13402) List Spark R dependencies

2016-02-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154505#comment-15154505 ] Sean Owen commented on SPARK-13402: --- https://github.com/apache/spark/blob/master/docs/README.md says

[jira] [Created] (SPARK-13402) List Spark R dependencies

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13402: --- Summary: List Spark R dependencies Key: SPARK-13402 URL: https://issues.apache.org/jira/browse/SPARK-13402 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-13400) Stop using deprecated Octal escape literals

2016-02-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154483#comment-15154483 ] holdenk commented on SPARK-13400: - oh yah for sure, I was going to try running some test generation tools

[jira] [Commented] (SPARK-13400) Stop using deprecated Octal escape literals

2016-02-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154479#comment-15154479 ] Sean Owen commented on SPARK-13400: --- Yeah all these are good. We need to zap virtually all the warnings

[jira] [Created] (SPARK-13401) Fix SQL test warnings

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13401: --- Summary: Fix SQL test warnings Key: SPARK-13401 URL: https://issues.apache.org/jira/browse/SPARK-13401 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-13400) Stop using deprecated Octal escape literals

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13400: --- Summary: Stop using deprecated Octal escape literals Key: SPARK-13400 URL: https://issues.apache.org/jira/browse/SPARK-13400 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13399) Investigate type erasure warnings in CheckpointSuite

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13399: --- Summary: Investigate type erasure warnings in CheckpointSuite Key: SPARK-13399 URL: https://issues.apache.org/jira/browse/SPARK-13399 Project: Spark Issue Type:

[jira] [Created] (SPARK-13398) Move away from deprecated ThreadPoolTaskSupport

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13398: --- Summary: Move away from deprecated ThreadPoolTaskSupport Key: SPARK-13398 URL: https://issues.apache.org/jira/browse/SPARK-13398 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13397) Cleanup transient annotations which aren't being applied

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13397: --- Summary: Cleanup transient annotations which aren't being applied Key: SPARK-13397 URL: https://issues.apache.org/jira/browse/SPARK-13397 Project: Spark Issue Type:

[jira] [Created] (SPARK-13396) Stop using our internal deprecated .metrics on ExceptionFailure instead use accumUpdates

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13396: --- Summary: Stop using our internal deprecated .metrics on ExceptionFailure instead use accumUpdates Key: SPARK-13396 URL: https://issues.apache.org/jira/browse/SPARK-13396

[jira] [Created] (SPARK-13395) Silence or skip unsafe deprecation warnings

2016-02-19 Thread holdenk (JIRA)
holdenk created SPARK-13395: --- Summary: Silence or skip unsafe deprecation warnings Key: SPARK-13395 URL: https://issues.apache.org/jira/browse/SPARK-13395 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-13395) Silence or skip unsafe deprecation warnings

2016-02-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13395: Component/s: (was: Streaming) (was: Examples) > Silence or skip unsafe

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-02-19 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154393#comment-15154393 ] Herman van Hovell commented on SPARK-13393: --- I think you are encountering the following bug:

  1   2   >