[jira] [Updated] (SPARK-10909) Spark sql jdbc fails for Oracle NUMBER type columns

[ https://issues.apache.org/jira/browse/SPARK-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10909: Labels: (was: jdbc newbie sql) > Spark sql jdbc fails for Oracle NUMBER type columns > --

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Affects Version/s: (was: 1.0.0) > Introduce common memory management interface for execution and st

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Priority: Major (was: Critical) > Introduce common memory management interface for execution and stora

[jira] [Created] (SPARK-10984) Simplify *MemoryManager class structure

Andrew Or created SPARK-10984: - Summary: Simplify *MemoryManager class structure Key: SPARK-10984 URL: https://issues.apache.org/jira/browse/SPARK-10984 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-10983) Implement unified memory manager

Andrew Or created SPARK-10983: - Summary: Implement unified memory manager Key: SPARK-10983 URL: https://issues.apache.org/jira/browse/SPARK-10983 Project: Spark Issue Type: Sub-task Com

[jira] [Resolved] (SPARK-10856) SQL Server dialect needs to map java.sql.Timestamp to DATETIME instead of TIMESTAMP

[ https://issues.apache.org/jira/browse/SPARK-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10856. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 1.6.0 > SQL Server dia

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Description: The first step towards implementing a solution for SPARK-1 is to refactor the existin

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Description: The first step towards implementing a solution for SPARK-1 is to refactor the existin

[jira] [Updated] (SPARK-10000) Consolidate cache memory management and execution memory management

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Attachment: unified-memory-management-spark-1.pdf > Consolidate cache memory management and executi

[jira] [Updated] (SPARK-10000) Consolidate cache memory management and execution memory management

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Description: Memory management in Spark is currently broken down into two disjoint regions: one for ex

[jira] [Updated] (SPARK-10000) Consolidate cache memory management and execution memory management

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Description: Memory management in Spark is currently broken down into two disjoint regions: one for ex

[jira] [Resolved] (SPARK-10300) Use tags to control which tests to run depending on changes being tested

[ https://issues.apache.org/jira/browse/SPARK-10300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10300. Resolution: Fixed Second time is the charm? > Use tags to control which tests to run depen

[jira] [Commented] (SPARK-10000) Consolidate cache memory management and execution memory management

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947558#comment-14947558 ] Bowen Zhang commented on SPARK-1: - [~rxin], sounds good. > Consolidate cache mem

[jira] [Assigned] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10980: Assignee: Apache Spark (was: Davies Liu) > Create wrong decimal if unscaled > 1e18 and sc

[jira] [Commented] (SPARK-10942) Not all cached RDDs are unpersisted

[ https://issues.apache.org/jira/browse/SPARK-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947533#comment-14947533 ] Sean Owen commented on SPARK-10942: --- I tried this on master in spark-shell: {code} imp

[jira] [Commented] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947534#comment-14947534 ] Apache Spark commented on SPARK-10980: -- User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10980: Assignee: Davies Liu (was: Apache Spark) > Create wrong decimal if unscaled > 1e18 and sc

[jira] [Commented] (SPARK-10000) Consolidate cache memory management and execution memory management

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947530#comment-14947530 ] Reynold Xin commented on SPARK-1: - [~bowenzhangusa] thanks for the interest. Thi

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10956: Issue Type: Sub-task (was: Improvement) Parent: SPARK-1 > Introduce common memory mana

[jira] [Assigned] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10982: Assignee: Reynold Xin (was: Apache Spark) > Rename ExpressionAggregate -> DeclarativeAggr

[jira] [Commented] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947526#comment-14947526 ] Apache Spark commented on SPARK-10982: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10982: Assignee: Apache Spark (was: Reynold Xin) > Rename ExpressionAggregate -> DeclarativeAggr

[jira] [Updated] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10980: --- Description: Decimal(100L, 20, 2) will become 100 instead of 1000

[jira] [Created] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

Reynold Xin created SPARK-10982: --- Summary: Rename ExpressionAggregate -> DeclarativeAggregate Key: SPARK-10982 URL: https://issues.apache.org/jira/browse/SPARK-10982 Project: Spark Issue Type:

[jira] [Updated] (SPARK-10981) R semijoin leads to Java errors, R leftsemi leads to Spark errors

[ https://issues.apache.org/jira/browse/SPARK-10981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Monica Liu updated SPARK-10981: --- Description: I am using SparkR from RStudio, and I ran into an error with the join function that I r

[jira] [Updated] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10980: --- Summary: Create wrong decimal if unscaled > 1e18 and scale > 0 (was: Create wrong decimal with unsca

[jira] [Resolved] (SPARK-10941) .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity

[ https://issues.apache.org/jira/browse/SPARK-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10941. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8973 [https://github.com/a

[jira] [Created] (SPARK-10981) R semijoin leads to Java errors, R leftsemi leads to Spark errors

Monica Liu created SPARK-10981: -- Summary: R semijoin leads to Java errors, R leftsemi leads to Spark errors Key: SPARK-10981 URL: https://issues.apache.org/jira/browse/SPARK-10981 Project: Spark

[jira] [Updated] (SPARK-10980) Create wrong decimal with unscaled value and precision > 18

[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10980: --- Affects Version/s: 1.2.2 1.3.1 1.4.1 > Create wrong dec

[jira] [Created] (SPARK-10980) Create wrong decimal with unscaled value and precision > 18

Davies Liu created SPARK-10980: -- Summary: Create wrong decimal with unscaled value and precision > 18 Key: SPARK-10980 URL: https://issues.apache.org/jira/browse/SPARK-10980 Project: Spark Issu

[jira] [Assigned] (SPARK-10979) SparkR: Add merge to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10979: Assignee: (was: Apache Spark) > SparkR: Add merge to DataFrame > -

[jira] [Commented] (SPARK-10979) SparkR: Add merge to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947403#comment-14947403 ] Apache Spark commented on SPARK-10979: -- User 'NarineK' has created a pull request fo

[jira] [Assigned] (SPARK-10979) SparkR: Add merge to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10979: Assignee: Apache Spark > SparkR: Add merge to DataFrame > -- >

[jira] [Updated] (SPARK-10979) SparkR: Add merge to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Narine Kokhlikyan updated SPARK-10979: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-9315 > SparkR: Add merge

[jira] [Created] (SPARK-10979) SparkR: Add merge to DataFrame

Narine Kokhlikyan created SPARK-10979: - Summary: SparkR: Add merge to DataFrame Key: SPARK-10979 URL: https://issues.apache.org/jira/browse/SPARK-10979 Project: Spark Issue Type: New Feat

[jira] [Comment Edited] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947387#comment-14947387 ] Khaled Ammar edited comment on SPARK-10945 at 10/7/15 7:07 PM:

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947387#comment-14947387 ] Khaled Ammar commented on SPARK-10945: -- Thank you ~Ankur and ~Sean, I really appreci

[jira] [Resolved] (SPARK-9841) Params.clear needs to be public

[ https://issues.apache.org/jira/browse/SPARK-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-9841. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8619 [https

[jira] [Updated] (SPARK-9963) ML RandomForest cleanup: Move predictNodeIndex to LearningNode

[ https://issues.apache.org/jira/browse/SPARK-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9963: - Description: (updated form original description) Move ml.tree.impl.RandomForest.predictNo

[jira] [Updated] (SPARK-9963) ML RandomForest cleanup: Move predictNodeIndex to LearningNode

[ https://issues.apache.org/jira/browse/SPARK-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9963: - Summary: ML RandomForest cleanup: Move predictNodeIndex to LearningNode (was: ML RandomFo

[jira] [Resolved] (SPARK-10964) YARN + dynamic allocation not working with new RPC backend

[ https://issues.apache.org/jira/browse/SPARK-10964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10964. Resolution: Fixed Assignee: Marcelo Vanzin > YARN + dynamic allocation not working wi

[jira] [Commented] (SPARK-9963) ML RandomForest cleanup: replace predictNodeIndex with predictImpl

[ https://issues.apache.org/jira/browse/SPARK-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947339#comment-14947339 ] Joseph K. Bradley commented on SPARK-9963: -- The PR's waiting for me to take a loo

[jira] [Updated] (SPARK-9963) ML RandomForest cleanup: replace predictNodeIndex with predictImpl

[ https://issues.apache.org/jira/browse/SPARK-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9963: - Shepherd: Joseph K. Bradley > ML RandomForest cleanup: replace predictNodeIndex with predi

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947330#comment-14947330 ] Ankur Dave commented on SPARK-10945: Thanks [~kammar] and [~srowen] for the careful a

[jira] [Commented] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947307#comment-14947307 ] Sean Owen commented on SPARK-10914: --- It fails with a small heap, right -- if you set -X

[jira] [Assigned] (SPARK-10960) SQL with windowing function cannot reference column in inner select block

[ https://issues.apache.org/jira/browse/SPARK-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10960: Assignee: (was: Apache Spark) > SQL with windowing function cannot reference column in

[jira] [Commented] (SPARK-10960) SQL with windowing function cannot reference column in inner select block

[ https://issues.apache.org/jira/browse/SPARK-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947306#comment-14947306 ] Apache Spark commented on SPARK-10960: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-10960) SQL with windowing function cannot reference column in inner select block

[ https://issues.apache.org/jira/browse/SPARK-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10960: Assignee: Apache Spark > SQL with windowing function cannot reference column in inner sele

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947289#comment-14947289 ] Sean Owen commented on SPARK-10945: --- My guess is there is possibly some issue in ... {

[jira] [Created] (SPARK-10978) Allow PrunedFilterScan to eliminate predicates from further evaluation

Russell Alexander Spitzer created SPARK-10978: - Summary: Allow PrunedFilterScan to eliminate predicates from further evaluation Key: SPARK-10978 URL: https://issues.apache.org/jira/browse/SPARK-10978

[jira] [Comment Edited] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947249#comment-14947249 ] Sean Owen edited comment on SPARK-10945 at 10/7/15 5:34 PM: I

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947249#comment-14947249 ] Sean Owen commented on SPARK-10945: --- I mean that and edge from 2 to 3 should appear as

[jira] [Commented] (SPARK-10977) SQL injection bugs in JdbcUtils and DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-10977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947236#comment-14947236 ] Sean Owen commented on SPARK-10977: --- Since this is JBDC and you're making a PreparedSta

[jira] [Commented] (SPARK-10965) Optimize filesEqualRecursive

[ https://issues.apache.org/jira/browse/SPARK-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947234#comment-14947234 ] Mark Grover commented on SPARK-10965: - Thanks Sean. I haven't really decided on the

[jira] [Resolved] (SPARK-10966) Code-generation framework cleanup

[ https://issues.apache.org/jira/browse/SPARK-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10966. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9006 [http

[jira] [Comment Edited] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947211#comment-14947211 ] Khaled Ammar edited comment on SPARK-10945 at 10/7/15 4:58 PM:

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947211#comment-14947211 ] Khaled Ammar commented on SPARK-10945: -- The sort operation sort edges based on the s

[jira] [Updated] (SPARK-10952) Don't explode the environment when HIVE_HOME isn't set.

[ https://issues.apache.org/jira/browse/SPARK-10952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10952: --- Assignee: Kevin Cox > Don't explode the environment when HIVE_HOME isn't set. > -

[jira] [Updated] (SPARK-10952) Don't explode the environment when HIVE_HOME isn't set.

[ https://issues.apache.org/jira/browse/SPARK-10952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10952: --- Fix Version/s: 1.5.2 1.4.2 > Don't explode the environment when HIVE_HOME isn't se

[jira] [Created] (SPARK-10977) SQL injection bugs in JdbcUtils and DataFrameWriter

Rick Hillegas created SPARK-10977: - Summary: SQL injection bugs in JdbcUtils and DataFrameWriter Key: SPARK-10977 URL: https://issues.apache.org/jira/browse/SPARK-10977 Project: Spark Issue T

[jira] [Resolved] (SPARK-10952) Don't explode the environment when HIVE_HOME isn't set.

[ https://issues.apache.org/jira/browse/SPARK-10952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10952. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8994 [https://github.c

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947202#comment-14947202 ] Khaled Ammar commented on SPARK-10945: -- One more followup, In order to further inve

[jira] [Updated] (SPARK-10967) ignore - Incorrect UNION ALL behavior

[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10967: --- Fix Version/s: (was: 1.5.0) > ignore - Incorrect UNION ALL behavior > ---

[jira] [Created] (SPARK-10976) java.lang.UnsupportedOperationException: taskSucceeded() called on a finished JobWaiter

Dr Stephen A Hellberg created SPARK-10976: - Summary: java.lang.UnsupportedOperationException: taskSucceeded() called on a finished JobWaiter Key: SPARK-10976 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Resolved] (SPARK-10752) Implement corr() and cov in DataFrameStatFunctions

[ https://issues.apache.org/jira/browse/SPARK-10752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10752. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8869 [https://github.c

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947171#comment-14947171 ] Sean Owen commented on SPARK-10945: --- (The sort operation you show is just sorting the e

[jira] [Commented] (SPARK-10945) GraphX computes Pagerank with NaN (with some datasets)

[ https://issues.apache.org/jira/browse/SPARK-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947095#comment-14947095 ] Khaled Ammar commented on SPARK-10945: -- Hi, I apologize for the delay but I ran sev

[jira] [Commented] (SPARK-10590) Spark with YARN build is broken

[ https://issues.apache.org/jira/browse/SPARK-10590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947046#comment-14947046 ] Fernando Otero commented on SPARK-10590: [~kevintsai] That PR was not merged (or

[jira] [Created] (SPARK-10975) Shuffle files left behind on Mesos without dynamic allocation

Iulian Dragos created SPARK-10975: - Summary: Shuffle files left behind on Mesos without dynamic allocation Key: SPARK-10975 URL: https://issues.apache.org/jira/browse/SPARK-10975 Project: Spark

[jira] [Assigned] (SPARK-10974) Add progress bar for output operation column

[ https://issues.apache.org/jira/browse/SPARK-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10974: Assignee: Apache Spark > Add progress bar for output operation column > --

[jira] [Commented] (SPARK-10974) Add progress bar for output operation column

[ https://issues.apache.org/jira/browse/SPARK-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947045#comment-14947045 ] Apache Spark commented on SPARK-10974: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-10974) Add progress bar for output operation column

[ https://issues.apache.org/jira/browse/SPARK-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10974: Assignee: (was: Apache Spark) > Add progress bar for output operation column > ---

[jira] [Created] (SPARK-10974) Add progress bar for output operation column

Shixiong Zhu created SPARK-10974: Summary: Add progress bar for output operation column Key: SPARK-10974 URL: https://issues.apache.org/jira/browse/SPARK-10974 Project: Spark Issue Type: Impr

[jira] [Assigned] (SPARK-10973) __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry.

[ https://issues.apache.org/jira/browse/SPARK-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10973: Assignee: Apache Spark > __gettitem__ method throws IndexError exception when we try to ac

[jira] [Updated] (SPARK-10973) __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry.

[ https://issues.apache.org/jira/browse/SPARK-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-10973: --- External issue URL: https://github.com/apache/spark/pull/9009 > __gettitem__ method t

[jira] [Updated] (SPARK-10973) __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry.

[ https://issues.apache.org/jira/browse/SPARK-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-10973: --- External issue URL: (was: https://github.com/apache/spark/pull/9009) > __gettitem__

[jira] [Assigned] (SPARK-10973) __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry.

[ https://issues.apache.org/jira/browse/SPARK-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10973: Assignee: (was: Apache Spark) > __gettitem__ method throws IndexError exception when w

[jira] [Commented] (SPARK-10973) __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry.

[ https://issues.apache.org/jira/browse/SPARK-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947008#comment-14947008 ] Apache Spark commented on SPARK-10973: -- User 'zero323' has created a pull request fo

[jira] [Updated] (SPARK-10972) UDFs in SQL joins

[ https://issues.apache.org/jira/browse/SPARK-10972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Malak updated SPARK-10972: -- Description: Currently expressions used to .join() in DataFrames are limited to column names p

[jira] [Created] (SPARK-10973) __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry.

Maciej Szymkiewicz created SPARK-10973: -- Summary: __gettitem__ method throws IndexError exception when we try to access index after the last non-zero entry. Key: SPARK-10973 URL: https://issues.apache.org/jir

[jira] [Created] (SPARK-10972) UDFs in SQL joins

Michael Malak created SPARK-10972: - Summary: UDFs in SQL joins Key: SPARK-10972 URL: https://issues.apache.org/jira/browse/SPARK-10972 Project: Spark Issue Type: New Feature Compone

[jira] [Closed] (SPARK-10841) Add pushdown support of UDF for parquet

[ https://issues.apache.org/jira/browse/SPARK-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-10841. --- Resolution: Won't Fix > Add pushdown support of UDF for parquet > ---

[jira] [Updated] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-4066: -- Description: Here is the thread Koert started: http://search-hadoop.com/m/JW1q5j8z422/scalastyle+annoys+me+a+lit

[jira] [Updated] (SPARK-10561) Provide tooling for auto-generating Spark SQL reference manual

[ https://issues.apache.org/jira/browse/SPARK-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-10561: --- Description: Here is the discussion thread: http://search-hadoop.com/m/q3RTtcD20F1o62xE Richard Hillegas mad

[jira] [Created] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

Thomas Graves created SPARK-10971: - Summary: sparkR: RRunner should allow setting path to Rscript Key: SPARK-10971 URL: https://issues.apache.org/jira/browse/SPARK-10971 Project: Spark Issue

[jira] [Updated] (SPARK-10669) Link to each language's API in codetabs in ML docs: spark.mllib

[ https://issues.apache.org/jira/browse/SPARK-10669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10669: -- Assignee: Xin Ren > Link to each language's API in codetabs in ML docs: spark.mllib > -

[jira] [Resolved] (SPARK-10669) Link to each language's API in codetabs in ML docs: spark.mllib

[ https://issues.apache.org/jira/browse/SPARK-10669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10669. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8977 [https://github.com

[jira] [Created] (SPARK-10970) Executors overload Hive metastore by making massive connections at execution time

Cheolsoo Park created SPARK-10970: - Summary: Executors overload Hive metastore by making massive connections at execution time Key: SPARK-10970 URL: https://issues.apache.org/jira/browse/SPARK-10970 P

[jira] [Commented] (SPARK-10794) Spark-SQL- select query on table column with binary Data Type displays error message- java.lang.ClassCastException: java.lang.String cannot be cast to [B

[ https://issues.apache.org/jira/browse/SPARK-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946865#comment-14946865 ] Anilkumar Kalshetti commented on SPARK-10794: - Jia Li Thanks for trying. Loo

[jira] [Created] (SPARK-10969) Spark Streaming Kinesis: Allow specifying separate credentials for Kinesis and DynamoDB

Christoph Pirkl created SPARK-10969: --- Summary: Spark Streaming Kinesis: Allow specifying separate credentials for Kinesis and DynamoDB Key: SPARK-10969 URL: https://issues.apache.org/jira/browse/SPARK-10969

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946710#comment-14946710 ] Tony Lin commented on SPARK-10528: -- The discussion thread is really helpful. I encounter

[jira] [Commented] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946623#comment-14946623 ] Ben Moran commented on SPARK-10914: --- I added a properties file to set spark.test.useCom

[jira] [Commented] (SPARK-10968) Incorrect Join behavior in filter conditions

[ https://issues.apache.org/jira/browse/SPARK-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946493#comment-14946493 ] RaviShankar KS commented on SPARK-10968: not actually incorrect. DataFrame d5.val

[jira] [Commented] (SPARK-10965) Optimize filesEqualRecursive

[ https://issues.apache.org/jira/browse/SPARK-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946469#comment-14946469 ] Sean Owen commented on SPARK-10965: --- You don't need it to be assigned to you, just go a

<    1   2