[jira] [Updated] (SPARK-14670) Allow updating SQLMetrics on driver

2016-06-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14670: --- Assignee: Wenchen Fan (was: Andrew Or) > Allow updating SQLMetrics on driver >

[jira] [Resolved] (SPARK-14670) Allow updating SQLMetrics on driver

2016-06-08 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14670. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13189

[jira] [Updated] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15832: Priority: Major (was: Minor) > Embedded IN/EXISTS predicate subquery throws TreeNodeException >

[jira] [Commented] (SPARK-15837) PySpark ML Word2Vec should support maxSentenceLength

2016-06-08 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321899#comment-15321899 ] Weichen Xu commented on SPARK-15837: I'll work on it and create a PR soon ! > PySpark ML Word2Vec

[jira] [Created] (SPARK-15837) PySpark ML Word2Vec should support maxSentenceLength

2016-06-08 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-15837: --- Summary: PySpark ML Word2Vec should support maxSentenceLength Key: SPARK-15837 URL: https://issues.apache.org/jira/browse/SPARK-15837 Project: Spark Issue

[jira] [Created] (SPARK-15836) Spark 2.0/master maven snapshots are broken

2016-06-08 Thread Yin Huai (JIRA)
Yin Huai created SPARK-15836: Summary: Spark 2.0/master maven snapshots are broken Key: SPARK-15836 URL: https://issues.apache.org/jira/browse/SPARK-15836 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321730#comment-15321730 ] Reynold Xin commented on SPARK-15086: - This is really tricky. On one hand it'd be great not having to

[jira] [Commented] (SPARK-15835) The read path of json doesn't support write path when schema contains Options

2016-06-08 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321729#comment-15321729 ] Burak Yavuz commented on SPARK-15835: - cc [~cloud_fan] > The read path of json doesn't support write

[jira] [Created] (SPARK-15835) The read path of json doesn't support write path when schema contains Options

2016-06-08 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-15835: --- Summary: The read path of json doesn't support write path when schema contains Options Key: SPARK-15835 URL: https://issues.apache.org/jira/browse/SPARK-15835 Project:

[jira] [Assigned] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15369: Assignee: (was: Apache Spark) > Investigate selectively using Jython for parts of

[jira] [Commented] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321684#comment-15321684 ] Apache Spark commented on SPARK-15369: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15369: Assignee: Apache Spark > Investigate selectively using Jython for parts of PySpark >

[jira] [Updated] (SPARK-11415) Catalyst DateType Shifts Input Data by Local Timezone

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-11415: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15834 > Catalyst DateType Shifts Input Data

[jira] [Updated] (SPARK-14057) sql time stamps do not respect time zones

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14057: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15834 > sql time stamps do not respect time

[jira] [Created] (SPARK-15834) Time zone / locale sensitivity umbrella

2016-06-08 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-15834: -- Summary: Time zone / locale sensitivity umbrella Key: SPARK-15834 URL: https://issues.apache.org/jira/browse/SPARK-15834 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-15613) Incorrect days to millis conversion

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-15613: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15834 > Incorrect days to millis conversion

[jira] [Updated] (SPARK-13268) SQL Timestamp stored as GMT but toString returns GMT-08:00

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13268: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15834 > SQL Timestamp stored as GMT but

[jira] [Closed] (SPARK-14923) Support "Extended" in "Describe" table DDL

2016-06-08 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Meng closed SPARK-14923. --- Resolution: Fixed > Support "Extended" in "Describe" table DDL > --

[jira] [Resolved] (SPARK-15735) Allow specifying min time to run in microbenchmarks

2016-06-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15735. --- Resolution: Resolved Assignee: Eric Liang Target Version/s:

[jira] [Created] (SPARK-15833) REST API for streaming

2016-06-08 Thread Lin Chan (JIRA)
Lin Chan created SPARK-15833: Summary: REST API for streaming Key: SPARK-15833 URL: https://issues.apache.org/jira/browse/SPARK-15833 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-8426) Add blacklist mechanism for YARN container allocation

2016-06-08 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321637#comment-15321637 ] Imran Rashid commented on SPARK-8426: - [~kayousterhout] shoudl work now, sorry about that. > Add

[jira] [Commented] (SPARK-8426) Add blacklist mechanism for YARN container allocation

2016-06-08 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321626#comment-15321626 ] Kay Ousterhout commented on SPARK-8426: --- Imran can you enable commenting on the design doc? > Add

[jira] [Updated] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-08 Thread Ioana Delaney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ioana Delaney updated SPARK-15832: -- Component/s: SQL > Embedded IN/EXISTS predicate subquery throws TreeNodeException >

[jira] [Assigned] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15832: Assignee: Apache Spark > Embedded IN/EXISTS predicate subquery throws TreeNodeException >

[jira] [Commented] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321617#comment-15321617 ] Apache Spark commented on SPARK-15832: -- User 'ioana-delaney' has created a pull request for this

[jira] [Assigned] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15832: Assignee: (was: Apache Spark) > Embedded IN/EXISTS predicate subquery throws

[jira] [Created] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-08 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-15832: - Summary: Embedded IN/EXISTS predicate subquery throws TreeNodeException Key: SPARK-15832 URL: https://issues.apache.org/jira/browse/SPARK-15832 Project: Spark

[jira] [Assigned] (SPARK-15791) NPE in ScalarSubquery

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15791: Assignee: Eric Liang (was: Apache Spark) > NPE in ScalarSubquery > -

[jira] [Commented] (SPARK-15791) NPE in ScalarSubquery

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321574#comment-15321574 ] Apache Spark commented on SPARK-15791: -- User 'ericl' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15791) NPE in ScalarSubquery

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15791: Assignee: Apache Spark (was: Eric Liang) > NPE in ScalarSubquery > -

[jira] [Commented] (SPARK-11695) Set s3a credentials by default similarly to s3 and s3n

2016-06-08 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321539#comment-15321539 ] Steve Loughran commented on SPARK-11695: There's some interesting ramifications to this code: it

[jira] [Updated] (SPARK-15743) Prevent saving with all-column partitioning

2016-06-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-15743: - Labels: releasenotes (was: ) > Prevent saving with all-column partitioning >

[jira] [Updated] (SPARK-15830) Spark application should get hive tokens only when it is required

2016-06-08 Thread Yesha Vora (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated SPARK-15830: --- Summary: Spark application should get hive tokens only when it is required (was: Spark application

[jira] [Created] (SPARK-15831) Kryo 2.21 TreeMap serialization bug causes random job failures with RDDs of HBase puts

2016-06-08 Thread JIRA
Charles Gariépy-Ikeson created SPARK-15831: -- Summary: Kryo 2.21 TreeMap serialization bug causes random job failures with RDDs of HBase puts Key: SPARK-15831 URL:

[jira] [Created] (SPARK-15830) Spark application should get hive tokens only when needed

2016-06-08 Thread Yesha Vora (JIRA)
Yesha Vora created SPARK-15830: -- Summary: Spark application should get hive tokens only when needed Key: SPARK-15830 URL: https://issues.apache.org/jira/browse/SPARK-15830 Project: Spark Issue

[jira] [Created] (SPARK-15829) spark master webpage links to application UI broke when running in cluster mode

2016-06-08 Thread Andrew Davidson (JIRA)
Andrew Davidson created SPARK-15829: --- Summary: spark master webpage links to application UI broke when running in cluster mode Key: SPARK-15829 URL: https://issues.apache.org/jira/browse/SPARK-15829

[jira] [Created] (SPARK-15828) YARN is not aware of Spark's External Shuffle Service

2016-06-08 Thread Miles Crawford (JIRA)
Miles Crawford created SPARK-15828: -- Summary: YARN is not aware of Spark's External Shuffle Service Key: SPARK-15828 URL: https://issues.apache.org/jira/browse/SPARK-15828 Project: Spark

[jira] [Updated] (SPARK-15828) YARN is not aware of Spark's External Shuffle Service

2016-06-08 Thread Miles Crawford (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miles Crawford updated SPARK-15828: --- Description: When using Spark with dynamic allocation, it is common for all containers on a

[jira] [Commented] (SPARK-12712) test-dependencies.sh script fails when run against empty .m2 cache

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321415#comment-15321415 ] Apache Spark commented on SPARK-12712: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Updated] (SPARK-15807) Support varargs for dropDuplicates in Dataset/DataFrame

2016-06-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15807: -- Description: This issue adds `varargs`-types `dropDuplicates` functions in

[jira] [Commented] (SPARK-12712) test-dependencies.sh script fails when run against empty .m2 cache

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321402#comment-15321402 ] Josh Rosen commented on SPARK-12712: Minimal local reproduction: {code} rm -rf

[jira] [Updated] (SPARK-12712) test-dependencies.sh script fails when run against empty .m2 cache

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12712: --- Summary: test-dependencies.sh script fails when run against empty .m2 cache (was:

[jira] [Updated] (SPARK-12712) test-dependencies.sh script fails when run against empty .m2 cache

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-12712: --- Target Version/s: 1.6.2, 2.0.0 Component/s: Project Infra > test-dependencies.sh script

[jira] [Reopened] (SPARK-12712) test-dependencies.sh fails with difference in manifests

2016-06-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-12712: Assignee: Josh Rosen I've managed to reproduce this problem in one of my own CI environments.

[jira] [Commented] (SPARK-11765) Avoid assign UI port between browser unsafe ports (or just 4045: lockd)

2016-06-08 Thread Willy Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321311#comment-15321311 ] Willy Lee commented on SPARK-11765: --- So users should be picking their own ports hoping they aren't

[jira] [Commented] (SPARK-15827) Publish Spark's forked sbt-pom-reader to Maven Central

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321248#comment-15321248 ] Apache Spark commented on SPARK-15827: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15827) Publish Spark's forked sbt-pom-reader to Maven Central

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15827: Assignee: Apache Spark (was: Josh Rosen) > Publish Spark's forked sbt-pom-reader to

[jira] [Commented] (SPARK-15678) Not use cache on appends and overwrites

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321249#comment-15321249 ] Apache Spark commented on SPARK-15678: -- User 'sameeragarwal' has created a pull request for this

[jira] [Assigned] (SPARK-15827) Publish Spark's forked sbt-pom-reader to Maven Central

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15827: Assignee: Josh Rosen (was: Apache Spark) > Publish Spark's forked sbt-pom-reader to

[jira] [Commented] (SPARK-15783) Fix more flakiness: o.a.s.scheduler.BlacklistIntegrationSuite

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321247#comment-15321247 ] Apache Spark commented on SPARK-15783: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

2016-06-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321238#comment-15321238 ] Marcelo Vanzin commented on SPARK-14485: Just to distill my comment a little bit more, without

[jira] [Created] (SPARK-15827) Publish Spark's forked sbt-pom-reader to Maven Central

2016-06-08 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-15827: -- Summary: Publish Spark's forked sbt-pom-reader to Maven Central Key: SPARK-15827 URL: https://issues.apache.org/jira/browse/SPARK-15827 Project: Spark Issue

[jira] [Commented] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

2016-06-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321202#comment-15321202 ] Marcelo Vanzin commented on SPARK-14485: I commented on the PR, but will mostly repeat it here.

[jira] [Commented] (SPARK-15826) PipedRDD to strictly use UTF-8 and not rely on default encoding

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321203#comment-15321203 ] Apache Spark commented on SPARK-15826: -- User 'tejasapatil' has created a pull request for this

[jira] [Assigned] (SPARK-15826) PipedRDD to strictly use UTF-8 and not rely on default encoding

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15826: Assignee: (was: Apache Spark) > PipedRDD to strictly use UTF-8 and not rely on

[jira] [Assigned] (SPARK-15826) PipedRDD to strictly use UTF-8 and not rely on default encoding

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15826: Assignee: Apache Spark > PipedRDD to strictly use UTF-8 and not rely on default encoding

[jira] [Updated] (SPARK-15826) PipedRDD to strictly use UTF-8 and not rely on default encoding

2016-06-08 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15826: Description: Encountered an issue wherein the code works in some cluster but fails on another one

[jira] [Updated] (SPARK-15826) PipedRDD to strictly use UTF-8 and not rely on default encoding

2016-06-08 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15826: Summary: PipedRDD to strictly use UTF-8 and not rely on default encoding (was: PipedRDD relies on

[jira] [Updated] (SPARK-15826) PipedRDD relies on JVM default encoding

2016-06-08 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15826: Description: Encountered an issue wherein the code works in some cluster but fails on another one

[jira] [Created] (SPARK-15826) PipedRDD relies on JVM default encoding

2016-06-08 Thread Tejas Patil (JIRA)
Tejas Patil created SPARK-15826: --- Summary: PipedRDD relies on JVM default encoding Key: SPARK-15826 URL: https://issues.apache.org/jira/browse/SPARK-15826 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

2016-06-08 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321177#comment-15321177 ] Kay Ousterhout commented on SPARK-14485: I commented on the pull request, but want to continue

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2016-06-08 Thread Sandeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321116#comment-15321116 ] Sandeep commented on SPARK-2984: Can this bug be reopened please ? I am seeing the issue with spark 1.6.1.

[jira] [Created] (SPARK-15825) sort-merge-join gives invalid results when joining on a tupled key

2016-06-08 Thread Andres Perez (JIRA)
Andres Perez created SPARK-15825: Summary: sort-merge-join gives invalid results when joining on a tupled key Key: SPARK-15825 URL: https://issues.apache.org/jira/browse/SPARK-15825 Project: Spark

[jira] [Updated] (SPARK-15046) When running hive-thriftserver with yarn on a secure cluster the workers fail with java.lang.NumberFormatException

2016-06-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-15046: --- Target Version/s: 2.0.0 Priority: Blocker (was: Major) Component/s:

[jira] [Commented] (SPARK-11183) enable support for mesos 0.24+

2016-06-08 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320914#comment-15320914 ] Charles Allen commented on SPARK-11183: --- Eventually it could be wroth adopting something like

[jira] [Assigned] (SPARK-15812) Allow sorting on aggregated streaming dataframe when the output mode is Complete

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15812: Assignee: Tathagata Das (was: Apache Spark) > Allow sorting on aggregated streaming

[jira] [Commented] (SPARK-15812) Allow sorting on aggregated streaming dataframe when the output mode is Complete

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320908#comment-15320908 ] Apache Spark commented on SPARK-15812: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15812) Allow sorting on aggregated streaming dataframe when the output mode is Complete

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15812: Assignee: Apache Spark (was: Tathagata Das) > Allow sorting on aggregated streaming

[jira] [Commented] (SPARK-15784) Add Power Iteration Clustering to spark.ml

2016-06-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320879#comment-15320879 ] Miao Wang commented on SPARK-15784: --- I can work on this. Thanks! > Add Power Iteration Clustering to

[jira] [Commented] (SPARK-11183) enable support for mesos 0.24+

2016-06-08 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320840#comment-15320840 ] Charles Allen commented on SPARK-11183: --- Being able to enable the fetch cache

[jira] [Commented] (SPARK-15821) Should we use mvn -T for multithreaded Spark builds?

2016-06-08 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320614#comment-15320614 ] Adam Roberts commented on SPARK-15821: -- Agreed with your comment on tests, the above pull request is

[jira] [Commented] (SPARK-15821) Should we use mvn -T for multithreaded Spark builds?

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320604#comment-15320604 ] Apache Spark commented on SPARK-15821: -- User 'a-roberts' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15821) Should we use mvn -T for multithreaded Spark builds?

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15821: Assignee: (was: Apache Spark) > Should we use mvn -T for multithreaded Spark builds?

[jira] [Assigned] (SPARK-15821) Should we use mvn -T for multithreaded Spark builds?

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15821: Assignee: Apache Spark > Should we use mvn -T for multithreaded Spark builds? >

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320495#comment-15320495 ] Sean Owen commented on SPARK-15086: --- I think we can alter the API if that makes sense. Those really

[jira] [Commented] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-08 Thread Gabor Ratky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320466#comment-15320466 ] Gabor Ratky commented on SPARK-15811: - I was able to reproduce the issue on a Databricks Cluster

[jira] [Commented] (SPARK-15824) Run 'with ... insert ... select' failed when use spark thriftserver

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320456#comment-15320456 ] Apache Spark commented on SPARK-15824: -- User 'Sephiroth-Lin' has created a pull request for this

[jira] [Assigned] (SPARK-15824) Run 'with ... insert ... select' failed when use spark thriftserver

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15824: Assignee: Apache Spark > Run 'with ... insert ... select' failed when use spark

[jira] [Assigned] (SPARK-15824) Run 'with ... insert ... select' failed when use spark thriftserver

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15824: Assignee: (was: Apache Spark) > Run 'with ... insert ... select' failed when use

[jira] [Commented] (SPARK-13566) Deadlock between MemoryStore and BlockManager

2016-06-08 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-13566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320431#comment-15320431 ] Josef Lindman Hörnlund commented on SPARK-13566: Do we know if this affects 1.5 as well?

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-08 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320428#comment-15320428 ] Weichen Xu commented on SPARK-15086: So, if considering java API compatibility with old version, the

[jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol

2016-06-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320416#comment-15320416 ] Takeshi Yamamuro commented on SPARK-15816: -- Oh, this is a good start point for that. About Q1:

[jira] [Created] (SPARK-15824) Run 'with ... insert ... select' failed when use spark thriftserver

2016-06-08 Thread Weizhong (JIRA)
Weizhong created SPARK-15824: Summary: Run 'with ... insert ... select' failed when use spark thriftserver Key: SPARK-15824 URL: https://issues.apache.org/jira/browse/SPARK-15824 Project: Spark

[jira] [Assigned] (SPARK-15823) Add @property for 'property' in MulticlassMetrics

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15823: Assignee: Apache Spark > Add @property for 'property' in MulticlassMetrics >

[jira] [Assigned] (SPARK-15823) Add @property for 'property' in MulticlassMetrics

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15823: Assignee: (was: Apache Spark) > Add @property for 'property' in MulticlassMetrics >

[jira] [Commented] (SPARK-15823) Add @property for 'property' in MulticlassMetrics

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320393#comment-15320393 ] Apache Spark commented on SPARK-15823: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-15823) Add @property for 'property' in MulticlassMetrics

2016-06-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320392#comment-15320392 ] Sean Owen commented on SPARK-15823: --- OK. Can you review these changes for similar issues in one pass?

[jira] [Created] (SPARK-15823) Add @property for 'property' in MulticlassMetrics

2016-06-08 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-15823: Summary: Add @property for 'property' in MulticlassMetrics Key: SPARK-15823 URL: https://issues.apache.org/jira/browse/SPARK-15823 Project: Spark Issue

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320386#comment-15320386 ] Sean Owen commented on SPARK-15086: --- Yes, that's clear, but this doesn't address the 3 questions above,

[jira] [Commented] (SPARK-15821) Should we use mvn -T for multithreaded Spark builds?

2016-06-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320385#comment-15320385 ] Sean Owen commented on SPARK-15821: --- For building (i.e. compiling and packaging) -- sure. For testing,

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-08 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320380#comment-15320380 ] Pete Robbins commented on SPARK-15822: -- I'm investigating this and will attach the app and config

[jira] [Created] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-08 Thread Pete Robbins (JIRA)
Pete Robbins created SPARK-15822: Summary: segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true Key: SPARK-15822 URL: https://issues.apache.org/jira/browse/SPARK-15822

[jira] [Commented] (SPARK-14146) Imported implicits can't be found in Spark REPL in some cases

2016-06-08 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320357#comment-15320357 ] Prashant Sharma commented on SPARK-14146: - So I tried that option, looks like it does not help

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-08 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320351#comment-15320351 ] Weichen Xu commented on SPARK-15086: I think the Java API should be the same to scala API if

[jira] [Created] (SPARK-15821) Should we use mvn -T for multithreaded Spark builds?

2016-06-08 Thread Adam Roberts (JIRA)
Adam Roberts created SPARK-15821: Summary: Should we use mvn -T for multithreaded Spark builds? Key: SPARK-15821 URL: https://issues.apache.org/jira/browse/SPARK-15821 Project: Spark Issue

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-08 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15820: --- Summary: Add Catalog.refreshTable into python API (was: Add spark-SQL Catalog.refreshTable into

[jira] [Assigned] (SPARK-15820) Add spark-SQL Catalog.refreshTable into python api

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15820: Assignee: (was: Apache Spark) > Add spark-SQL Catalog.refreshTable into python api >

[jira] [Updated] (SPARK-15820) Add spark-SQL Catalog.refreshTable into python api

2016-06-08 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15820: --- External issue ID: (was: SPARK-15367) > Add spark-SQL Catalog.refreshTable into python api >

[jira] [Commented] (SPARK-15820) Add spark-SQL Catalog.refreshTable into python api

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320304#comment-15320304 ] Apache Spark commented on SPARK-15820: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-15820) Add spark-SQL Catalog.refreshTable into python api

2016-06-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15820: Assignee: Apache Spark > Add spark-SQL Catalog.refreshTable into python api >

  1   2   >