[jira] [Resolved] (SPARK-17449) Relation between heartbeatInterval and network timeout

[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17449. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15042 [https://github.co

[jira] [Assigned] (SPARK-17449) Relation between heartbeatInterval and network timeout

[ https://issues.apache.org/jira/browse/SPARK-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-17449: - Assignee: Sean Owen > Relation between heartbeatInterval and network timeout > -

[jira] [Resolved] (SPARK-17496) missing int to float coercion in df.sample() signature

[ https://issues.apache.org/jira/browse/SPARK-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17496. --- Resolution: Not A Problem > missing int to float coercion in df.sample() signature >

[jira] [Resolved] (SPARK-17525) SparkContext.clearFiles() still present in the PySpark bindings though the underlying Scala method was removed in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-17525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17525. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Updated] (SPARK-17525) SparkContext.clearFiles() still present in the PySpark bindings though the underlying Scala method was removed in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-17525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17525: -- Assignee: Sami Jaktholm > SparkContext.clearFiles() still present in the PySpark bindings though the >

[jira] [Resolved] (SPARK-17480) CompressibleColumnBuilder inefficiently call gatherCompressibilityStats

[ https://issues.apache.org/jira/browse/SPARK-17480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17480. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Updated] (SPARK-17480) CompressibleColumnBuilder inefficiently call gatherCompressibilityStats

[ https://issues.apache.org/jira/browse/SPARK-17480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17480: -- Assignee: Ergin Seyfe > CompressibleColumnBuilder inefficiently call gatherCompressibilityStats >

[jira] [Updated] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17445: -- Assignee: Sean Owen Priority: Minor (was: Major) Wiki changes have been made, and both PRs merged.

[jira] [Resolved] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17445. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15075 [https://github.co

[jira] [Updated] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17445: -- Fix Version/s: 2.0.1 > Reference an ASF page as the main place to find third-party packages > -

[jira] [Updated] (SPARK-17418) Spark release must NOT distribute Kinesis related assembly artifact

[ https://issues.apache.org/jira/browse/SPARK-17418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17418: -- Target Version/s: 2.0.1 Priority: Blocker (was: Critical) Summary: Spark relea

[jira] [Updated] (SPARK-17422) Verify that Ganglia, Kinesis (non-assembly) artifacts can be distributed.

[ https://issues.apache.org/jira/browse/SPARK-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17422: -- Target Version/s: 2.0.1 Priority: Critical (was: Major) Description: Ganglia and K

[jira] [Created] (SPARK-17533) I think it's necessary to have an overrided method of union in sparkContext

WangJianfei created SPARK-17533: --- Summary: I think it's necessary to have an overrided method of union in sparkContext Key: SPARK-17533 URL: https://issues.apache.org/jira/browse/SPARK-17533 Project: S

[jira] [Commented] (SPARK-17533) I think it's necessary to have an overrided method of union in sparkContext

[ https://issues.apache.org/jira/browse/SPARK-17533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489971#comment-15489971 ] Sean Owen commented on SPARK-17533: --- You can always repartition() the result. I'm not s

[jira] [Commented] (SPARK-10063) Remove DirectParquetOutputCommitter

[ https://issues.apache.org/jira/browse/SPARK-10063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490023#comment-15490023 ] Steve Loughran commented on SPARK-10063: Amazon EMR's s3 is its own codebase; afr

[jira] [Closed] (SPARK-17533) I think it's necessary to have an overrided method of union in sparkContext

[ https://issues.apache.org/jira/browse/SPARK-17533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangJianfei closed SPARK-17533. --- Resolution: Won't Fix > I think it's necessary to have an overrided method of union in sparkContext

[jira] [Created] (SPARK-17534) Increase timeouts for DirectKafkaStreamSuite tests

Adam Roberts created SPARK-17534: Summary: Increase timeouts for DirectKafkaStreamSuite tests Key: SPARK-17534 URL: https://issues.apache.org/jira/browse/SPARK-17534 Project: Spark Issue Type

[jira] [Commented] (SPARK-17480) CompressibleColumnBuilder inefficiently call gatherCompressibilityStats

[ https://issues.apache.org/jira/browse/SPARK-17480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490397#comment-15490397 ] Apache Spark commented on SPARK-17480: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-17534) Increase timeouts for DirectKafkaStreamSuite tests

[ https://issues.apache.org/jira/browse/SPARK-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17534: Assignee: (was: Apache Spark) > Increase timeouts for DirectKafkaStreamSuite tests > -

[jira] [Commented] (SPARK-17534) Increase timeouts for DirectKafkaStreamSuite tests

[ https://issues.apache.org/jira/browse/SPARK-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490416#comment-15490416 ] Apache Spark commented on SPARK-17534: -- User 'a-roberts' has created a pull request

[jira] [Assigned] (SPARK-17534) Increase timeouts for DirectKafkaStreamSuite tests

[ https://issues.apache.org/jira/browse/SPARK-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17534: Assignee: Apache Spark > Increase timeouts for DirectKafkaStreamSuite tests >

[jira] [Created] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

WangJianfei created SPARK-17535: --- Summary: Performance Improvement of Signleton pattern in SparkContext Key: SPARK-17535 URL: https://issues.apache.org/jira/browse/SPARK-17535 Project: Spark I

[jira] [Updated] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangJianfei updated SPARK-17535: Description: I think the singleton pattern of SparkContext is inefficient if there are many reques

[jira] [Updated] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangJianfei updated SPARK-17535: Description: I think the singleton pattern of SparkContext is inefficient if there are many reques

[jira] [Commented] (SPARK-17496) missing int to float coercion in df.sample() signature

[ https://issues.apache.org/jira/browse/SPARK-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490528#comment-15490528 ] Max Moroz commented on SPARK-17496: --- Agreed, I thought 1 means randomly permute the Dat

[jira] [Closed] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangJianfei closed SPARK-17535. --- Resolution: Duplicate > Performance Improvement of Signleton pattern in SparkContext > --

[jira] [Commented] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490610#comment-15490610 ] Sean Owen commented on SPARK-17535: --- This is called "double checked locking" and it doe

[jira] [Created] (SPARK-17536) Minor performance improvement to JDBC batch inserts

John Muller created SPARK-17536: --- Summary: Minor performance improvement to JDBC batch inserts Key: SPARK-17536 URL: https://issues.apache.org/jira/browse/SPARK-17536 Project: Spark Issue Type:

[jira] [Created] (SPARK-17537) Improve performance for reading parquet schema

Yang Wang created SPARK-17537: - Summary: Improve performance for reading parquet schema Key: SPARK-17537 URL: https://issues.apache.org/jira/browse/SPARK-17537 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490654#comment-15490654 ] Andrew Or commented on SPARK-15917: --- Hi Jonathan, I think your proposal makes sense. T

[jira] [Updated] (SPARK-17537) Improve performance for reading parquet schema

[ https://issues.apache.org/jira/browse/SPARK-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated SPARK-17537: -- Description: spark.read.parquet would issue a spark jobs to read parquet schema. When spark.sql.parquet

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490657#comment-15490657 ] Abel Rincón commented on SPARK-16742: - We at Stratio are working on this issue, Str

[jira] [Commented] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490663#comment-15490663 ] Andrew Or commented on SPARK-15917: --- By the way is there a pull request? > Define the

[jira] [Resolved] (SPARK-17409) Query in CTAS is Optimized Twice

[ https://issues.apache.org/jira/browse/SPARK-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17409. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15048 [https://githu

[jira] [Updated] (SPARK-17537) Improve performance for reading parquet schema

[ https://issues.apache.org/jira/browse/SPARK-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated SPARK-17537: -- Description: spark.read.parquet would issue a spark job to read parquet schema. When spark.sql.parquet.

[jira] [Assigned] (SPARK-17537) Improve performance for reading parquet schema

[ https://issues.apache.org/jira/browse/SPARK-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17537: Assignee: (was: Apache Spark) > Improve performance for reading parquet schema > -

[jira] [Commented] (SPARK-17537) Improve performance for reading parquet schema

[ https://issues.apache.org/jira/browse/SPARK-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490684#comment-15490684 ] Apache Spark commented on SPARK-17537: -- User 'yangw1234' has created a pull request

[jira] [Assigned] (SPARK-17537) Improve performance for reading parquet schema

[ https://issues.apache.org/jira/browse/SPARK-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17537: Assignee: Apache Spark > Improve performance for reading parquet schema >

[jira] [Updated] (SPARK-17537) Improve performance for reading parquet schema

[ https://issues.apache.org/jira/browse/SPARK-17537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated SPARK-17537: -- Description: spark.read.parquet would issue a spark jobs to read parquet schema. When spark.sql.parquet

[jira] [Reopened] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangJianfei reopened SPARK-17535: - > Performance Improvement of Signleton pattern in SparkContext >

[jira] [Commented] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490711#comment-15490711 ] WangJianfei commented on SPARK-17535: - Oh sorry, I should use volatile as a class fie

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490731#comment-15490731 ] Jeff Nadler commented on SPARK-17510: - I consider this to be a separate issue from th

[jira] [Created] (SPARK-17538) sqlContext.registerDataFrameAsTable is not working sometimes in spark 2.0

Srinivas Rishindra Pothireddi created SPARK-17538: - Summary: sqlContext.registerDataFrameAsTable is not working sometimes in spark 2.0 Key: SPARK-17538 URL: https://issues.apache.org/jira/browse/SP

[jira] [Updated] (SPARK-17538) sqlContext.registerDataFrameAsTable is not working sometimes in pyspark 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-17538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinivas Rishindra Pothireddi updated SPARK-17538: -- Summary: sqlContext.registerDataFrameAsTable is not working some

[jira] [Created] (SPARK-17539) Streaming Backpressure Starves DirectStream When Used In Combination With Receivers

Jeff Nadler created SPARK-17539: --- Summary: Streaming Backpressure Starves DirectStream When Used In Combination With Receivers Key: SPARK-17539 URL: https://issues.apache.org/jira/browse/SPARK-17539 Pro

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490746#comment-15490746 ] Jeff Nadler commented on SPARK-17510: - I filed SPARK-17539 for the backpressure bug.

[jira] [Updated] (SPARK-17538) sqlContext.registerDataFrameAsTable is not working sometimes in pyspark 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-17538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinivas Rishindra Pothireddi updated SPARK-17538: -- Description: I have a production job in spark 1.6.2 that regist

[jira] [Commented] (SPARK-17535) Performance Improvement of Signleton pattern in SparkContext

[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490737#comment-15490737 ] Sean Owen commented on SPARK-17535: --- I think this happens to work out because the conte

[jira] [Created] (SPARK-17540) SparkR array serde cannot work correctly when array length == 0

Weichen Xu created SPARK-17540: -- Summary: SparkR array serde cannot work correctly when array length == 0 Key: SPARK-17540 URL: https://issues.apache.org/jira/browse/SPARK-17540 Project: Spark

[jira] [Commented] (SPARK-17540) SparkR array serde cannot work correctly when array length == 0

[ https://issues.apache.org/jira/browse/SPARK-17540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490790#comment-15490790 ] Apache Spark commented on SPARK-17540: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-17540) SparkR array serde cannot work correctly when array length == 0

[ https://issues.apache.org/jira/browse/SPARK-17540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17540: Assignee: (was: Apache Spark) > SparkR array serde cannot work correctly when array le

[jira] [Assigned] (SPARK-17540) SparkR array serde cannot work correctly when array length == 0

[ https://issues.apache.org/jira/browse/SPARK-17540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17540: Assignee: Apache Spark > SparkR array serde cannot work correctly when array length == 0 >

[jira] [Commented] (SPARK-15835) The read path of json doesn't support write path when schema contains Options

[ https://issues.apache.org/jira/browse/SPARK-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490836#comment-15490836 ] Chris Horn commented on SPARK-15835: You can work around this issue by providing the

[jira] [Assigned] (SPARK-17536) Minor performance improvement to JDBC batch inserts

[ https://issues.apache.org/jira/browse/SPARK-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17536: Assignee: Apache Spark > Minor performance improvement to JDBC batch inserts > ---

[jira] [Commented] (SPARK-17536) Minor performance improvement to JDBC batch inserts

[ https://issues.apache.org/jira/browse/SPARK-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490849#comment-15490849 ] Apache Spark commented on SPARK-17536: -- User 'blue666man' has created a pull request

[jira] [Assigned] (SPARK-17536) Minor performance improvement to JDBC batch inserts

[ https://issues.apache.org/jira/browse/SPARK-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17536: Assignee: (was: Apache Spark) > Minor performance improvement to JDBC batch inserts >

[jira] [Created] (SPARK-17541) fix some DDL bugs about table management when same-name temp view exists

Wenchen Fan created SPARK-17541: --- Summary: fix some DDL bugs about table management when same-name temp view exists Key: SPARK-17541 URL: https://issues.apache.org/jira/browse/SPARK-17541 Project: Spark

[jira] [Assigned] (SPARK-17541) fix some DDL bugs about table management when same-name temp view exists

[ https://issues.apache.org/jira/browse/SPARK-17541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17541: Assignee: Wenchen Fan (was: Apache Spark) > fix some DDL bugs about table management when

[jira] [Assigned] (SPARK-17541) fix some DDL bugs about table management when same-name temp view exists

[ https://issues.apache.org/jira/browse/SPARK-17541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17541: Assignee: Apache Spark (was: Wenchen Fan) > fix some DDL bugs about table management when

[jira] [Commented] (SPARK-17541) fix some DDL bugs about table management when same-name temp view exists

[ https://issues.apache.org/jira/browse/SPARK-17541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490922#comment-15490922 ] Apache Spark commented on SPARK-17541: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-17542) Compiler warning in UnsafeInMemorySorter class

Frederick Reiss created SPARK-17542: --- Summary: Compiler warning in UnsafeInMemorySorter class Key: SPARK-17542 URL: https://issues.apache.org/jira/browse/SPARK-17542 Project: Spark Issue Ty

[jira] [Created] (SPARK-17543) Missing log4j config file for tests in common/network-shuffle

Frederick Reiss created SPARK-17543: --- Summary: Missing log4j config file for tests in common/network-shuffle Key: SPARK-17543 URL: https://issues.apache.org/jira/browse/SPARK-17543 Project: Spark

[jira] [Resolved] (SPARK-17514) df.take(1) and df.limit(1).collect() perform differently in Python

[ https://issues.apache.org/jira/browse/SPARK-17514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17514. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull reque

[jira] [Commented] (SPARK-17317) Add package vignette to SparkR

[ https://issues.apache.org/jira/browse/SPARK-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490995#comment-15490995 ] Apache Spark commented on SPARK-17317: -- User 'junyangq' has created a pull request f

[jira] [Updated] (SPARK-17542) Compiler warning in UnsafeInMemorySorter class

[ https://issues.apache.org/jira/browse/SPARK-17542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17542: -- Issue Type: Improvement (was: Bug) There are unfortunately a number of warnings, and I don't think we

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491087#comment-15491087 ] Cody Koeninger commented on SPARK-17510: I use direct stream for multiple topic j

[jira] [Comment Edited] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491087#comment-15491087 ] Cody Koeninger edited comment on SPARK-17510 at 9/14/16 6:12 PM: --

[jira] [Commented] (SPARK-16534) Kafka 0.10 Python support

[ https://issues.apache.org/jira/browse/SPARK-16534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491107#comment-15491107 ] Maciej Bryński commented on SPARK-16534: [~rxin] Could you explain your decision

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491165#comment-15491165 ] Jeff Nadler commented on SPARK-17510: - Yes you're right - it's partly about differin

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491163#comment-15491163 ] Bryan Cutler commented on SPARK-17508: -- I had a similar discussion in this PR https

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491186#comment-15491186 ] Evan Zamir commented on SPARK-17508: Honestly, if the documentation was just more exp

[jira] [Comment Edited] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491186#comment-15491186 ] Evan Zamir edited comment on SPARK-17508 at 9/14/16 6:52 PM: -

[jira] [Comment Edited] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491186#comment-15491186 ] Evan Zamir edited comment on SPARK-17508 at 9/14/16 6:53 PM: -

[jira] [Resolved] (SPARK-10747) add support for NULLS FIRST|LAST in ORDER BY clause

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-10747. --- Resolution: Fixed Assignee: Xin Wu Fix Version/s: 2.1.0 > add support

[jira] [Created] (SPARK-17544) Timeout waiting for connection from pool, DataFrame Reader's not closing S3 connections?

Brady Auen created SPARK-17544: -- Summary: Timeout waiting for connection from pool, DataFrame Reader's not closing S3 connections? Key: SPARK-17544 URL: https://issues.apache.org/jira/browse/SPARK-17544

[jira] [Commented] (SPARK-16424) Add support for Structured Streaming to the ML Pipeline API

[ https://issues.apache.org/jira/browse/SPARK-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491250#comment-15491250 ] holdenk commented on SPARK-16424: - Just an update - we have a really early proof of conce

[jira] [Resolved] (SPARK-17511) Dynamic allocation race condition: Containers getting marked failed while releasing

[ https://issues.apache.org/jira/browse/SPARK-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-17511. --- Resolution: Fixed Assignee: Kishor Patil Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491389#comment-15491389 ] Cody Koeninger commented on SPARK-17510: Just for clarity's sake, compute time is

[jira] [Resolved] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17463. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull reque

[jira] [Commented] (SPARK-17544) Timeout waiting for connection from pool, DataFrame Reader's not closing S3 connections?

[ https://issues.apache.org/jira/browse/SPARK-17544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491397#comment-15491397 ] Davies Liu commented on SPARK-17544: Could you post some code to reproduce the issue?

[jira] [Resolved] (SPARK-17472) Better error message for serialization failures of large objects in Python

[ https://issues.apache.org/jira/browse/SPARK-17472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17472. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15026 [https://github.

[jira] [Created] (SPARK-17545) Spark SQL Catalyst doesn't handle ISO 8601 date with colon in offset

Nathan Beyer created SPARK-17545: Summary: Spark SQL Catalyst doesn't handle ISO 8601 date with colon in offset Key: SPARK-17545 URL: https://issues.apache.org/jira/browse/SPARK-17545 Project: Spark

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491412#comment-15491412 ] Jeff Nadler commented on SPARK-17510: - Well... both streams use updateStateByKey.

[jira] [Created] (SPARK-17546) start-* scripts should use hostname --fqdn

Kevin Burton created SPARK-17546: Summary: start-* scripts should use hostname --fqdn Key: SPARK-17546 URL: https://issues.apache.org/jira/browse/SPARK-17546 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17465. Resolution: Fixed Fix Version/s: 1.6.3 Issue resolved by pull request 15022 [https://github.

[jira] [Updated] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17465: --- Assignee: Xing Shi > Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may >

[jira] [Commented] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491437#comment-15491437 ] Apache Spark commented on SPARK-17114: -- User 'hvanhovell' has created a pull request

[jira] [Commented] (SPARK-17544) Timeout waiting for connection from pool, DataFrame Reader's not closing S3 connections?

[ https://issues.apache.org/jira/browse/SPARK-17544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491445#comment-15491445 ] Brady Auen commented on SPARK-17544: http://stackoverflow.com/questions/39498492/spar

[jira] [Updated] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17465: --- Fix Version/s: 2.1.0 2.0.1 > Inappropriate memory management in `org.apache.spark.

[jira] [Assigned] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-17100: -- Assignee: Davies Liu > pyspark filter on a udf column after join gives > java.lang.Unsupporte

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491455#comment-15491455 ] Cody Koeninger commented on SPARK-17510: Ok, next time I get some free hacking ti

[jira] [Commented] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

[ https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491460#comment-15491460 ] Jeff Nadler commented on SPARK-17510: - That would be incredible, thank you very much

[jira] [Assigned] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17346: Assignee: (was: Apache Spark) > Kafka 0.10 support in Structured Streaming > -

[jira] [Assigned] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17346: Assignee: Apache Spark > Kafka 0.10 support in Structured Streaming >

[jira] [Commented] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491487#comment-15491487 ] Apache Spark commented on SPARK-17346: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17100: Assignee: Apache Spark (was: Davies Liu) > pyspark filter on a udf column after join give

[jira] [Assigned] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17100: Assignee: Davies Liu (was: Apache Spark) > pyspark filter on a udf column after join give

[jira] [Commented] (SPARK-17100) pyspark filter on a udf column after join gives java.lang.UnsupportedOperationException

[ https://issues.apache.org/jira/browse/SPARK-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491518#comment-15491518 ] Apache Spark commented on SPARK-17100: -- User 'davies' has created a pull request for

[jira] [Created] (SPARK-17547) Temporary shuffle data files may be leaked following exception in write

Josh Rosen created SPARK-17547: -- Summary: Temporary shuffle data files may be leaked following exception in write Key: SPARK-17547 URL: https://issues.apache.org/jira/browse/SPARK-17547 Project: Spark

[jira] [Assigned] (SPARK-17547) Temporary shuffle data files may be leaked following exception in write

[ https://issues.apache.org/jira/browse/SPARK-17547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17547: Assignee: Apache Spark (was: Josh Rosen) > Temporary shuffle data files may be leaked fol

  1   2   >