[jira] [Commented] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528967#comment-16528967 ] Apache Spark commented on SPARK-24707: -- User 'sidhavratha' has created a pull request for this

[jira] [Assigned] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24707: Assignee: (was: Apache Spark) > Enable spark-kafka-streaming to maintain min buffer

[jira] [Assigned] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24707: Assignee: Apache Spark > Enable spark-kafka-streaming to maintain min buffer using async

[jira] [Updated] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sidhavratha Kumar updated SPARK-24707: -- Component/s: (was: Structured Streaming) DStreams > Enable

[jira] [Updated] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sidhavratha Kumar updated SPARK-24707: -- Description: Currently Spark Kafka RDD will block on kafka consumer poll. Specially

[jira] [Updated] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

[ https://issues.apache.org/jira/browse/SPARK-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sidhavratha Kumar updated SPARK-24707: -- Attachment: 40_partition_topic_without_buffer.pdf > Enable spark-kafka-streaming to

[jira] [Created] (SPARK-24707) Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll

Sidhavratha Kumar created SPARK-24707: - Summary: Enable spark-kafka-streaming to maintain min buffer using async thread to avoid blocking kafka poll Key: SPARK-24707 URL:

[jira] [Assigned] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24470: Assignee: (was: Apache Spark) > RestSubmissionClient to be robust against 404 & non

[jira] [Commented] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528955#comment-16528955 ] Apache Spark commented on SPARK-24470: -- User 'rekhajoshm' has created a pull request for this

[jira] [Assigned] (SPARK-24470) RestSubmissionClient to be robust against 404 & non json responses

[ https://issues.apache.org/jira/browse/SPARK-24470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24470: Assignee: Apache Spark > RestSubmissionClient to be robust against 404 & non json

[jira] [Resolved] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24654. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21640

[jira] [Commented] (SPARK-24507) Description in "Level of Parallelism in Data Receiving" section of Spark Streaming Programming Guide in is not relevan for the recent Kafka direct apprach

[ https://issues.apache.org/jira/browse/SPARK-24507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528928#comment-16528928 ] Apache Spark commented on SPARK-24507: -- User 'rekhajoshm' has created a pull request for this

[jira] [Assigned] (SPARK-24507) Description in "Level of Parallelism in Data Receiving" section of Spark Streaming Programming Guide in is not relevan for the recent Kafka direct apprach

[ https://issues.apache.org/jira/browse/SPARK-24507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24507: Assignee: Apache Spark > Description in "Level of Parallelism in Data Receiving" section

[jira] [Assigned] (SPARK-24507) Description in "Level of Parallelism in Data Receiving" section of Spark Streaming Programming Guide in is not relevan for the recent Kafka direct apprach

[ https://issues.apache.org/jira/browse/SPARK-24507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24507: Assignee: (was: Apache Spark) > Description in "Level of Parallelism in Data

[jira] [Commented] (SPARK-17181) [Spark2.0 web ui]The status of the certain jobs is still displayed as running even if all the stages of this job have already finished

[ https://issues.apache.org/jira/browse/SPARK-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528919#comment-16528919 ] Ron Kitay commented on SPARK-17181: --- [~tgraves] - This seems like a very old issue, however it is not

[jira] [Updated] (SPARK-21443) Very long planning duration for queries with lots of operations

[ https://issues.apache.org/jira/browse/SPARK-21443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated SPARK-21443: Description: Creating a streaming query with large amount of operations and fields (100+)

[jira] [Commented] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528878#comment-16528878 ] Yuming Wang commented on SPARK-24706: - Benchmark result: {noformat}

[jira] [Updated] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24706: Description: (was: Benchmark result: {noformat} ###[ Pushdown

[jira] [Updated] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24706: Description: Benchmark result: {noformat} ###[ Pushdown benchmark

[jira] [Assigned] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24706: Assignee: Apache Spark > Support ByteType and ShortType pushdown to parquet >

[jira] [Assigned] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24706: Assignee: (was: Apache Spark) > Support ByteType and ShortType pushdown to parquet >

[jira] [Commented] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528866#comment-16528866 ] Apache Spark commented on SPARK-24706: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

Yuming Wang created SPARK-24706: --- Summary: Support ByteType and ShortType pushdown to parquet Key: SPARK-24706 URL: https://issues.apache.org/jira/browse/SPARK-24706 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Description: [~smilegator] When loading data using jdbc and enabling spark.sql.adaptive.enabled=true , for

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Description: When loading data using jdbc and enabling spark.sql.adaptive.enabled=true , for example

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Description: When loading data using jdbc and enabling spark.sql.adaptive.enabled=true , for example

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Description: When loading data using jdbc and enabling spark.sql.adaptive.enabled=true , for example

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Attachment: Error stack.txt > Spark.sql.adaptive.enabled=true is enabled and self-join query >

[jira] [Issue Comment Deleted] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Comment: was deleted (was: For example, my query: device_loc table comes from the jdbc data source select

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Affects Version/s: 2.2.1 > Spark.sql.adaptive.enabled=true is enabled and self-join query >

[jira] [Updated] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-24705: -- Description: When loading data using jdbc and enabling spark.sql.adaptive.enabled=true , for example

[jira] [Commented] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

[ https://issues.apache.org/jira/browse/SPARK-24705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528711#comment-16528711 ] cheng commented on SPARK-24705: --- For example, my query: device_loc table comes from the jdbc data source

[jira] [Created] (SPARK-24705) Spark.sql.adaptive.enabled=true is enabled and self-join query

cheng created SPARK-24705: - Summary: Spark.sql.adaptive.enabled=true is enabled and self-join query Key: SPARK-24705 URL: https://issues.apache.org/jira/browse/SPARK-24705 Project: Spark Issue

[jira] [Commented] (SPARK-23725) Improve Hadoop's LineReader to support charsets different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528691#comment-16528691 ] Maxim Gekk commented on SPARK-23725: [~hyukjin.kwon] I am working on the implementation and have

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528629#comment-16528629 ] Hyukjin Kwon commented on SPARK-24530: -- [~vanzin] and [~jerryshao] FYI. > Sphinx doesn't render

[jira] [Assigned] (SPARK-24704) The order of stages in the DAG graph is incorrect

[ https://issues.apache.org/jira/browse/SPARK-24704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24704: Assignee: Apache Spark > The order of stages in the DAG graph is incorrect >

[jira] [Assigned] (SPARK-24704) The order of stages in the DAG graph is incorrect

[ https://issues.apache.org/jira/browse/SPARK-24704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24704: Assignee: (was: Apache Spark) > The order of stages in the DAG graph is incorrect >

[jira] [Commented] (SPARK-24704) The order of stages in the DAG graph is incorrect

[ https://issues.apache.org/jira/browse/SPARK-24704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528625#comment-16528625 ] Apache Spark commented on SPARK-24704: -- User 'stanzhai' has created a pull request for this issue:

[jira] [Updated] (SPARK-24704) The order of stages in the DAG graph is incorrect

[ https://issues.apache.org/jira/browse/SPARK-24704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] StanZhai updated SPARK-24704: - Attachment: WX20180630-161907.png > The order of stages in the DAG graph is incorrect >

[jira] [Created] (SPARK-24704) The order of stages in the DAG graph is incorrect

StanZhai created SPARK-24704: Summary: The order of stages in the DAG graph is incorrect Key: SPARK-24704 URL: https://issues.apache.org/jira/browse/SPARK-24704 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23773) JacksonGenerator does not include keys that have null value for StructTypes

[ https://issues.apache.org/jira/browse/SPARK-23773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23773. -- Resolution: Won't Fix Let me leave this resolved for now given the discussion in the PR but

[jira] [Assigned] (SPARK-24695) Unable to return calendar interval from udf

[ https://issues.apache.org/jira/browse/SPARK-24695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24695: Assignee: (was: Apache Spark) > Unable to return calendar interval from udf >

[jira] [Assigned] (SPARK-24695) Unable to return calendar interval from udf

[ https://issues.apache.org/jira/browse/SPARK-24695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24695: Assignee: Apache Spark > Unable to return calendar interval from udf >

[jira] [Commented] (SPARK-24695) Unable to return calendar interval from udf

[ https://issues.apache.org/jira/browse/SPARK-24695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528597#comment-16528597 ] Apache Spark commented on SPARK-24695: -- User 'priyankagargnitk' has created a pull request for this

[jira] [Resolved] (SPARK-24696) ColumnPruning rule fails to remove extra Project

[ https://issues.apache.org/jira/browse/SPARK-24696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24696. - Resolution: Fixed Fix Version/s: 2.4.0 2.3.2 > ColumnPruning rule fails to

[jira] [Created] (SPARK-24703) Unable to multiply calender interval with long/int

Priyanka Garg created SPARK-24703: - Summary: Unable to multiply calender interval with long/int Key: SPARK-24703 URL: https://issues.apache.org/jira/browse/SPARK-24703 Project: Spark Issue

[jira] [Created] (SPARK-24702) Unable to cast to calendar interval in spark sql.

Priyanka Garg created SPARK-24702: - Summary: Unable to cast to calendar interval in spark sql. Key: SPARK-24702 URL: https://issues.apache.org/jira/browse/SPARK-24702 Project: Spark Issue

[jira] [Assigned] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24621: Assignee: Apache Spark > WebUI - application 'name' urls point to http instead of https

[jira] [Assigned] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24621: Assignee: (was: Apache Spark) > WebUI - application 'name' urls point to http

[jira] [Commented] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528573#comment-16528573 ] Apache Spark commented on SPARK-24621: -- User 'tooptoop4' has created a pull request for this issue:

[jira] [Commented] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528572#comment-16528572 ] t oo commented on SPARK-24621: -- [https://github.com/apache/spark/pull/21514/commits]   

[jira] [Resolved] (SPARK-24638) StringStartsWith support push down

[ https://issues.apache.org/jira/browse/SPARK-24638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24638. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21623

[jira] [Assigned] (SPARK-24638) StringStartsWith support push down

[ https://issues.apache.org/jira/browse/SPARK-24638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24638: --- Assignee: Yuming Wang > StringStartsWith support push down >