[jira] [Commented] (SPARK-27055) Update Structured Streaming documentation because of DSv2 changes

2019-03-06 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785469#comment-16785469 ] Sandeep Katta commented on SPARK-27055: --- [~gsomogyi] thank you for reporting this, I will raise

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785581#comment-16785581 ] Stavros Kontopoulos commented on SPARK-26997: - [~shaneknapp] are the test nodes upgraded so

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 12:37 PM:

[jira] [Issue Comment Deleted] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-26997: Comment: was deleted (was: [~shaneknapp] are the test nodes upgraded so we can

[jira] [Created] (SPARK-27071) Expose additional metrics in status.api.v1.StageData

2019-03-06 Thread Tom van Bussel (JIRA)
Tom van Bussel created SPARK-27071: -- Summary: Expose additional metrics in status.api.v1.StageData Key: SPARK-27071 URL: https://issues.apache.org/jira/browse/SPARK-27071 Project: Spark

[jira] [Assigned] (SPARK-26509) Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.x's Vectorized Reader

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26509: Assignee: (was: Apache Spark) > Parquet DELTA_BYTE_ARRAY is not supported in Spark

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 1:34 PM: -

[jira] [Commented] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2019-03-06 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785583#comment-16785583 ] Gabor Somogyi commented on SPARK-24621: --- I've analyzed through this issue with the latest master

[jira] [Assigned] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27070: Assignee: (was: Apache Spark) > DefaultPartitionCoalescer can lock up driver for

[jira] [Assigned] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27070: Assignee: Apache Spark > DefaultPartitionCoalescer can lock up driver for hours >

[jira] [Commented] (SPARK-26330) Duplicate query execution events generated for SQL commands

2019-03-06 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785466#comment-16785466 ] Sandeep Katta commented on SPARK-26330: --- [~vanzin] I will work on this issue, and will raise the

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785575#comment-16785575 ] Stavros Kontopoulos commented on SPARK-26742: - We need to backport it to 2.4.x the client

[jira] [Assigned] (SPARK-26509) Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.x's Vectorized Reader

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26509: Assignee: Apache Spark > Parquet DELTA_BYTE_ARRAY is not supported in Spark 2.x's

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785581#comment-16785581 ] Stavros Kontopoulos edited comment on SPARK-26997 at 3/6/19 12:37 PM:

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 12:34 PM:

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

2019-03-06 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785640#comment-16785640 ] Chakravarthi commented on SPARK-27017: -- Hi [~uNxe] ,can you please provide which hive version you

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785575#comment-16785575 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 12:35 PM:

[jira] [Updated] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Yuli Fiterman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuli Fiterman updated SPARK-27070: -- Patch:https://github.com/apache/spark/pull/23986 > DefaultPartitionCoalescer can lock up driver

[jira] [Commented] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-06 Thread Andy A (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785434#comment-16785434 ] Andy A commented on SPARK-27059: That was it - thanks. > spark-submit on kubernetes cluster does not

[jira] [Closed] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-06 Thread Andy A (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy A closed SPARK-27059. -- > spark-submit on kubernetes cluster does not recognise k8s --master property >

[jira] [Commented] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785427#comment-16785427 ] Shahid K I commented on SPARK-27068: Hi, In the jobs page, both completed and failed jobs will

[jira] [Comment Edited] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785451#comment-16785451 ] Shahid K I edited comment on SPARK-27068 at 3/6/19 9:39 AM: Okay. This is

[jira] [Commented] (SPARK-27068) Support failed jobs ui and completed jobs ui use different queue

2019-03-06 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785451#comment-16785451 ] Shahid K I commented on SPARK-27068: Okay. This is configuration issue. This can be fixed by

[jira] [Created] (SPARK-27070) DefaultPartitionCoalescer can lock up driver for hours

2019-03-06 Thread Yuli Fiterman (JIRA)
Yuli Fiterman created SPARK-27070: - Summary: DefaultPartitionCoalescer can lock up driver for hours Key: SPARK-27070 URL: https://issues.apache.org/jira/browse/SPARK-27070 Project: Spark

[jira] [Commented] (SPARK-27017) Creating orc table with special symbols in column name via spark.sql

2019-03-06 Thread Henryk Cesnolovic (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785647#comment-16785647 ] Henryk Cesnolovic commented on SPARK-27017: --- Hive 1.2.1000.2.6.5.0-292 > Creating orc table

[jira] [Updated] (SPARK-26016) Document that UTF-8 is required in text data source

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26016: -- Issue Type: Improvement (was: Bug) Summary: Document that UTF-8 is required in text data

[jira] [Updated] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Chandu Kavar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandu Kavar updated SPARK-27061: - Summary: Expose Driver UI port on driver service to access UI using service (was: Expose 4040

[jira] [Assigned] (SPARK-27047) Document stop-slave.sh in spark-standalone

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27047: - Assignee: Ajith S > Document stop-slave.sh in spark-standalone >

[jira] [Resolved] (SPARK-27031) Avoid double formatting in timestampToString

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27031. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23936

[jira] [Created] (SPARK-27073) Handling of IdleStateEvent causes the normal connection to close

2019-03-06 Thread dzcxzl (JIRA)
dzcxzl created SPARK-27073: -- Summary: Handling of IdleStateEvent causes the normal connection to close Key: SPARK-27073 URL: https://issues.apache.org/jira/browse/SPARK-27073 Project: Spark Issue

[jira] [Assigned] (SPARK-27073) Handling of IdleStateEvent causes the normal connection to close

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27073: Assignee: Apache Spark > Handling of IdleStateEvent causes the normal connection to

[jira] [Created] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27074: --- Summary: Refactor HiveClientImpl runHive Key: SPARK-27074 URL: https://issues.apache.org/jira/browse/SPARK-27074 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Haripriya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haripriya updated SPARK-27072: -- Summary: Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

[jira] [Created] (SPARK-27072) Changing the parameter value of completedStage.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Haripriya (JIRA)
Haripriya created SPARK-27072: - Summary: Changing the parameter value of completedStage.sort to X prints stacktrace in sparkWebUI Key: SPARK-27072 URL: https://issues.apache.org/jira/browse/SPARK-27072

[jira] [Assigned] (SPARK-27073) Handling of IdleStateEvent causes the normal connection to close

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27073: Assignee: (was: Apache Spark) > Handling of IdleStateEvent causes the normal

[jira] [Updated] (SPARK-27035) Current time with microsecond resolution

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27035: -- Docs Text: In Spark version 2.4 and earlier, the `current_timestamp` function returns a timestamp

[jira] [Commented] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785713#comment-16785713 ] Chakravarthi commented on SPARK-27072: -- Hi [~vanzin] , Please confirm, is it ok to print stacktrace

[jira] [Assigned] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27061: Assignee: (was: Apache Spark) > Expose Driver UI port on driver service to access UI

[jira] [Assigned] (SPARK-26981) Add 'Recall_at_k' metric to RankingMetrics

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26981: - Assignee: Masahiro Kazama > Add 'Recall_at_k' metric to RankingMetrics >

[jira] [Resolved] (SPARK-26981) Add 'Recall_at_k' metric to RankingMetrics

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26981. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23881

[jira] [Resolved] (SPARK-26602) Subsequent queries are failing after querying the UDF which is loaded with wrong hdfs path

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26602. --- Resolution: Not A Problem > Subsequent queries are failing after querying the UDF which is loaded

[jira] [Resolved] (SPARK-27047) Document stop-slave.sh in spark-standalone

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27047. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23960

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785734#comment-16785734 ] Sean Owen commented on SPARK-26742: --- The problem is that that's a major dependency upgrade in a

[jira] [Assigned] (SPARK-27031) Avoid double formatting in timestampToString

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27031: - Assignee: Maxim Gekk > Avoid double formatting in timestampToString >

[jira] [Assigned] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27061: Assignee: Apache Spark > Expose Driver UI port on driver service to access UI using

[jira] [Resolved] (SPARK-27035) Current time with microsecond resolution

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27035. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23945

[jira] [Assigned] (SPARK-27035) Current time with microsecond resolution

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27035: - Assignee: Maxim Gekk > Current time with microsecond resolution >

[jira] [Commented] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

2019-03-06 Thread Chandu Kavar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785724#comment-16785724 ] Chandu Kavar commented on SPARK-27061: -- I started working on it.  > Expose Driver UI port on

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785829#comment-16785829 ] shane knapp commented on SPARK-26742: - [~skonto] – the nodes all have the latest minikube on them

[jira] [Assigned] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27074: Assignee: (was: Apache Spark) > Refactor HiveClientImpl runHive >

[jira] [Commented] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

2019-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785889#comment-16785889 ] Marcelo Vanzin commented on SPARK-27072: I'd expect an error 401 here, but you get a status 200.

[jira] [Commented] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-03-06 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785900#comment-16785900 ] Udbhav Agrawal commented on SPARK-24669: Thankyou [~dongjoon] > Managed table was not cleared

[jira] [Commented] (SPARK-24624) Can not mix vectorized and non-vectorized UDFs

2019-03-06 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785920#comment-16785920 ] peay commented on SPARK-24624: -- Are there plans to support something similar for aggregation functions? >

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785970#comment-16785970 ] shane knapp commented on SPARK-26742: - you can submit a PR now. it will fail on the existing ubuntu

[jira] [Commented] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-06 Thread Terry Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786001#comment-16786001 ] Terry Kim commented on SPARK-27006: --- Thanks for the feedback. We are happy to work on addressing some

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786019#comment-16786019 ] Stavros Kontopoulos commented on SPARK-26742: - Cool! The question is does it pass with

[jira] [Commented] (SPARK-18748) UDF multiple evaluations causes very poor performance

2019-03-06 Thread Qingbo Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786029#comment-16786029 ] Qingbo Hu commented on SPARK-18748: --- We have the same problem when using Spark structured streaming.

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786023#comment-16786023 ] shane knapp commented on SPARK-26742: - i think you mean: s/minikube/k8s anyways, exactly which

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:59 PM: -

[jira] [Resolved] (SPARK-27023) Kubernetes client timeouts should be configurable

2019-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27023. --- Resolution: Fixed Assignee: Onur Satici Fix Version/s: 3.0.0 This is

[jira] [Updated] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Description: Hive 3.1.1's {{CommandProcessor}} have 2 changes: # HIVE-17626(Hive 3.0.0) add

[jira] [Updated] (SPARK-27045) SQL tab in UI shows actual SQL instead of callsite

2019-03-06 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Summary: SQL tab in UI shows actual SQL instead of callsite (was: SQL tab in UI shows callsite instead

[jira] [Updated] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

2019-03-06 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Issue Type: Improvement (was: Bug) > SQL tab in UI shows callsite instead of actual SQL >

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:58 PM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:58 PM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786012#comment-16786012 ] shane knapp commented on SPARK-26742: - 2.4 k8s integration tests pass w/the client upgrade to 4.1.2:

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786047#comment-16786047 ] shane knapp edited comment on SPARK-26742 at 3/6/19 7:17 PM: - your PR works

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786047#comment-16786047 ] shane knapp commented on SPARK-26742: - works for me against k8s v1.11.7: {noformat} [INFO] [INFO]

[jira] [Assigned] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27074: Assignee: Apache Spark > Refactor HiveClientImpl runHive >

[jira] [Assigned] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple tim

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-25250: Assignee: Parth Gandhi > Race condition with tasks running when new attempt for same

[jira] [Updated] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple time

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25250: - Fix Version/s: 3.0.0 2.4.1 > Race condition with tasks running when new

[jira] [Resolved] (SPARK-25250) Race condition with tasks running when new attempt for same stage is created leads to other task in the next attempt running on the same partition id retry multiple tim

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-25250. -- Resolution: Fixed > Race condition with tasks running when new attempt for same stage is

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785962#comment-16785962 ] Stavros Kontopoulos commented on SPARK-26742: - I created one for the 2.4 branch. > Bump

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785987#comment-16785987 ] shane knapp commented on SPARK-26742: - ok, i found the 2.4 PR:

[jira] [Commented] (SPARK-27063) Spark on K8S Integration Tests timeouts are too short for some test clusters

2019-03-06 Thread Rob Vesse (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786051#comment-16786051 ] Rob Vesse commented on SPARK-27063: --- [~skonto] Yes we have experienced the same problem, I think my

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785842#comment-16785842 ] Sean Owen commented on SPARK-26742: --- [~shaneknapp] ping me when ready and I can try the PR again. >

[jira] [Updated] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Description: Hive 3.1.1's {{CommandProcessor}} have 2 changes: # HIVE-17626(Hive 3.0.0) add

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785962#comment-16785962 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:14 PM: -

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786019#comment-16786019 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 6:59 PM: -

[jira] [Commented] (SPARK-27006) SPIP: .NET bindings for Apache Spark

2019-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786030#comment-16786030 ] Sean Owen commented on SPARK-27006: --- You can use the Apache license, and follow Apache processes,

[jira] [Resolved] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24669. --- Resolution: Fixed Assignee: Udbhav Agrawal Fix Version/s: 3.0.0

[jira] [Resolved] (SPARK-27065) avoid more than one active task set managers for a stage

2019-03-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-27065. -- Resolution: Fixed Fix Version/s: 2.3.4 2.4.1

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785983#comment-16785983 ] shane knapp commented on SPARK-26742: - please link to the PRs (master + 2.4) here and i can test

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786146#comment-16786146 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 9:27 PM: -

[jira] [Created] (SPARK-27076) Getting the timeout error while reading parquet files from s3

2019-03-06 Thread srinivas rao gajjala (JIRA)
srinivas rao gajjala created SPARK-27076: Summary: Getting the timeout error while reading parquet files from s3 Key: SPARK-27076 URL: https://issues.apache.org/jira/browse/SPARK-27076

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786069#comment-16786069 ] Stavros Kontopoulos commented on SPARK-26742: - That is awesome! > Bump Kubernetes Client

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786097#comment-16786097 ] shane knapp commented on SPARK-26742: - [~skonto] -- i was able to downgrade k8s successfully w/o

[jira] [Created] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

2019-03-06 Thread Shahid K I (JIRA)
Shahid K I created SPARK-27075: -- Summary: Sorting table column in SQL WEBUI page throws 'IllegalArgumentException' Key: SPARK-27075 URL: https://issues.apache.org/jira/browse/SPARK-27075 Project: Spark

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Jiaxin Shan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786140#comment-16786140 ] Jiaxin Shan commented on SPARK-26742: - Agree to target v1.13.x even though 4.1.2 may not pass

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786142#comment-16786142 ] shane knapp commented on SPARK-26742: - my testing shows that 1.13.x passes our integration tests w/o

[jira] [Resolved] (SPARK-27019) Spark UI's SQL tab shows inconsistent values

2019-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27019. Resolution: Fixed Fix Version/s: 2.4.2 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-27019) Spark UI's SQL tab shows inconsistent values

2019-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27019: -- Assignee: Shahid K I > Spark UI's SQL tab shows inconsistent values >

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786060#comment-16786060 ] shane knapp commented on SPARK-26742: - i was unable to get 1.11.7 to work until i did a `minikube

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786056#comment-16786056 ] Stavros Kontopoulos commented on SPARK-26742: - I mean `minikube` binary because that flag

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786074#comment-16786074 ] shane knapp commented on SPARK-26742: - [~skonto] the only problems i've had are when i'm going

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786102#comment-16786102 ] Stavros Kontopoulos commented on SPARK-26742: - Yeah I was able to do so thanks. I was also

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786102#comment-16786102 ] Stavros Kontopoulos edited comment on SPARK-26742 at 3/6/19 8:29 PM: -

[jira] [Commented] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786146#comment-16786146 ] Stavros Kontopoulos commented on SPARK-26742: - That is a good point [~seedjeffwan] I expect

[jira] [Comment Edited] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

2019-03-06 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786068#comment-16786068 ] shane knapp edited comment on SPARK-26742 at 3/6/19 9:18 PM: - 1.12.6 passes:

[jira] [Updated] (SPARK-27023) Kubernetes client timeouts should be configurable

2019-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27023: -- Issue Type: Improvement (was: New Feature) > Kubernetes client timeouts should be

  1   2   >