[jira] [Created] (SPARK-24506) Spark.ui.filters not applied to /sqlserver/ url

2018-06-09 Thread t oo (JIRA)
t oo created SPARK-24506: Summary: Spark.ui.filters not applied to /sqlserver/ url Key: SPARK-24506 URL: https://issues.apache.org/jira/browse/SPARK-24506 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-24504) Implement SparkSQL authorization plugin in Apache Ranger

2018-06-10 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo resolved SPARK-24504. -- Resolution: Duplicate > Implement SparkSQL authorization plugin in Apache Ranger >

[jira] [Updated] (SPARK-24511) Spark WebUI allows Weak TLS Protocols

2018-06-10 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24511: - Attachment: SSL.PNG > Spark WebUI allows Weak TLS Protocols > - > >

[jira] [Created] (SPARK-24512) SparkSQL ThriftServer port (ie 10015) supports TLSv1.0

2018-06-10 Thread t oo (JIRA)
t oo created SPARK-24512: Summary: SparkSQL ThriftServer port (ie 10015) supports TLSv1.0 Key: SPARK-24512 URL: https://issues.apache.org/jira/browse/SPARK-24512 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-24511) Spark WebUI allows Weak TLS Protocols

2018-06-10 Thread t oo (JIRA)
t oo created SPARK-24511: Summary: Spark WebUI allows Weak TLS Protocols Key: SPARK-24511 URL: https://issues.apache.org/jira/browse/SPARK-24511 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-06-15 Thread t oo (JIRA)
t oo created SPARK-24570: Summary: SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc) Key: SPARK-24570 URL: https://issues.apache.org/jira/browse/SPARK-24570

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-06-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Attachment: connect-to-sql-db-ssms-locate-table.png > SparkSQL - show schemas/tables in dropdowns of SQL client

[jira] [Commented] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-06-16 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514771#comment-16514771 ] t oo commented on SPARK-24570: -- [~maropu] - with same sql tool (I have tried several), connecting to

[jira] [Created] (SPARK-24509) Spark WebUI [security] - Web Server Version Disclosure

2018-06-10 Thread t oo (JIRA)
t oo created SPARK-24509: Summary: Spark WebUI [security] - Web Server Version Disclosure Key: SPARK-24509 URL: https://issues.apache.org/jira/browse/SPARK-24509 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-24508) Spark WebUIs [Security] - Inadequate Cache Directive Headers

2018-06-10 Thread t oo (JIRA)
t oo created SPARK-24508: Summary: Spark WebUIs [Security] - Inadequate Cache Directive Headers Key: SPARK-24508 URL: https://issues.apache.org/jira/browse/SPARK-24508 Project: Spark Issue Type:

[jira] [Created] (SPARK-24510) Spark WebUI filters use Basic Authentication [security]

2018-06-10 Thread t oo (JIRA)
t oo created SPARK-24510: Summary: Spark WebUI filters use Basic Authentication [security] Key: SPARK-24510 URL: https://issues.apache.org/jira/browse/SPARK-24510 Project: Spark Issue Type:

[jira] [Created] (SPARK-24503) Implement SparkSQL authorization plugin in Apache Ranger

2018-06-09 Thread t oo (JIRA)
t oo created SPARK-24503: Summary: Implement SparkSQL authorization plugin in Apache Ranger Key: SPARK-24503 URL: https://issues.apache.org/jira/browse/SPARK-24503 Project: Spark Issue Type: New

[jira] [Created] (SPARK-24504) Implement SparkSQL authorization plugin in Apache Ranger

2018-06-09 Thread t oo (JIRA)
t oo created SPARK-24504: Summary: Implement SparkSQL authorization plugin in Apache Ranger Key: SPARK-24504 URL: https://issues.apache.org/jira/browse/SPARK-24504 Project: Spark Issue Type: New

[jira] [Comment Edited] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2018-07-01 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528572#comment-16528572 ] t oo edited comment on SPARK-24621 at 7/1/18 9:36 AM: --

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-06-29 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528412#comment-16528412 ] t oo commented on SPARK-23534: -- Should this jira title be renamed to hadoop 3.1? as it looks like hadoop

[jira] [Commented] (SPARK-16859) History Server storage information is missing

2018-06-29 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528406#comment-16528406 ] t oo commented on SPARK-16859: -- still an issue in spark 2.3.1, storage tab on the SparkHistoryServer UI is

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-06-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Description: An end-user SQL client tool (ie in the screenshot) can list tables from hiveserver2 and major DBs

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-06-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Description: An end-user SQL client tool (ie in the screenshot) can list tables from hiveserver2 and major DBs

[jira] [Created] (SPARK-24729) Spark - stackoverflow error - org.apache.spark.sql.catalyst.plans.QueryPlan

2018-07-02 Thread t oo (JIRA)
t oo created SPARK-24729: Summary: Spark - stackoverflow error - org.apache.spark.sql.catalyst.plans.QueryPlan Key: SPARK-24729 URL: https://issues.apache.org/jira/browse/SPARK-24729 Project: Spark

[jira] [Commented] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2018-06-30 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528572#comment-16528572 ] t oo commented on SPARK-24621: -- [https://github.com/apache/spark/pull/21514/commits]   

[jira] [Updated] (SPARK-24701) SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app

2018-06-29 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24701: - Labels: master security ui web web-ui (was: ) > SparkMaster WebUI allow all appids to be shown in detail on

[jira] [Created] (SPARK-24701) SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app

2018-06-29 Thread t oo (JIRA)
t oo created SPARK-24701: Summary: SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app Key: SPARK-24701 URL: https://issues.apache.org/jira/browse/SPARK-24701

[jira] [Updated] (SPARK-24620) [security] Disable 'kill' button next to applications on Spark WebUI

2018-06-21 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24620: - Attachment: spark_master-one-app.png > [security] Disable 'kill' button next to applications on Spark WebUI >

[jira] [Updated] (SPARK-24620) [security] Disable 'kill' button next to applications on Spark WebUI

2018-06-21 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24620: - Attachment: (was: spark_master-one-app.png) > [security] Disable 'kill' button next to applications on

[jira] [Updated] (SPARK-24621) WebUI - application 'name' urls point to IP instead of DNS name

2018-06-21 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24621: - Attachment: spark_master-one-app.png > WebUI - application 'name' urls point to IP instead of DNS name >

[jira] [Created] (SPARK-24621) WebUI - application 'name' urls point to IP instead of DNS name

2018-06-21 Thread t oo (JIRA)
t oo created SPARK-24621: Summary: WebUI - application 'name' urls point to IP instead of DNS name Key: SPARK-24621 URL: https://issues.apache.org/jira/browse/SPARK-24621 Project: Spark Issue Type:

[jira] [Created] (SPARK-24618) Allow ability to consume driver memory on worker hosts not master (option for clustermode to wait for returncode?)

2018-06-21 Thread t oo (JIRA)
t oo created SPARK-24618: Summary: Allow ability to consume driver memory on worker hosts not master (option for clustermode to wait for returncode?) Key: SPARK-24618 URL:

[jira] [Created] (SPARK-24619) DOCO - usage of authenticate.secret between masters/workers not clear

2018-06-21 Thread t oo (JIRA)
t oo created SPARK-24619: Summary: DOCO - usage of authenticate.secret between masters/workers not clear Key: SPARK-24619 URL: https://issues.apache.org/jira/browse/SPARK-24619 Project: Spark Issue

[jira] [Created] (SPARK-24620) [security] Disable 'kill' button next to applications on Spark WebUI

2018-06-21 Thread t oo (JIRA)
t oo created SPARK-24620: Summary: [security] Disable 'kill' button next to applications on Spark WebUI Key: SPARK-24620 URL: https://issues.apache.org/jira/browse/SPARK-24620 Project: Spark Issue

[jira] [Created] (SPARK-24617) Spark driver not requesting another executor once original executor exits due to 'lost worker'

2018-06-21 Thread t oo (JIRA)
t oo created SPARK-24617: Summary: Spark driver not requesting another executor once original executor exits due to 'lost worker' Key: SPARK-24617 URL: https://issues.apache.org/jira/browse/SPARK-24617

[jira] [Updated] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2018-06-21 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24621: - Summary: WebUI - application 'name' urls point to http instead of https (even when ssl enabled) (was: WebUI -

[jira] [Updated] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2018-06-21 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24621: - Description: See attached ApplicationID correctly points to DNS url but Name points to IP address Update: I

[jira] [Commented] (SPARK-24701) SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app

2018-07-19 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548920#comment-16548920 ] t oo commented on SPARK-24701: -- [~guoxiaolongzte] - see attached screenshot > SparkMaster WebUI allow all

[jira] [Updated] (SPARK-24701) SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather than different ports per app

2018-07-19 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24701: - Attachment: spark_ports.png > SparkMaster WebUI allow all appids to be shown in detail on port 4040 rather >

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2018-09-05 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604167#comment-16604167 ] t oo commented on SPARK-18112: -- [~hyukjin.kwon] [~srowen] can this ticket be re-opened? This code is still

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2018-09-05 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604162#comment-16604162 ] t oo commented on SPARK-13446: -- [~cloud_fan] I am hitting same issue as [~elgalu] :( > Spark need to

[jira] [Commented] (SPARK-23443) Spark with Glue as external catalog

2018-09-05 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604331#comment-16604331 ] t oo commented on SPARK-23443: -- [~ameen.tayy...@gmail.com] any luck with the first PR? > Spark with Glue

[jira] [Commented] (SPARK-24360) Support Hive 3.0 metastore

2018-09-05 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604292#comment-16604292 ] t oo commented on SPARK-24360: -- [~dongjoon] Can this be merged to master? Also, can hive3.1 support be

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-07-04 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Attachment: hive.png spark.png > SparkSQL - show schemas/tables in dropdowns of SQL client

[jira] [Commented] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-07-04 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532403#comment-16532403 ] t oo commented on SPARK-24570: -- hive.png shows tables, spark.png does not. Using simba driver > SparkSQL -

[jira] [Created] (SPARK-24789) SparkThriftServer - support for hive.server2.authentication.ldap.userFilter

2018-07-11 Thread t oo (JIRA)
t oo created SPARK-24789: Summary: SparkThriftServer - support for hive.server2.authentication.ldap.userFilter Key: SPARK-24789 URL: https://issues.apache.org/jira/browse/SPARK-24789 Project: Spark

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

2018-04-12 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436400#comment-16436400 ] t oo commented on SPARK-23710: -- Hive 2.3.3 has just been released, any plans for support? > Upgrade Hive to

[jira] [Created] (SPARK-24686) Provide spark distributions for hadoop-2.8 rather than hadoop-2.7 as releases on apache mirrors

2018-06-28 Thread t oo (JIRA)
t oo created SPARK-24686: Summary: Provide spark distributions for hadoop-2.8 rather than hadoop-2.7 as releases on apache mirrors Key: SPARK-24686 URL: https://issues.apache.org/jira/browse/SPARK-24686

[jira] [Commented] (SPARK-8659) Spark SQL Thrift Server does NOT honour hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory

2018-06-29 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527325#comment-16527325 ] t oo commented on SPARK-8659: - _"show_ current _roles" gives an error about sql catalyst parser_ > Spark SQL

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-10-16 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652449#comment-16652449 ] t oo commented on SPARK-18673: -- bump > Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2018-10-16 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652440#comment-16652440 ] t oo commented on SPARK-20202: -- bump > Remove references to org.spark-project.hive >

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2018-10-28 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1572#comment-1572 ] t oo commented on SPARK-24437: -- can this be merged? > Memory leak in UnsafeHashedRelation >

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

2018-10-28 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1573#comment-1573 ] t oo commented on SPARK-22575: -- can this be merged? > Making Spark Thrift Server clean up its cache >

[jira] [Commented] (SPARK-24422) Add JDK9+ in our Jenkins' build servers

2018-10-02 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635250#comment-16635250 ] t oo commented on SPARK-24422: -- https://jenkins.io/blog/2018/06/17/running-jenkins-with-java10-11/ > Add

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK9+

2018-10-02 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635253#comment-16635253 ] t oo commented on SPARK-24417: -- [http://jdk.java.net/11/]  i guess can skip java9-10 and go to 11 > Build

[jira] [Created] (SPARK-25596) TLS1.3 support

2018-10-02 Thread t oo (JIRA)
t oo created SPARK-25596: Summary: TLS1.3 support Key: SPARK-25596 URL: https://issues.apache.org/jira/browse/SPARK-25596 Project: Spark Issue Type: Sub-task Components: Project Infra

[jira] [Commented] (SPARK-24360) Support Hive 3.1 metastore

2018-09-19 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621218#comment-16621218 ] t oo commented on SPARK-24360: -- bump > Support Hive 3.1 metastore > -- > >

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

2018-09-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:43 AM: bump @shahid was

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

2018-09-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:46 AM: bump was (Author:

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

2018-09-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628561#comment-16628561 ] t oo commented on SPARK-25502: -- related https://jira.apache.org/jira/browse/SPARK-16859 ? > [Spark Job

[jira] [Comment Edited] (SPARK-16859) History Server storage information is missing

2018-09-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo edited comment on SPARK-16859 at 9/26/18 10:45 AM: bump [~ashahid] was

[jira] [Commented] (SPARK-18112) Spark2.x does not support read data from Hive 2.x metastore

2018-09-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629420#comment-16629420 ] t oo commented on SPARK-18112: -- here, here! > Spark2.x does not support read data from Hive 2.x metastore

[jira] [Commented] (SPARK-16859) History Server storage information is missing

2018-09-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628554#comment-16628554 ] t oo commented on SPARK-16859: -- bump > History Server storage information is missing >

[jira] [Created] (SPARK-25526) SparkSQL over REST API not JDBC

2018-09-25 Thread t oo (JIRA)
t oo created SPARK-25526: Summary: SparkSQL over REST API not JDBC Key: SPARK-25526 URL: https://issues.apache.org/jira/browse/SPARK-25526 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-26354) Ability to return schema prefix before dataframe column names

2018-12-12 Thread t oo (JIRA)
t oo created SPARK-26354: Summary: Ability to return schema prefix before dataframe column names Key: SPARK-26354 URL: https://issues.apache.org/jira/browse/SPARK-26354 Project: Spark Issue Type:

[jira] [Created] (SPARK-26373) Spark UI 'environment' tab - column to indicate default vs overridden values

2018-12-14 Thread t oo (JIRA)
t oo created SPARK-26373: Summary: Spark UI 'environment' tab - column to indicate default vs overridden values Key: SPARK-26373 URL: https://issues.apache.org/jira/browse/SPARK-26373 Project: Spark

[jira] [Created] (SPARK-27284) Spark Standalone aggregated logs in 1 file per appid (ala yarn -logs -applicationId)

2019-03-26 Thread t oo (JIRA)
t oo created SPARK-27284: Summary: Spark Standalone aggregated logs in 1 file per appid (ala yarn -logs -applicationId) Key: SPARK-27284 URL: https://issues.apache.org/jira/browse/SPARK-27284 Project: Spark

[jira] [Commented] (SPARK-27284) Spark Standalone aggregated logs in 1 file per appid (ala yarn -logs -applicationId)

2019-04-01 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806462#comment-16806462 ] t oo commented on SPARK-27284: -- [~srowen] in yarn mode if you want all logs for a single spark app you can

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-02 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782330#comment-16782330 ] t oo commented on SPARK-26998: -- [https://github.com/apache/spark/pull/23820] is only about hiding password

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-05 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784791#comment-16784791 ] t oo commented on SPARK-26998: -- [~gsomogyi] please take it forward. [~kabhwan] truststore password being

[jira] [Created] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-02-26 Thread t oo (JIRA)
t oo created SPARK-26998: Summary: spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode Key: SPARK-26998 URL: https://issues.apache.org/jira/browse/SPARK-26998

[jira] [Updated] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-02-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-26998: - Labels: SECURITY Security secur security security-issue (was: ) > spark.ssl.keyStorePassword in plaintext on

[jira] [Commented] (SPARK-25757) Upgrade netty-all from 4.1.17.Final to 4.1.30.Final

2019-02-23 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775873#comment-16775873 ] t oo commented on SPARK-25757: -- [~lipzhu] want to upg netty to 4.1.33? > Upgrade netty-all from

[jira] [Commented] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2019-02-17 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770373#comment-16770373 ] t oo commented on SPARK-22860: -- Not just in worker log, but in 'ps -ef' process list > Spark workers log

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

2019-02-17 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770344#comment-16770344 ] t oo commented on SPARK-20286: -- gentle ping > dynamicAllocation.executorIdleTimeout is ignored after

[jira] [Commented] (SPARK-25594) OOM in long running applications even with UI disabled

2019-02-16 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770230#comment-16770230 ] t oo commented on SPARK-25594: -- please merge > OOM in long running applications even with UI disabled >

[jira] [Commented] (SPARK-17556) Executor side broadcast for broadcast joins

2019-02-17 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770481#comment-16770481 ] t oo commented on SPARK-17556: -- please don't leave us [~scwf] > Executor side broadcast for broadcast

[jira] [Commented] (SPARK-16859) History Server storage information is missing

2019-02-17 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770482#comment-16770482 ] t oo commented on SPARK-16859: -- please don't leave us [~Hauer] > History Server storage information is

[jira] [Commented] (SPARK-16859) History Server storage information is missing

2019-02-17 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770679#comment-16770679 ] t oo commented on SPARK-16859: -- I know how you feel, I submitted a PR to fix a security flaw (SPARK-22860)

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2019-02-12 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766577#comment-16766577 ] t oo commented on SPARK-23534: -- time for hadoop 3.2 profile? am curious to know if hadoop3 offers much

[jira] [Commented] (SPARK-26777) SQL worked in 2.3.2 and fails in 2.4.0

2019-02-12 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766526#comment-16766526 ] t oo commented on SPARK-26777: -- [~yuri.budilov] - can you try in spark-shell instead of pyspark? Also with

[jira] [Commented] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2019-02-11 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765482#comment-16765482 ] t oo commented on SPARK-22860: -- gentle ping, fix waiting to be committed > Spark workers log ssl passwords

[jira] [Commented] (SPARK-8659) Spark SQL Thrift Server does NOT honour hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory

2019-02-11 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765481#comment-16765481 ] t oo commented on SPARK-8659: - bump > Spark SQL Thrift Server does NOT honour >

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

2019-02-11 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765487#comment-16765487 ] t oo commented on SPARK-24437: -- [~dvogelbacher] any luck on the test? > Memory leak in

[jira] [Commented] (SPARK-25998) TorrentBroadcast holds strong reference to broadcast object

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769708#comment-16769708 ] t oo commented on SPARK-25998: -- any chance for a backport to 2.x? > TorrentBroadcast holds strong

[jira] [Commented] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769851#comment-16769851 ] t oo commented on SPARK-24621: -- :( > WebUI - application 'name' urls point to http instead of https (even

[jira] [Commented] (SPARK-24338) Spark SQL fails to create a Hive table when running in a Apache Sentry-secured Environment

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769871#comment-16769871 ] t oo commented on SPARK-24338: -- gentle ping > Spark SQL fails to create a Hive table when running in a

[jira] [Commented] (SPARK-21074) Parquet files are read fully even though only count() is requested

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769866#comment-16769866 ] t oo commented on SPARK-21074: -- @[~spektom] gentle ping > Parquet files are read fully even though only

[jira] [Commented] (SPARK-24010) Select from table needs read access on DB folder when storage based auth is enabled

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769872#comment-16769872 ] t oo commented on SPARK-24010: -- gentle ping > Select from table needs read access on DB folder when

[jira] [Commented] (SPARK-11248) Spark hivethriftserver is using the wrong user to while getting HDFS permissions

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769881#comment-16769881 ] t oo commented on SPARK-11248: -- gentle ping > Spark hivethriftserver is using the wrong user to while

[jira] [Commented] (SPARK-17700) org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769888#comment-16769888 ] t oo commented on SPARK-17700: -- did u fix/\? > org.apache.hadoop.hive.ql.metadata.HiveException: Failed to

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769887#comment-16769887 ] t oo commented on SPARK-5159: - gentle ping > Thrift server does not respect hive.server2.enable.doAs=true >

[jira] [Commented] (SPARK-23576) SparkSQL - Decimal data missing decimal point

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769864#comment-16769864 ] t oo commented on SPARK-23576: -- gentle ping > SparkSQL - Decimal data missing decimal point >

[jira] [Commented] (SPARK-11499) Spark History Server UI should respect protocol when doing redirection

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769873#comment-16769873 ] t oo commented on SPARK-11499: -- gentle ping > Spark History Server UI should respect protocol when doing

[jira] [Commented] (SPARK-8325) Ability to provide role based row level authorization through Spark SQL

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769877#comment-16769877 ] t oo commented on SPARK-8325: - gentle ping > Ability to provide role based row level authorization through

[jira] [Commented] (SPARK-8033) spark-sql thriftserver security authorization bugs!

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769880#comment-16769880 ] t oo commented on SPARK-8033: - [~nilone] did u manage to get RBAC in spark? > spark-sql thriftserver

[jira] [Commented] (SPARK-24789) SparkThriftServer - support for hive.server2.authentication.ldap.userFilter

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769847#comment-16769847 ] t oo commented on SPARK-24789: -- [~Riza] did u fix? > SparkThriftServer - support for

[jira] [Commented] (SPARK-24686) Provide spark distributions for hadoop-2.8 rather than hadoop-2.7 as releases on apache mirrors

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769852#comment-16769852 ] t oo commented on SPARK-24686: -- [~Riza] did u fix? > Provide spark distributions for hadoop-2.8 rather

[jira] [Commented] (SPARK-12008) Spark hive security authorization doesn't work as Apache hive's

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769853#comment-16769853 ] t oo commented on SPARK-12008: -- what are folks doing as workaround? > Spark hive security authorization

[jira] [Commented] (SPARK-2750) Add Https support for Web UI

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769869#comment-16769869 ] t oo commented on SPARK-2750: - this is not completely fixed - see 

[jira] [Commented] (SPARK-18665) Spark ThriftServer jobs where are canceled are still “STARTED”

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769884#comment-16769884 ] t oo commented on SPARK-18665: -- gentle ping > Spark ThriftServer jobs where are canceled are still

[jira] [Commented] (SPARK-15046) When running hive-thriftserver with yarn on a secure cluster the workers fail with java.lang.NumberFormatException

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769890#comment-16769890 ] t oo commented on SPARK-15046: -- gentle ping > When running hive-thriftserver with yarn on a secure cluster

[jira] [Commented] (SPARK-22374) STS ran into OOM in a secure cluster

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769885#comment-16769885 ] t oo commented on SPARK-22374: -- gentle ping > STS ran into OOM in a secure cluster >

[jira] [Commented] (SPARK-17516) Current user info is not checked on STS in DML queries

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769889#comment-16769889 ] t oo commented on SPARK-17516: -- gentle ping > Current user info is not checked on STS in DML queries >

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

2019-02-15 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16769710#comment-16769710 ] t oo commented on SPARK-22575: -- gentle ping > Making Spark Thrift Server clean up its cache >

  1   2   3   >