[
https://issues.apache.org/jira/browse/SPARK-28930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-28930:
------------------------------------
Assignee: Sujith Chacko
> Spark DESC FORMATTED TABLENAME information display issues
> ---------------------------------------------------------
>
> Key: SPARK-28930
> URL: https://issues.apache.org/jira/browse/SPARK-28930
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.3
> Reporter: jobit mathew
> Assignee: Sujith Chacko
> Priority: Minor
>
> Spark DESC FORMATTED TABLENAME information display issues.Showing incorrect
> *Last Access time and* feeling some information displays can make it better.
> Test steps:
> 1. Open spark sql
> 2. Create table with partition
> CREATE EXTERNAL TABLE IF NOT EXISTS employees_info_extended ( id INT, name
> STRING, usd_flag STRING, salary DOUBLE, deductions MAP<STRING, DOUBLE>,
> address STRING ) PARTITIONED BY (entrytime STRING) STORED AS TEXTFILE
> location 'hdfs://hacluster/user/sparkhive/warehouse';
> 3. from spark sql check the table description
> desc formatted tablename;
> 4. From scala shell check the table description
> sql("desc formatted tablename").show()
> *Issue1:*
> If there is no comment for spark scala shell shows *"null" in small letters*
> but all other places Hive beeline/Spark beeline/Spark SQL it is showing in
> *CAPITAL "NULL*". Better to show same in all places.
>
> {code:java}
> *scala>* sql("desc formatted employees_info_extended").show(false);
> +-----------------------------+---------------------------++-------
> |col_name|data_type|*comment*|
> +-----------------------------+---------------------------++-------
> |id|int|*null*|
> |name|string|*null*|
> |usd_flag|string|*null*|
> |salary|double|*null*|
> |deductions|map<string,double>|*null*|
> |address|string|null|
> |entrytime|string|null|
> | # Partition Information| | |
> | # col_name|data_type|comment|
> |entrytime|string|null|
> | | | |
> | # Detailed Table Information| | |
> |Database|sparkdb__| |
> |Table|employees_info_extended| |
> |Owner|root| |
> *|Created Time |Tue Aug 20 13:42:06 CST 2019| |*
> *|Last Access |Thu Jan 01 08:00:00 CST 1970| |*
> |Created By|Spark 2.4.3| |
> |Type|EXTERNAL| |
> |Provider|hive| |
> +-----------------------------+---------------------------++-------
> only showing top 20 rows
> *scala>*
> {code}
> *Issue 2:*
> Spark SQL "desc formatted tablename" is not showing the header [#
> col_name,data_type,comment|#col_name,data_type,comment] in the top of the
> query result.But header is showing on top of partition description. For
> Better understanding show the header on Top of the query result.Other than in
> spark sql ,we are able to see the header like [#
> col_name,data_type,comment|#col_name,data_type,comment] in spark-beeline &
> hive beeline .
> {code:java}
> *spark-sql>* desc formatted employees_info_extended1;
> id int *NULL*
> name string *NULL*
> usd_flag string NULL
> salary double NULL
> deductions map<string,double> NULL
> address string NULL
> entrytime string NULL
> *
> ## Partition Information*
> ## col_name data_type comment*
> entrytime string *NULL*
>
> *spark-sql>* |
> | |
> {code}
>
> *This is Scala shell showing the headers "|col_name|data_type|*comment*| "*
>
> *scala>* sql("desc formatted employees_info_extended").show(false);
> +-----------------------------+---------------------------++-------
> *|col_name|data_type|*comment*|*
> +-----------------------------+---------------------------++-------
> |id|int|*null*| |name|string|*null*| |usd_flag|string|*null*|
>
> *This is the spark-beeline which is showing the headers*
> 0: jdbc:hive2://10.186.60.158:23040/default> desc formatted employees;
> +-------------------------------+---------------------------------------------------------------------------------+------------------+--+
> | *col_name | data_type | comment |
> +-*------------------------------+---------------------------------------------------------------------------------+------------------+--+
> | name | string | Employee name | | salary | float | Employee salary | | | |
> |
>
> *This is Hive Beeline showing headers*
> *0: jdbc:hive2://10.18.98.147:21066/> desc formatted mytest1;*
> INFO : Concurrency mode is disabled, not creating a lock manager
> +-------------------------------+------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> *| col_name | data_type | comment |*
> +-------------------------------+------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> | # col_name | data_type | comment |
> | col | array<string> | from deserializer |
> | value | string
> *Issue 3:*
> I created the table on Aug 20.So it is showing created time correct .*But
> Last access time showing 1970 Jan 01*. It is not good to show Last access
> time earlier time than the created time.Better to show the correct date and
> time else show UNKNOWN.
> *[Created Time,Tue Aug 20 13:42:06 CST 2019,]*
> *[Last Access,Thu Jan 01 08:00:00 CST 1970,]*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
