[
https://issues.apache.org/jira/browse/IMPALA-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634577#comment-17634577
]
Michael Smith commented on IMPALA-9487:
---------------------------------------
h2. SHOW FILES
Add a new column to SHOW FILES output. Only add this column when at least one
file is erasure coded to avoid changing the format for existing users.
{code:java}
Query: show files in ect
+---------------------------------------------------------------------------------------------------------------------+------+---------------------+--------------+
| Path
| Size | Partition | EC Policy
|
+---------------------------------------------------------------------------------------------------------------------+------+---------------------+--------------+
|
hdfs://localhost:20500/ec-warehouse/ect/date_col=2009-01-01/1d46b6cbf747c7a8-a818238e00000000_1347890556_data.0.txt
| 338B | date_col=2009-01-01 | RS-3-2-1024k |
|
hdfs://localhost:20500/ec-warehouse/ect/date_col=2009-01-02/1d46b6cbf747c7a8-a818238e00000000_588711848_data.0.txt
| 353B | date_col=2009-01-02 | RS-3-2-1024k |
|
hdfs://localhost:20500/ec-warehouse/ect/date_col=2009-01-03/1d46b6cbf747c7a8-a818238e00000000_140483252_data.0.txt
| 353B | date_col=2009-01-03 | RS-3-2-1024k |
|
hdfs://localhost:20500/ec-warehouse/ect/date_col=2009-01-04/1d46b6cbf747c7a8-a818238e00000000_795616328_data.0.txt
| 353B | date_col=2009-01-04 | RS-3-2-1024k |
|
hdfs://localhost:20500/ec-warehouse/ect/date_col=2009-01-05/1d46b6cbf747c7a8-a818238e00000000_234666509_data.0.txt
| 353B | date_col=2009-01-05 | RS-3-2-1024k |
+---------------------------------------------------------------------------------------------------------------------+------+---------------------+--------------+
Fetched 5 row(s) in 0.07s
{code}
h2. DESCRIBE EXTENDED
If the table location (a directory) has an erasure coding policy, list it under
storage information.
{code:java}
Query: describe extended ect
+------------------------------+------------------------------------------------------------+----------------------+
| name | type
| comment |
+------------------------------+------------------------------------------------------------+----------------------+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| id | int
| Add a comment |
| bool_col | boolean
| NULL |
| tinyint_col | tinyint
| NULL |
| smallint_col | smallint
| NULL |
| int_col | int
| NULL |
| bigint_col | bigint
| NULL |
| float_col | float
| NULL |
| double_col | double
| NULL |
| string_col | string
| NULL |
| timestamp_col | timestamp
| NULL |
| | NULL
| NULL |
| # Partition Information | NULL
| NULL |
| # col_name | data_type
| comment |
| | NULL
| NULL |
| date_col | date
| NULL |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| OwnerType: | USER
| NULL |
| Owner: | michael
| NULL |
| CreateTime: | Tue Nov 15 15:10:21 PST 2022
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Location: | hdfs://localhost:20500/ec-warehouse/ect
| NULL |
| Table Type: | EXTERNAL_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | DO_NOT_UPDATE_STATS
| true |
| | EXTERNAL
| TRUE |
| | OBJCAPABILITIES
| EXTREAD,EXTWRITE |
| | STATS_GENERATED
| TASK |
| | TRANSLATED_TO_EXTERNAL
| TRUE |
| | external.table.purge
| TRUE |
| | impala.lastComputeStatsTime
| 1668549634 |
| | numRows
| -1 |
| | totalSize
| 27923 |
| | transient_lastDdlTime
| 1668548361 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
|
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat
| NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | 0
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| Erasure coding policy: | RS-3-2-1024k
| NULL |
| | NULL
| NULL |
| # Constraints | NULL
| NULL |
+------------------------------+------------------------------------------------------------+----------------------+
Fetched 50 row(s) in 0.02s
{code}
> SHOW and DESCRIBE statements should display EC policies
> -------------------------------------------------------
>
> Key: IMPALA-9487
> URL: https://issues.apache.org/jira/browse/IMPALA-9487
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Sahil Takiar
> Assignee: Michael Smith
> Priority: Major
> Labels: observability
>
> Since EC policies can be set per-file, the {{show files}} command should
> display if a file is an EC file, and what the EC policy is.
> EC policies can be set on a table level directory, so it would be useful if
> 'describe extended [table-name]' indicated if the table had an EC policy set
> or not.
> For partitioned tables, {{show partitions}} should list out the EC policy of
> each partition directory (we already do something similar for HDFS cacheing).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
