[
https://issues.apache.org/jira/browse/DRILL-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16971694#comment-16971694
]
ASF GitHub Bot commented on DRILL-7273:
---------------------------------------
arina-ielchiieva commented on pull request #1886: DRILL-7273: Introduce
operators for handling metadata
URL: https://github.com/apache/drill/pull/1886#discussion_r344774068
##########
File path: docs/dev/MetastoreAnalyze.md
##########
@@ -60,14 +64,57 @@ The following tables are populated with table metadata
from the metastore:
- `LOCATION` - segment location, `null` for partitions:
`/tmp/nation/part_int=3`
- `LAST_MODIFIED_TIME` - last modification time
-# Metastore-related options
+# Metastore related options
- - `metastore.enabled` Enables Drill Metastore usage to be able to store table
metadata during `ANALYZE TABLE` commands
+ - `metastore.enabled` - enables Drill Metastore usage to be able to store
table metadata during `ANALYZE TABLE` commands
execution and to be able to read table metadata during regular queries
execution or when querying some `INFORMATION_SCHEMA` tables.
- - `metastore.metadata.store.depth_level` Specifies maximum level depth for
collecting metadata.
- - `metastore.metadata.use_schema` Enables schema usage, stored to the
Metastore.
- - `metastore.metadata.use_statistics` Enables statistics usage, stored in the
Metastore, at the planning stage.
- - `metastore.metadata.fallback_to_file_metadata` Allows using file metadata
cache for the case when required metadata is absent in the Metastore.
- - `metastore.retrieval.retry_attempts` Specifies the number of attempts for
retrying query planning after detecting that query metadata is changed.
+ - `metastore.metadata.store.depth_level` - specifies maximum level depth for
collecting metadata.
+ Possible values : `TABLE`, `SEGMENT`, `PARTITION`, `FILE`, `ROW_GROUP`, `ALL`.
+ - `metastore.metadata.use_schema` - enables schema usage, stored to the
Metastore.
+ - `metastore.metadata.use_statistics` - enables statistics usage, stored in
the Metastore, at the planning stage.
+ - `metastore.metadata.fallback_to_file_metadata` - allows using file metadata
cache for the case when required metadata is absent in the Metastore.
+ - `metastore.retrieval.retry_attempts` - specifies the number of attempts for
retrying query planning after detecting that query metadata is changed.
If the number of retries was exceeded, query will be planned without metadata
information from the Metastore.
-
\ No newline at end of file
+
+# Analyze operators description
+
+Entry point for `ANALYZE` command is `MetastoreAnalyzeTableHandler` class. It
creates plan which includes some
+Metastore specific operators for collecting metadata.
+
+`MetastoreAnalyzeTableHandler` uses `AnalyzeInfoProvider` for providing the
information
+required for building a suitable plan for collecting metadata.
+Every group scan should provide each `AnalyzeInfoProvider` implementation and
annotate implementation class with
Review comment:
... should provide corresponding `AnalyzeInfoProvider` implementation class.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Create operator for handling metadata
> -------------------------------------
>
> Key: DRILL-7273
> URL: https://issues.apache.org/jira/browse/DRILL-7273
> Project: Apache Drill
> Issue Type: Sub-task
> Reporter: Arina Ielchiieva
> Assignee: Vova Vysotskyi
> Priority: Major
> Labels: doc-impacting
> Fix For: 1.17.0
>
>
> As described in the design document for [File metastore table
> metadata|https://docs.google.com/document/d/14pSIzKqDltjLEEpEebwmKnsDPxyS_6jGrPOjXu6M_NM/edit#heading=h.j079wdhtehuk]
> it is required to create an operator for handling metadata to be able to do
> incremental analyze.
> Add options:
> metastore.enabled
> metastore.metadata.store.depth_level
> Statement:
> ANALYZE TABLE REFRESH METADATA ...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
