vvysotskyi commented on a change in pull request #1986: Additional changes for
Drill Metastore docs
URL: https://github.com/apache/drill/pull/1986#discussion_r387075673
##########
File path:
_docs/performance-tuning/drill-metastore/030-drill-iceberg-metastore.md
##########
@@ -16,21 +16,25 @@ If the file system does not support atomic rename, it
could lead to inconsistenc
### Iceberg Tables Location
-Iceberg tables will reside on the file system in the location based on
-Iceberg Metastore base location `drill.metastore.iceberg.location.base_path`
and component specific location.
-If Iceberg Metastore base location is `/drill/metastore/iceberg`
-and tables component location is `tables`. Iceberg table for tables component
-will be located in `/drill/metastore/iceberg/tables` folder.
+Iceberg is essentially a file system within a file. The Iceberg table is
stored in a file system. In the tutorial, we
+ stored the metastore in your local file system. Drill is a distributed query
engine, so production deployments store
+ the metastore in your DFS such as HDFS.
-Metastore metadata will be stored inside Iceberg table location provided
-in the configuration file. Drill table metadata location will be constructed
-based on specific component storage keys. For example, for `tables` component,
-storage keys are storage plugin, workspace and table name: unique table
identifier in Drill.
+Iceberg Metastore configuration can be set in `drill-metastore-distrib.conf`
or `drill-metastore-override.conf` files.
+The default configuration is indicated in `drill-metastore-module.conf` file.
-Assume Iceberg table location is `/drill/metastore/iceberg/tables`, metadata
for the table
-`dfs.tmp.nation` will be stored in the
`/drill/metastore/iceberg/tables/dfs/tmp/nation` folder.
+Within the Metastore directory, the Metastore stores the data for each table
as a set of records within a single Iceberg table.
-Example of base Metastore configuration file `drill-metastore-override.conf`,
where Iceberg tables will be stored in
+Iceberg tables will reside on the file system in the location based on Iceberg
Metastore base location
+ `drill.metastore.iceberg.location.base_path` configuration property and
component specific location.
+Table metadata resides in the
`${drill.metastore.iceberg.location.base_path}/tables` directory.
+
+Within the tables directory, there is one file per analyzed table in the form
`my-dfs/my-workspace/table-name/UUID.parquet`
Review comment:
It's a parquet file created by Iceberg, where metadata actually stored.
Removed this sentence to avoid confusion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
