vvysotskyi commented on a change in pull request #2030: Update docs for Metastore to point that all format plugins are supported URL: https://github.com/apache/drill/pull/2030#discussion_r395790624
########## File path: _docs/performance-tuning/drill-metastore/010-using-drill-metastore.md ########## @@ -103,20 +105,50 @@ Schema information and summary statistics also computed and stored for table seg The detailed metadata schema is described [here](https://github.com/apache/drill/tree/master/metastore/metastore-api#metastore-tables). You can try out the metadata to get a sense of what is available, by using the - [Inspect the Metastore using `INFORMATION_SCHEMA` tables]({{site.baseurl}}/docs/using-drill-metastore/#inspect-the-metastore-using-information_schema-tables) tutorial. + [Inspect the Metastore using `INFORMATION_SCHEMA` tables](#inspect-the-metastore-using-information_schema-tables) tutorial. Every table described by the Metastore may be a bare file or one or more files that reside in one or more directories. If a table consists of a single directory or file, then it is non-partitioned. The single directory can contain any number of files. Larger tables tend to have subdirectories. Each subdirectory is a partition and such a table are called "partitioned". -Please refer to [Exposing Drill Metastore metadata through `INFORMATION_SCHEMA` tables]({{site.baseurl}}/docs/using-drill-metastore/#exposing-drill-metastore-metadata-through-information_schema-tables) +Please refer to [Exposing Drill Metastore metadata through `INFORMATION_SCHEMA` tables](#exposing-drill-metastore-metadata-through-information_schema-tables) for information, how to query partitions and segments metadata. A traditional database divides tables into schemas and tables. Drill can connect to any number of data sources, each of which may have its own schema. As a result, the Metastore labels tables with a combination of (plugin configuration name, workspace name, table name). Note that if before renaming any of these items, you must delete table's Metadata entry and recreate it after renaming. +### Using schema provisioning feature with Drill Metastore + +The Drill Metastore holds both schema and statistics information for a table. The `ANALYZE` command can infer the table + schema for well-defined tables (such as many Parquet tables). Some tables are too complex or variable for Drill's + schema inference to work well. For example, JSON tables often omit fields or have long runs of nulls so that Drill + cannot determine column types. In these cases, you can specify the correct schema based on your knowledge of the + table's structure. You specify a schema in the `ANALYZE` command using the + [Schema provisioning]({{site.baseurl}}/docs/plugin-configuration-basics/#specifying-the-schema-as-table-function-parameter) syntax. + +Please refer to [Provisioning schema for Drill Metastore](#provisioning-schema-for-drill-metastore) for examples of usage. + +### Schema priority + +Drill allows the following ways for providing table schema: + - providing schema with table function: + - specifying inline schema; + - specifying path to the schema file; + - using schema file in table root directory; + - using schema from Drill Metastore. + +The highest priority has schema provided in table function. + +Second priority has schema file (if `store.table.use_schema_file` is enabled). + +If neither of the above schema sources wasn't specified, schema from Drill Metastore will be used. + +Regardless of the source of the schema, it will be used and handled in the same way. + +Table metadata from Drill Metastore will be used if it is available regardless of the schema source. + Review comment: Thanks, replaced. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
