MehulBatra commented on code in PR #3424:
URL: https://github.com/apache/fluss/pull/3424#discussion_r3458767399
##########
website/docs/streaming-lakehouse/integrate-data-lakes/catalogs/hive.md:
##########
@@ -0,0 +1,432 @@
+---
+title: Hive Metastore
+sidebar_position: 3
+---
+
+# Hive Metastore
+
+## Introduction
+
+The **Hive Metastore (HMS)** is a central metadata repository commonly used in
Apache Hadoop and other big data ecosystems to store schema and metadata
information for tables. Apache Iceberg provides native integration with Hive
Metastore, storing Iceberg table names and metadata locations directly within
HMS.
+
+This guide explains how to configure Fluss to use Hive Metastore as its
Iceberg catalog. For general Iceberg integration details (table mapping, data
types, limitations), see [Iceberg](../formats/iceberg.md).
+
+## How It Works
+
+When Fluss is configured with Hive Metastore as its Iceberg catalog:
+
+1. **Data ingestion**: Applications write data to Fluss tables using the Fluss
client (Java/Python), Flink SQL, or any Kafka-compatible producer. Fluss stores
this data in its real-time log.
+2. **Tiering to Iceberg**: A separate Flink job (the [tiering
service](maintenance/tiered-storage/lakehouse-storage.md#start-the-datalake-tiering-service))
periodically reads accumulated data from Fluss, converts it to Parquet format,
writes the files to HDFS (or S3/OSS), and commits an Iceberg snapshot to the
Hive Metastore.
+3. **Query via Spark/Trino/Flink**: Any Iceberg-compatible engine configured
with Hive catalog can discover and query the tiered tables through HMS.
+
+```
+┌─ Data Sources ─────────┐ ┌─ Fluss Cluster ──────┐ ┌─ Hadoop /
On-Prem ────────┐
+│ │ │ │ │
│
+│ App (Fluss client) ─┼────▶│ Coordinator │ │ Hive Metastore
│
+│ Flink CDC job ─┼────▶│ TabletServer(s) │ │ (table
metadata) │
+│ Kafka producer ─┼────▶│ ZooKeeper │ │
│
+│ │ │ │ │ HDFS / S3 /
OSS │
+└─────────────────────────┘ └──────────┬───────────┘ │ (Parquet data
files) │
+ │ │
│
Review Comment:
Can you create a proper diagram, put an image using draw.io or excali draw.
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
