danhuawang commented on code in PR #9173: URL: https://github.com/apache/gravitino/pull/9173#discussion_r2596968641
########## docs/lakehouse-generic-catalog.md: ########## @@ -0,0 +1,588 @@ +--- +title: "Generic Lakehouse Catalog" +slug: /lakehouse-generic-catalog +keywords: + - lakehouse + - lance + - metadata + - generic catalog + - file system +license: "This software is licensed under the Apache License version 2." +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +## Overview + +The Generic Lakehouse Catalog is a Gravitino catalog implementation designed to seamlessly integrate with lakehouse storage systems built on file system-based architectures. This catalog enables unified metadata management for lakehouse tables stored on various storage backends, providing a consistent interface for data discovery, governance, and access control. + +### What is a Lakehouse? + +A lakehouse combines the best features of data lakes and data warehouses: + +- **Data Lake Benefits**: + - Low-cost storage for massive volumes of raw data + - Support for diverse data formats (structured, semi-structured, unstructured) + - Decoupled storage and compute for flexible scaling + +- **Data Warehouse Benefits**: + - ACID transactions for data consistency + - Schema enforcement and evolution + - High-performance analytical queries + - Time travel and versioning + +### Supported Storage Systems + +The catalog works with lakehouse systems built on top of: + +**Storage Backends:** +- **Object Stores:** Amazon S3, Azure Blob Storage, Google Cloud Storage, MinIO +- **Distributed File Systems:** HDFS, Apache Ozone +- **Local File Systems:** For development and testing + +**Lakehouse Formats:** +- **Lance** ✅ (We only support Lance format fully at present) Review Comment: I recommend we could explicitly note which Storage Backends that Lance format support. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
