vaijosh opened a new issue, #3079:
URL: https://github.com/apache/hugegraph/issues/3079
### Feature Description (功能描述)
**Motivation**
n modern cloud-native environments, compute instances are
ephemeral—frequently created and destroyed based on autoscaling policies.
HugeGraph currently relies on local disk storage (RocksDB), which creates
significant risks for data durability and complicates operational lifecycle
management in these dynamic environments. We propose integrating a pluggable
cloud-native storage tier to offload data to object storage, effectively
decoupling compute from storage to ensure data persistence, reliability, and
operational flexibility.
**High-Level Requirements:**
- Decoupled Storage: Enable offloading and retrieving HugeGraph data from
cloud object storage (e.g., S3, GCS, Azure Blob).
- Extensible Architecture: Implement a provider-agnostic framework using an
SPI (Service Provider Interface) mechanism. This design ensures the core
storage engine remains decoupled, allowing the community to add new storage
backends without requiring modifications to the core codebase.
- Initial Cloud Support: Provide out-of-the-box support for few popular
cloud object storage services e.g. AWS S3, Azure Data Lake Storage Gen2. Google
Cloud Storage (GCS) etc.
**Functional Requirements:**
- Automated Data Offloading: Mechanism to persist local SST data to
configured cloud object storage.
- Intelligent Hydration: Ability to recover/hydrate data from cloud storage
to local compute nodes upon startup or during read-miss scenarios.
- Configuration: Standardized property-based configuration for buckets,
endpoints, and caching strategies.
**Non-Functional Requirements:**
- Performance: Minimize read/write latency overhead via local caching layers.
- Scalability: Support elastic scaling of compute nodes without requiring
complex manual data migration.
- Maintainability: Enable the addition of new storage backends via an
SPI-based plugin model to minimize technical debt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
