imbajin commented on code in PR #3471: URL: https://github.com/apache/hugegraph/pull/3471#discussion_r4505200040
########## README.md: ########## @@ -1,414 +1,137 @@ <h1 align="center"> - <img width="720" alt="hugegraph-logo" src="https://github.com/apache/hugegraph/assets/38098239/e02ffaed-4562-486b-ba8f-e68d02bb0ea6"; style="zoom:100%;" /> + <img width="720" alt="Apache HugeGraph" src="https://github.com/apache/hugegraph/assets/38098239/e02ffaed-4562-486b-ba8f-e68d02bb0ea6"; /> </h1> -<h3 align="center">A graph database that supports more than 10 billion vertices & edges, high performance and scalability</h3> +<h3 align="center">A scalable graph database for connected data and real-time graph workloads</h3> <div align="center"> [](https://www.apache.org/licenses/LICENSE-2.0.html) -[](https://github.com/apache/hugegraph/actions/workflows/server-ci.yml) -[](https://github.com/apache/hugegraph/actions/workflows/pd-store-ci.yml) -[](https://github.com/apache/hugegraph/releases) +[](https://github.com/apache/hugegraph/actions/workflows/server-ci.yml) +[](https://github.com/apache/hugegraph/actions/workflows/pd-store-ci.yml) +[](https://github.com/apache/hugegraph/releases) [](https://artifacthub.io/packages/helm/hugegraph-helm/hugegraph-helm) -[](https://deepwiki.com/apache/hugegraph) </div> ---- +**Start here:** [Quick start](#quick-start) · [Deployment](#deployment) · [Documentation](#documentation) · [Development](#development) · [Ecosystem](#ecosystem) -**Quick Navigation:** [Architecture](#architecture) • [Quick Start](#quick-start) • [Module Map](#module-map) • [Ecosystem](#ecosystem) • [For Contributors](#contributing) • [Community](#community) +## Apache HugeGraph ---- +Apache HugeGraph is a graph database for storing, querying and operating connected data at scale. It combines schema management, indexes, graph traversals, pluggable storage and standalone or distributed deployment in one system. -## What is Apache HugeGraph? +Applications can use REST APIs, Gremlin, Cypher or client SDKs. Gremlin compatibility is provided through Apache TinkerPop; it is one access path alongside HugeGraph's native APIs and services. -[HugeGraph](https://hugegraph.apache.org/) is a fast and highly-scalable [graph database](https://en.wikipedia.org/wiki/Graph_database). -Billions of vertices and edges can be easily stored into and queried from HugeGraph due to its excellent OLTP capabilities. -HugeGraph is compliant with the [Apache TinkerPop 3](https://tinkerpop.apache.org/) framework allowing complicated graph queries to be -achieved through the powerful [Gremlin](https://tinkerpop.apache.org/gremlin.html) graph traversal language. +## Core capabilities -## Features +| Area | Capability | +|---|---| +| Data model | Property graph with explicit vertex labels, edge labels, property keys and indexes | +| Query and APIs | REST, Gremlin, Cypher, traverser APIs and language-specific clients | +| Storage | Embedded RocksDB for standalone deployments and HStore for distributed clusters | +| Operations | Authentication, metrics, tasks, backup and restore tooling, Docker and Kubernetes deployment | +| Scale | Single-node development through horizontally scalable Server, PD and Store clusters | +| Ecosystem | Data loading, visualization, graph computation, GraphRAG and AI integrations | -- **Schema Metadata Management**: VertexLabel, EdgeLabel, PropertyKey, and IndexLabel -- **Multi-type Indexes**: Exact query, range query, and complex conditions combination query -- **Plug-in Backend Store Framework**: RocksDB powers standalone deployments and HStore powers distributed clusters. See the [backend evolution guide](hugegraph-server/README.md#backend-evolution-and-compatibility) for compatibility details. -- **Big Data Integration**: Seamless integration with `Flink`/`Spark`/`HDFS` -- **Complete Graph Ecosystem**: In/out-memory Graph Computing + Graph Visualization & Tools + Graph Learning & AI -- **Dual Query Language Support**: [Gremlin](https://tinkerpop.apache.org/gremlin.html) (via [Apache TinkerPop 3](https://tinkerpop.apache.org/)) and [Cypher](https://en.wikipedia.org/wiki/Cypher_(query_language)) (OpenCypher) +## Quick start -## Ecosystem - -Complete **HugeGraph** ecosystem components: - -1. **[hugegraph-toolchain](https://github.com/apache/hugegraph-toolchain)** - Graph tools suite - - [Loader](https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-loader) - Data import tool - - [Dashboard](https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-hubble) - Web visualization platform - - [Tool](https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-tools) - Command-line utilities - - [Client](https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-client) - Java/Python client SDK - -2. **[hugegraph-computer](https://github.com/apache/hugegraph-computer)** - Integrated **graph computing** system - -3. **[hugegraph-ai](https://github.com/apache/hugegraph-ai)** - **Graph AI/LLM/Knowledge Graph** integration - -4. **[hugegraph-website](https://github.com/apache/hugegraph-doc)** - **Documentation & website** repository - -## Architecture - -HugeGraph supports both **standalone** and **distributed** deployments: - -``` - ┌─────────────────────────────────────────────────────┐ - │ Client Layer │ - │ Gremlin Console │ REST API │ Cypher │ SDK/Tools │ - └─────────────────────────┬───────────────────────────┘ - │ - ┌─────────────────────────▼───────────────────────────┐ - │ HugeGraph Server (:8080) │ - │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ - │ │ REST API │ │ Gremlin │ │ Cypher Engine │ │ - │ │(Jersey 3)│ │(TP 3.8.1)│ │ (OpenCypher) │ │ - │ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │ - │ └─────────────┼─────────────────┘ │ - │ ┌────────▼────────┐ │ - │ │ Graph Engine │ │ - │ │(hugegraph-core) │ │ - │ └────────┬────────┘ │ - └─────────────────────┼───────────────────────────────┘ - │ - ┌─────────────────┴─────────────────┐ - │ │ - ┌──────────────▼──────────────┐ ┌──────────────▼──────────────┐ - │ Standalone Mode │ │ Distributed Mode │ - │ ┌───────────────────────┐ │ │ ┌───────────────────────┐ │ - │ │ RocksDB │ │ │ │ HugeGraph-PD │ │ - │ │ (embedded) │ │ │ │ (Raft, 3-5 nodes) │ │ - │ └───────────────────────┘ │ │ │ :8620/:8686 │ │ - │ │ │ └───────────┬───────────┘ │ - │ Use Case: │ │ │ │ - │ Development/Testing │ │ ┌───────────▼───────────┐ │ - │ Single Node │ │ │ HStore │ │ - │ │ │ │ (Raft, 3+ nodes) │ │ - │ Data Scale: < 1TB │ │ │ :8520 │ │ - └─────────────────────────────┘ │ └───────────────────────┘ │ - │ │ - │ Use Case: │ - │ Production/HA/Cluster │ - │ │ - │ Data Scale: < 1000 TB │ - └─────────────────────────────┘ -``` - -See the [backend evolution guide](hugegraph-server/README.md#backend-evolution-and-compatibility) for lifecycle and historical compatibility guidance. - -To use ToplingDB with the RocksDB backend, follow the [three-step switching guide](docs/storage-lifecycle.md#storage-provider-selection-and-toplingdb). - -### Deployment Mode Comparison - -| Mode | Components | Use Case | Data Scale | High Availability | -|------|------------|----------|------------|-------------------| -| **Standalone** | Server + RocksDB | Development, Testing, Single Node | < 1TB | Basic | -| **Distributed** | Server + PD + HStore | Production, HA, Horizontal Scaling | < 1000 TB | Yes | - -### Module Overview - -| Module | Description | -|--------|-------------| -| [hugegraph-server](hugegraph-server) | Core graph engine with REST API, Gremlin/Cypher support, and pluggable backends (RocksDB default) | -| [hugegraph-pd](hugegraph-pd/README.md) | Placement Driver for distributed mode - handles meta storage, partition management and cluster scheduling | -| [hugegraph-store](hugegraph-store/README.md) | Distributed storage with Raft consensus for high availability and horizontal scaling | -| [hugegraph-commons](hugegraph-commons) | Shared utilities, RPC framework and common components | -| [hugegraph-struct](hugegraph-struct/README.md) | Shared schema, IDs, base elements, queries and encoding used by Server and Store | - -See the [1.8.0 shared-foundation migration guide](docs/shared-foundation-migration.md) for Java API changes, module ownership and coordinated upgrade requirements. - -<details> -<summary><b>📊 Click to view detailed architecture diagram (Mermaid)</b></summary> - -```mermaid -flowchart TB - CLIENTS["Client Layer<br/>Gremlin Console · REST Client · Cypher Client · SDK/Tools"] - - subgraph Server["HugeGraph Server :8080"] - API[REST API<br/>Jersey 3] - GS[Gremlin Server<br/>TinkerPop 3.8.1] - CS[Cypher Engine<br/>OpenCypher] - CORE[Graph Engine<br/>hugegraph-core] - - API --> CORE - GS --> CORE - CS --> CORE - end - - subgraph Storage["Storage Layer"] - subgraph Standalone["Standalone Mode"] - ROCKS[(RocksDB<br/>Embedded)] - end - - subgraph Distributed["Distributed Mode"] - PD_HSTORE[PD + HStore<br/>Raft Cluster] - end - end - - CLIENTS --> Server - CORE --> ROCKS - CORE --> PD_HSTORE - - style Server fill:#e1f5ff - style Distributed fill:#fff4e1 - style Standalone fill:#f0f0f0 -``` - -</details> - -## Quick Start - -<!-- TODO: update release downloads, package names, image tags and version examples after 1.8.0 is published. --> - -### 5 Minutes Quick Start +Start a standalone server for local evaluation, bind it to the loopback interface and wait for the version endpoint before continuing: ```bash -# Start HugeGraph (standalone mode) -docker run -itd --name=hugegraph -p 8080:8080 hugegraph/hugegraph:1.7.0 - -# Verify server is running -curl http://localhost:8080/versions - -# Try a Gremlin query -curl -X POST http://localhost:8080/gremlin \ - -H "Content-Type: application/json" \ - -d '{"gremlin":"g.V().limit(5)"}' +docker run -d --name hugegraph \ + -p 127.0.0.1:8080:8080 hugegraph/hugegraph + +for attempt in $(seq 1 60); do + if curl -fs http://127.0.0.1:8080/versions; then + break + fi + if [ "${attempt}" -eq 60 ]; then + docker logs hugegraph + exit 1 Review Comment: Fixed in cf29756: the timeout path now prints an explicit failure and logs, then leaves the loop without terminating an interactive shell. Both readiness and timeout paths were simulated. ########## helm/hugegraph/README.md: ########## @@ -1,427 +1,142 @@ -# HugeGraph HStore Helm Chart +# HugeGraph Helm chart -Deploys **3 PD + 3 Store + 3 Server and Hubble** by default, with authentication -and persistent storage enabled. Requires **Helm 3.10+ and Kubernetes 1.23+**. -The OCI chart is named `hugegraph-helm`; the application and default Kubernetes resource names remain -`hugegraph`. This chart targets HugeGraph 1.8.0 and later; earlier service releases -are not supported. +This chart deploys a distributed HugeGraph cluster on Kubernetes. The default topology is 3 PD, 3 Store and 3 Server replicas with Hubble, authentication and persistent storage enabled. SeaTunnel and Vermeer are optional and disabled by default. -Provide a default StorageClass or prepare matching PVs. The default installation -needs **7 volumes**: 3 × 10 GiB for PD, 3 × 50 GiB for Store, and 1 × 1 GiB -for Hubble. The chart creates PVCs; it does not create a storage provisioner. -SeaTunnel and Vermeer are **disabled by default**, including in the single-node -and cluster presets. Optional components add resources only when enabled. +**Requirements:** Helm 3.10 or later, Kubernetes 1.23 or later, and a default StorageClass or prepared persistent volumes. -## Install a published Chart +## Documentation map -For a quick trial, install the latest available Chart from GHCR: +| Goal | Document | +|---|---| +| Install the chart and choose common values | This README | +| Understand every detailed chart document | [Documentation index](docs/README.md) | +| Operate, scale, upgrade or troubleshoot a cluster | [Operations](docs/operations.md) | +| Install from OCI, mirror images or work offline | [Distribution](docs/distribution.md) | +| Deploy the RAG service | [RAG](docs/rag.md) | +| Enable data import and export | [SeaTunnel](docs/seatunnel.md) | +| Enable graph computation | [Vermeer](docs/vermeer.md) | +| Start from reviewed value files | [Examples](examples/README.md) | +| Inspect every supported value | [`values.yaml`](values.yaml) | + +The root [repository documentation index](../../docs/README.md) links this chart to the wider Server, Docker and development documentation. + +## Install a published chart ```bash kubectl config current-context kubectl get storageclass helm install hugegraph oci://ghcr.io/hugegraph/hugegraph-helm --devel \ - -n hugegraph --create-namespace --wait --timeout 15m + --namespace hugegraph --create-namespace --wait --timeout 15m ``` -> [!TIP] -> `--devel` selects the highest Chart version, including development and stable -> versions, without limiting the version series. For production or reproducible -> installs, replace it with `--version <published-version>` and review the images. - -See [distribution and offline installation](docs/distribution.md) for Docker Hub, -the single-node preset and internal OCI/HTTP repositories. Development Charts may -use `latest` images; pin component `image.digest` values when needed. -The source installation and configuration examples below assume a repository -checkout and commands run from its root. +`--devel` is convenient for evaluation because it can select development chart versions. For production or reproducible environments, use an explicit published version and pin image tags or digests after reviewing them. -The [stable bootstrap configuration](docs/operations.md#stable-bootstrap-topology) -always records initialization inputs separately from node replicas. Use a PD -image with separate bootstrap and runtime quorum checks. PD membership changes, -Store admission and safe retirement remain manual operations. Growing from one -Store to three does not change existing data from RF=1 to RF=3; replica policy -is configured independently with `pd.partition.defaultShardCount`. +## Install from source -## Install from source and open Hubble +Run from the repository root: ```bash -helm install hugegraph ./helm/hugegraph -n hugegraph \ - --create-namespace --wait --timeout 15m +helm install hugegraph ./helm/hugegraph \ + --namespace hugegraph --create-namespace --wait --timeout 15m +``` -# Find assigned ports and a reachable node IP -kubectl -n hugegraph get svc hugegraph-server hugegraph-hubble -kubectl get nodes -o wide +Check the deployed services and read the initial administrator password: -# Read the initial admin password +```bash +kubectl -n hugegraph get pods,svc kubectl -n hugegraph get secret hugegraph-admin \ -o jsonpath='{.data.password}' | base64 --decode ``` -Open **`http://<node-ip>:<hubble-node-port>`** and log in as `admin` with that -password. Server API uses `http://<node-ip>:<server-node-port>`. -HTTP works directly; a hostname, certificate or extra acknowledgement is not -required. PD remains internal. - -If node access is unavailable, use port-forward: +Use the assigned Hubble service address, or forward it locally: ```bash kubectl -n hugegraph port-forward svc/hugegraph-hubble 8088:8088 -# Open http://127.0.0.1:8088 ``` -Commands assume release/namespace `hugegraph`. For another release, use the names -shown by `kubectl get svc,secret -n <namespace>`. Default workloads use -`<fullname>-<component>`; generated Secrets use `<release>-admin`, -`<release>-auth-token`, and `<release>-pd-auth`. - -## Common configurations +Then open `http://127.0.0.1:8088` and sign in as `admin`. -### API only +## Choose a topology -Hubble is optional. Current Hubble login requires Server authentication. +| Configuration | Purpose | +|---|---| +| Default `values.yaml` | 3 PD, 3 Store, 3 Server and Hubble | +| `values-single.yaml` | Small 1 PD, 1 Store, 1 Server environment for evaluation | +| `values-cluster.yaml` | Cluster scheduling, anti-affinity and resource starting points | +| `values-strict.yaml` | Private service defaults and NetworkPolicy-oriented deployment | -```bash -# Authenticated API, without the UI -helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ - --set hubble.enabled=false - -# Also disable API authentication, if wanted -# Add: --set server.auth.enabled=false -``` +Topology presets do not automatically enable SeaTunnel or Vermeer. Enable optional components explicitly and choose their own standalone or cluster mode. -Inactive credential values may remain in the values file when auth is disabled. +## Common settings -### Optional components +| Goal | Value | +|---|---| +| Disable Hubble | `hubble.enabled=false` | +| Disable Server authentication for a trusted local environment | `server.auth.enabled=false` | +| Disable persistent storage for a disposable new installation | `persistence.enabled=false` | +| Select a StorageClass | `pd.storage.storageClassName`, `store.storage.storageClassName`, `hubble.persistence.storageClassName` | +| Use an internal image registry | `global.imageRegistry` | +| Enable SeaTunnel | `seatunnel.enabled=true` | +| Enable Vermeer | `vermeer.enabled=true` | +| Change Server, PD or Store replicas | `server.replicas`, `pd.replicas`, `store.replicas` | +| Choose public or private service entry | Component `service.type` or chart `mode` | Review Comment: Fixed in cf29756: restored the two linked Helm headings and their service-identity/inheritance guidance. The Seatunnel, Vermeer and Docker anchors were checked against the chart configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
