This is an automated email from the ASF dual-hosted git repository.
SbloodyS pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
The following commit(s) were added to refs/heads/dev by this push:
new 27784c62e7 [Chore][Docs] Add AGENT.md (#18271)
27784c62e7 is described below
commit 27784c62e7b45def6d5c1ca0cda65d4a6426b244
Author: Wenjun Ruan <[email protected]>
AuthorDate: Fri May 15 17:48:57 2026 +0800
[Chore][Docs] Add AGENT.md (#18271)
Co-authored-by: xiangzihao <[email protected]>
---
.gitignore | 1 +
AGENT.md | 161 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 162 insertions(+)
diff --git a/.gitignore b/.gitignore
index ff6e43c416..556f994dd7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -60,3 +60,4 @@ ds_schema_check_test
docs/superpowers/
.claude/worktrees
CLAUDE.local.md
+AGENT.local.md
diff --git a/AGENT.md b/AGENT.md
new file mode 100644
index 0000000000..ae7a549306
--- /dev/null
+++ b/AGENT.md
@@ -0,0 +1,161 @@
+# AGENT.md - Apache DolphinScheduler
+
+Apache DolphinScheduler is a distributed, visual DAG workflow-scheduling
platform. This is the monorepo: backend servers (master / worker / api /
alert), a Vue 3 frontend, plugin families for tasks / datasources / storage /
alerting / scheduling, and the release tooling.
+
+**This file is an agent-facing project index, adapted from `CLAUDE.md`.**
Module-specific details currently live in each module's `CLAUDE.md`; use those
files as the source of truth and do not duplicate module contents here.
+
+---
+
+## Tech stack (project-wide)
+
+- **Java 1.8** (do not assume 11+ APIs; `dolphinscheduler-api-test` is the
only Java 11 island).
+- **Spring Boot 2.6.1** across servers, **Jetty** (Tomcat is excluded
transitively).
+- **MyBatis-Plus** for ORM; **HikariCP** for the metadata DB pool, **Druid**
inside user-facing datasource plugins.
+- **Quartz** for cron scheduling (via `scheduler-plugin`).
+- **Netty / gRPC** for inter-server RPC (see `extract-base`).
+- **Vue 3 + Vite + TypeScript + Naive UI** for the frontend.
+- **Maven** multi-module reactor (26 modules in root `pom.xml` + 2 test
modules).
+- **Zookeeper 3.8** by default for the registry (Etcd and JDBC also supported).
+
+## Runnable services
+
+A production deployment runs **four independent services** (plus an external
registry and metadata DB). A fifth entry point, `StandaloneServer`, embeds all
four in one JVM for development.
+
+| Service | Module | Main class | Default ports |
+|---------|--------|------------|---------------|
+| **API** | [`dolphinscheduler-api`](dolphinscheduler-api/CLAUDE.md) |
`org.apache.dolphinscheduler.api.ApiApplicationServer` | `12345` (HTTP / UI +
REST) |
+| **Master** | [`dolphinscheduler-master`](dolphinscheduler-master/CLAUDE.md)
| `org.apache.dolphinscheduler.server.master.MasterServer` | `5679` (RPC) |
+| **Worker** | [`dolphinscheduler-worker`](dolphinscheduler-worker/CLAUDE.md)
| `org.apache.dolphinscheduler.server.worker.WorkerServer` | `1235` (RPC) |
+| **Alert** | [`dolphinscheduler-alert`](dolphinscheduler-alert/CLAUDE.md) (to
`-alert-server`) | `org.apache.dolphinscheduler.alert.AlertServer` | `50053`
(HTTP), `50052` (RPC) |
+| Standalone (dev only) |
[`dolphinscheduler-standalone-server`](dolphinscheduler-standalone-server/CLAUDE.md)
| `org.apache.dolphinscheduler.StandaloneServer` | `12345` + `50052` (API +
alert; master/worker use in-JVM calls) |
+
+Every service is a `@SpringBootApplication` on Jetty and implements
`IStoppable`. Scale Master / Worker / Alert horizontally; coordination happens
via the registry (Zookeeper by default). API is stateless and also scales
horizontally behind a load balancer.
+
+Ports are overridable via `server.port` / service-specific keys in each
service's `application.yaml`.
+
+## Build & run
+
+```bash
+# Full build (release profile; produces dist tarball)
+./mvnw clean install -Prelease
+
+# Zookeeper 3.4 legacy
+./mvnw clean install -Prelease -Dzk-3.4
+
+# Skip UI build (faster iteration on backend only)
+./mvnw -pl '!dolphinscheduler-ui' clean install
+
+# Build one module (+ its required siblings)
+./mvnw -pl dolphinscheduler-master -am clean install
+
+# Format (Spotless is configured)
+./mvnw spotless:apply
+
+# Standalone server (after building)
+cd dolphinscheduler-standalone-server/target && ./bin/start.sh
+```
+
+Binary artifact:
`dolphinscheduler-dist/target/apache-dolphinscheduler-*-bin.tar.gz`.
+
+## Test
+
+```bash
+# Unit tests for one module
+./mvnw -pl dolphinscheduler-master test
+
+# API integration tests (separate reactor, requires Docker)
+mvn -pl dolphinscheduler-api-test/dolphinscheduler-api-test-case test
+
+# E2E browser tests (Selenium + Docker)
+mvn -pl dolphinscheduler-e2e/dolphinscheduler-e2e-case test
+
+# Apple Silicon: add -Dm1_chip=true to the Docker-driven suites
+```
+
+---
+
+## Module index
+
+Click into a module's `CLAUDE.md` for details. Each description is one line
here on purpose.
+
+### Core execution
+
+- [`dolphinscheduler-master`](dolphinscheduler-master/CLAUDE.md) - workflow
orchestration engine; consumes `Command`s, runs the DAG state machine,
dispatches to workers.
+- [`dolphinscheduler-worker`](dolphinscheduler-worker/CLAUDE.md) - runs
physical tasks dispatched from master; hosts task plugins.
+- [`dolphinscheduler-task-executor`](dolphinscheduler-task-executor/CLAUDE.md)
- reusable task-lifecycle framework embedded by the worker.
+- [`dolphinscheduler-alert`](dolphinscheduler-alert/CLAUDE.md) - alert server
+ channel plugins (email, Feishu, DingTalk, ...).
+
+### API layer
+
+- [`dolphinscheduler-api`](dolphinscheduler-api/CLAUDE.md) - REST API server
(entry point for UI, Python SDK, external clients).
+- [`dolphinscheduler-api-test`](dolphinscheduler-api-test/CLAUDE.md) -
integration tests against the REST API (Docker Compose + Testcontainers).
+-
[`dolphinscheduler-authentication`](dolphinscheduler-authentication/CLAUDE.md)
- Actuator-endpoint auth + AWS credential helpers (NOT the main login path).
+
+### Shared libraries
+
+- [`dolphinscheduler-common`](dolphinscheduler-common/CLAUDE.md) - foundation
utilities (everything depends on this).
+- [`dolphinscheduler-dao`](dolphinscheduler-dao/CLAUDE.md) - MyBatis DAO layer
+ SQL migration scripts.
+- [`dolphinscheduler-service`](dolphinscheduler-service/CLAUDE.md) - business
logic between DAO and the servers.
+- [`dolphinscheduler-spi`](dolphinscheduler-spi/CLAUDE.md) - Service-Provider
Interface root (every plugin depends on this).
+- [`dolphinscheduler-extract`](dolphinscheduler-extract/CLAUDE.md) - RPC
interface contracts between servers.
+- [`dolphinscheduler-eventbus`](dolphinscheduler-eventbus/CLAUDE.md) -
in-process event-bus abstractions.
+- [`dolphinscheduler-registry`](dolphinscheduler-registry/CLAUDE.md) -
pluggable registry (Zookeeper / Etcd / JDBC).
+- [`dolphinscheduler-meter`](dolphinscheduler-meter/CLAUDE.md) - metrics
(Prometheus) + server load-protection primitives.
+
+### Plugin families
+
+- [`dolphinscheduler-task-plugin`](dolphinscheduler-task-plugin/CLAUDE.md) -
task-type plugins (shell, SQL, Spark, Flink, K8s, EMR, ...). 33 concrete
plugins.
+-
[`dolphinscheduler-datasource-plugin`](dolphinscheduler-datasource-plugin/CLAUDE.md)
- user-facing datasource plugins (MySQL, Hive, Trino, Snowflake, ...). 28
concrete plugins.
+-
[`dolphinscheduler-storage-plugin`](dolphinscheduler-storage-plugin/CLAUDE.md)
- resource storage (S3, HDFS, OSS, GCS, ABS, OBS, COS).
+-
[`dolphinscheduler-scheduler-plugin`](dolphinscheduler-scheduler-plugin/CLAUDE.md)
- cron scheduler (Quartz today).
+- [`dolphinscheduler-dao-plugin`](dolphinscheduler-dao-plugin/CLAUDE.md) -
metadata-DB dialect support (MySQL / PostgreSQL / H2).
+
+### Build, ops, tools
+
+- [`dolphinscheduler-bom`](dolphinscheduler-bom/CLAUDE.md) - Maven BOM;
central dependency version pinning.
+- [`dolphinscheduler-dist`](dolphinscheduler-dist/CLAUDE.md) - assembles the
release tarball + Docker images.
+-
[`dolphinscheduler-standalone-server`](dolphinscheduler-standalone-server/CLAUDE.md)
- all-in-one JVM with H2 (dev / smoke tests).
+- [`dolphinscheduler-tools`](dolphinscheduler-tools/CLAUDE.md) - CLIs for
schema upgrade + resource / lineage migration.
+- [`dolphinscheduler-microbench`](dolphinscheduler-microbench/CLAUDE.md) - JMH
micro-benchmarks.
+- [`dolphinscheduler-yarn-aop`](dolphinscheduler-yarn-aop/CLAUDE.md) - AspectJ
weaver capturing YARN ApplicationIds.
+
+### Frontend & E2E
+
+- [`dolphinscheduler-ui`](dolphinscheduler-ui/CLAUDE.md) - Vue 3 frontend.
+- [`dolphinscheduler-e2e`](dolphinscheduler-e2e/CLAUDE.md) - Selenium browser
tests.
+
+---
+
+## Architecture overview
+
+A **user** hits the UI, which calls the API server. The API server writes to
the **metadata DB** and, for runtime operations (start / kill / pause
workflow), talks to the **master** over RPC. The master consumes `t_ds_command`
rows, runs the workflow state machine, and dispatches tasks to **workers**.
Workers execute task plugins (shell, SQL, Spark, ...) and stream lifecycle
events back to master. Failures and SLA breaches flow to the **alert server**,
which fans out through alert plugin [...]
+
+## Where things live (quick lookup)
+
+| Looking for... | Start here |
+|----------------|------------|
+| A REST endpoint | `dolphinscheduler-api/src/main/java/.../api/controller/` |
+| Workflow execution logic |
`dolphinscheduler-master/src/main/java/.../server/master/engine/` |
+| Task execution logic | `dolphinscheduler-worker` + the specific
`task-plugin/<type>` |
+| How "X" is stored | `dolphinscheduler-dao/src/main/java/.../dao/entity/` |
+| SQL schema / upgrade | `dolphinscheduler-dao/src/main/resources/sql/` |
+| RPC contract between servers |
`dolphinscheduler-extract/dolphinscheduler-extract-<role>` |
+| UI page source | `dolphinscheduler-ui/src/views/<feature>/` |
+| API call in the UI | `dolphinscheduler-ui/src/service/modules/<resource>.ts`
|
+| Version of a dependency | `dolphinscheduler-bom/pom.xml` |
+
+## Project-wide conventions
+
+- **Formatting**: Run `./mvnw spotless:apply` before every commit/push.
Spotless covers Java sources, `pom.xml`, and Markdown files; CI runs `./mvnw
spotless:check` and will fail PRs that are not formatted. Java imports are
ordered; license headers are enforced.
+- **Commit style**: `[Type-ISSUE_ID][Scope] Subject`, e.g.
`[Fix-18168][Worker] ...`. All types except `Chore` require an issue ID. See
[commit-message.md](docs/docs/en/contribute/join/commit-message.md) for the
full convention.
+- **Branching**: `dev` is the main integration branch (not `main`/`master`).
+- **PRs must link a GitHub issue** and keep their scope tight: one module /
one concern. For `Chore` commits, no issue ID is required by the commit
convention.
+- **Do not break wire / DB compatibility** silently. Changes to `extract-*`
RPC interfaces, `dao` entities, enum values, and `spi.DbType` ripple to
deployed clusters mid-upgrade.
+- **Only one registry / storage / DB dialect is active at runtime**. Code
paths that check "which one" belong inside the plugin SPI, not sprinkled
through services.
+
+## External references
+
+- Release docs (version-specific):
https://dolphinscheduler.apache.org/en-us/docs
+- GitHub issues: https://github.com/apache/dolphinscheduler/issues
+- Python SDK: https://dolphinscheduler.apache.org/python/main/index.html
+- Contribution guide:
[`docs/docs/en/contribute/join/contribute.md`](docs/docs/en/contribute/join/contribute.md)
