wu-sheng opened a new pull request, #13703:
URL: https://github.com/apache/skywalking/pull/13703
### Replace DataCarrier with BatchQueue; name all thread pools
### New feature: `library-batch-queue` module
- [x] Update the documentation to include this new feature.
- [x] Tests(including UT, IT, E2E) are added to verify the new feature.
A partitioned, self-draining queue with type-based dispatch, adaptive
partitioning, idle backoff,
and throughput-weighted drain rebalancing. Designed to replace DataCarrier
in high-fan-out scenarios.
### Improve the performance of metrics aggregation and persistence pipeline
Thread count reduction from 36 to 15 on an 8-core machine (gRPC remote
client excluded — unchanged 1 thread per peer).
All metric types (OAL + MAL) now share unified queues instead of separate
OAL/MAL pools.
| Queue | Old threads | New threads | New policy |
|-------|-------------|-------------|------------|
| L1 Aggregation (OAL+MAL unified) | 26 | 8 | `cpuCores(1.0)` |
| L2 Persistence (OAL+MAL unified) | 3 | 3 | `cpuCoresWithBase(1, 0.25)` |
| TopN Persistence | 4 | 1 | `fixed(1)` |
| Exporters (gRPC/Kafka) | 3 | 3 | `fixed(1)` each |
| **Total** | **36** | **15** | |
### Add named ThreadFactory to all anonymous Executors pool threads
Replace default `pool-N-thread-M` naming with descriptive thread names
across all `Executors.newXxx()` calls
for easier thread dump analysis. Complete OAP server thread inventory:
| Catalog | Thread Name | Count (8-core) | Policy | Partitions |
|---------|-------------|----------------|--------|------------|
| Data Pipeline | `BatchQueue-METRICS_L1_AGGREGATION-N` | 8 |
`cpuCores(1.0)` | ~460 adaptive |
| Data Pipeline | `BatchQueue-METRICS_L2_PERSISTENCE-N` | 3 |
`cpuCoresWithBase(1, 0.25)` | ~460 adaptive |
| Data Pipeline | `BatchQueue-TOPN_PERSISTENCE-N` | 1 | `fixed(1)` | ~4
adaptive |
| Data Pipeline | `BatchQueue-GRPC_REMOTE_{host}_{port}-N` | 1 per peer |
`fixed(1)` | `fixed(1)` |
| Data Pipeline | `BatchQueue-EXPORTER_GRPC_METRICS-N` | 1 | `fixed(1)` |
`fixed(1)` |
| Data Pipeline | `BatchQueue-EXPORTER_KAFKA_TRACE-N` | 1 | `fixed(1)` |
`fixed(1)` |
| Data Pipeline | `BatchQueue-EXPORTER_KAFKA_LOG-N` | 1 | `fixed(1)` |
`fixed(1)` |
| Data Pipeline | `BatchQueue-JDBC_ASYNC_BATCH_PERSISTENT-N` | 4
(configurable) | `fixed(N)` | `fixed(N)` |
| Scheduler | `RemoteClientManager` | 1 | scheduled | — |
| Scheduler | `PersistenceTimer` | 1 | scheduled | — |
| Scheduler | `PersistenceTimer-prepare-N` | 2 (configurable) | fixed pool |
— |
| Scheduler | `DataTTLKeeper` | 1 | scheduled | — |
| Scheduler | `CacheUpdateTimer` | 1 | scheduled | — |
| Scheduler | `HierarchyAutoMatching` | 1 | scheduled | — |
| Scheduler | `WatermarkWatcher` | 1 | scheduled | — |
| Scheduler | `AlarmCore` | 1 | scheduled | — |
| Scheduler | `HealthChecker` | 1 | scheduled | — |
| Scheduler | `EndpointUriRecognition` | 1 (conditional) | scheduled | — |
| Scheduler | `FileChangeMonitor` | 1 | scheduled | — |
| Scheduler | `BanyanDB-ChannelManager` | 1 | scheduled | — |
| Scheduler | `GRPCClient-HealthCheck-{host}:{port}` | 1 per client |
scheduled | — |
| Scheduler | `EBPFProfiling-N` | configurable | fixed pool | — |
- [x] Update the [`CHANGES`
log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
