Richard-Durkee opened a new pull request, #2581:
URL: https://github.com/apache/datafusion-ballista/pull/2581
## Which issue does this PR close?
Follow-up to #2530.
## Rationale for this change
#2530 added executor memory pool metrics, but they only show up in the
executor's own logs. The scheduler can't see how much memory each executor is
using, which makes them hard to monitor and rules out using them for scheduling
later.
## What changes are included in this PR?
Push-mode executors now send three more metrics in their heartbeats, which
appear in `GET /api/executors` and `GET /api/executor/{executor_id}`:
- `memory_pool_reserved`: bytes currently reserved across the running tasks'
pools
- `memory_pool_size`: the executor's total pool budget if bounded
- `allocated_memory`: bytes outstanding from the accounting allocator (only
sent when a source is set; the standalone binary sets it to
`alloc_accounting::current_balance`)
They're process metrics, so they're sent under the `proc` (default) and
`all` collection policies. Only executors started with
`start_executor_process_with_memory_metrics` send them.
To support `allocated_memory`, `ExecutorMemoryMetrics` gets two public
methods, `set_allocated_bytes_source` and `allocated_bytes`.
Pull-mode executors don't send heartbeat metrics, so they report none of
these. I'd like to look at that in a follow-up.
## Are there any user-facing changes?
The new metrics appear in the executors REST API. No breaking changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
