Siddharth Krishna created YUNIKORN-3347:
-------------------------------------------
Summary: Improve scheduler config checksum generation and checks
Key: YUNIKORN-3347
URL: https://issues.apache.org/jira/browse/YUNIKORN-3347
Project: Apache YuniKorn
Issue Type: Bug
Components: core - scheduler, shim - kubernetes
Reporter: Siddharth Krishna
Assignee: Siddharth Krishna
Fix For: 1.9.0
The scheduler configuration checksum is not handled robustly. Need to improve
it as follows:
* Fix checksum detection in GetConfigurationString: detect the checksum line at
the start and the end of the serialised config, and stop scanning the entire
config to locate it. The current implementation walks and rebuilds the whole
string, which is inefficient for large configs (the ones large enough to be
gzip-compressed).
* When loading a config, override an incorrect or missing checksum in the
in-memory config with the correct calculated value, and log the action taken so
a bad checksum can be traced.
* Skip processing a config update when the checksum has not changed (i.e. the
config content is effectively unchanged even if the ConfigMap was updated for
another reason). Verify this path handles a not-yet-set previous config.
* Make the calculated checksum reproducible from the config object: stripping
the checksum line from a serialised config must equal the config serialised
with no checksum set, so consumers can recompute the checksum without
re-running the load code.
* Expose the checksum and the checksum-comparison result in the
ValidateConfResponse returned by the /ws/v1/validate-conf REST endpoint.
Note: This changes the calculated checksum value for existing serialised
configs, causing a one time config reload on upgrade. This must be called out
in the 1.9 release notes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
