Serge Huber created UNOMI-957:
---------------------------------
Summary: Integration test runs lack clear memory and resource
visibility
Key: UNOMI-957
URL: https://issues.apache.org/jira/browse/UNOMI-957
Project: Apache Unomi
Issue Type: Sub-task
Components: unomi(-core)
Affects Versions: unomi-3.1.0
Reporter: Serge Huber
Assignee: Serge Huber
Fix For: unomi-3.1.0
When our full integration test suite runs, we cannot easily see how much memory
the system is using or whether the run is running out of resources. That makes
it hard to tell whether failures are caused by real bugs or by the machine
being under too much strain—especially on shared CI servers.
h4. What is going wrong today
Running the full integration test suite is resource-heavy: the application
server, search engine, and supporting services all run together for a long
time. When a run fails—or behaves inconsistently—we often cannot tell whether
memory or overall system load played a role.
h4. Impact
* Hard to diagnose failures: A test failure may be a product bug, a timing
issue, or the system running low on memory. Without recorded resource usage
during the run, we guess instead of knowing.
* No baseline for sizing: We set memory limits for key components, but we lack
reliable measurements from real runs (local machines and CI) to know if those
limits are too high, too low, or about right.
* Weak run history: When we archive or compare runs, we capture test results
and logs, but not a consistent picture of how stressed the system was during
each run. That limits comparisons across machines, configurations, and over
time.
* CI is especially opaque: Shared GitHub runners have limited memory. When
tests fail there, we rarely have evidence of whether the runner was under
memory pressure.
* Inconsistent local vs CI experience: Developers may see passes locally while
CI fails (or the reverse), with no shared way to compare “how hard the system
was working” in each case.
h4. Who is affected
* Developers debugging flaky or failed integration test runs
* Release and CI maintainers tuning runner capacity and JVM settings
* Anyone reviewing archived runs to decide if a failure is environmental or a
real regression
h4. What “good” would look like (needs, not implementation)
* After a full integration test run, we can see how memory and system load
behaved during that run.
* That information is tied to the run (pass/fail, when, where it ran) and can
be kept with other run artifacts.
* The same kind of information is available on developer machines and on CI,
so comparisons are fair.
* Warnings or summaries highlight when a run likely ran under resource
pressure, without crying wolf on normal conditions (e.g. routine swap use on a
Mac with plenty of free RAM).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
