mattcasters commented on issue #7516:
URL: https://github.com/apache/hop/issues/7516#issuecomment-4960308131
## Plan to fix the 19 failing ITs (#2236)
Daily cycle is ~2h54m (down from ~4h15m) after #7506 (single-JVM suite
runner + Surefire Report Output + WorkflowExecutor keeps input fields on the
execution-result stream). Most new failures are age 2 since that landed.
### Failure groups
| Group | Count | Symptom | Likely cause |
| --- | --- | --- | --- |
| **A. HTTP** | 11 | `localhost:80` connection refused | Nested runner:
`HOSTNAME` falls back to parameter default instead of env `http` |
| **B. Kafka** | 3 | empty / unresolvable `bootstrap.servers` | Nested
runner: `BOOTSTRAP_SERVERS` not reaching child workflows |
| **C. AES** | 1 | decrypt yields empty plaintext | #7497 re-binds encoder
from hop-config (`Hop`) over docker AES2 env |
| **D. WorkflowExecutor** | 1 (`transforms.0083`) | output fields include
input `f1` | #7506 intentionally prepends input fields on execution-result
stream |
| **E. Beam** | 1 (`suite_startup`) | runner runs as Beam; missing
`filename` | Single-JVM runner incompatible with Beam project run config |
| **F. Env override** | 1 (`parameters…0012`) | `TEST_0012` stays default |
Env vars not winning under nested/single-JVM path |
| **G. Samples** | 1 (age 41) | chronic 30m suite | Pre-existing; partially
addressed by #7508 |
### Approach (prefer fix runner/contracts; keep single-JVM speedup)
1. **Confirm** with docker ITs (`CLIENT_UNZIP=false` against the built
client):
```bash
CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh http
CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh kafka
CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh aes
CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh
transforms
CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh
parameters_and_variables
CLIENT_UNZIP=false ./integration-tests/scripts/run-tests-docker.sh
beam_directrunner
```
Check whether `HOP_IT_PER_TEST_JVM=true` clears each failure (runner
nesting vs product bug).
2. **Fix variable/parameter plumbing** for A/B/F: extend `run-tests.sh` /
`run-workflow.hwf` so env hosts and suite params reach nested `main-*.hwf`
(HOSTNAME, BOOTSTRAP_SERVERS, CASTLEMOCK_HOSTNAME, etc.).
3. **Update 0083** expected field list for the new WorkflowExecutor schema
(D) — do not revert #7506.
4. **Beam (E):** force native local for the suite runner, or use per-test
JVM only for Beam projects.
5. **AES (C):** put `HOP_PASSWORD_ENCODER_PLUGIN=AES2` +
`HOP_AES_ENCODER_KEY` in the aes project/env config so project enable does not
wipe docker env.
6. **Samples (G):** find first failing sample in the long log; ignore
external-only samples or fix product as needed.
### Suggested PR split
- **PR1:** runner plumbing (A/B/F) + 0083 (D)
- **PR2:** Beam runner isolation (E)
- **PR3:** AES project encoder config (C)
- **PR4:** samples (G)
Escape hatch: `HOP_IT_PER_TEST_JVM=true` only where a suite truly cannot
share a JVM (e.g. Beam) — avoid a global flip that would undo the ~1h savings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
