mattcasters opened a new pull request, #7746: URL: https://github.com/apache/hop/pull/7746
## Summary Fixes [#7743](https://github.com/apache/hop/issues/7743) (thanks @nadment for the report and repro). With Caching Database execution info location, workflow rows were stored with: `parent_id = Internal.Spark.TransformOwnerId` instead of: - `null` for a top-level parent workflow - the parent workflow’s log-channel id for a nested child workflow ### Root cause This is **not** a bug in the Caching Database location. The denormalized `parent_id` column only made a registration bug obvious. In `LocalWorkflowEngine` (from the Native Spark engine work), parent rebind used: ```java resolve("Internal.Spark.TransformOwnerId") ``` `IVariables.resolve()` only expands `${...}` / `%%...%%` tokens. A bare name is returned unchanged, so **every** local workflow execution overwrote the correct parent id with that literal string. Spark is supposed to set the real owner id via `HopMapPartitionsFn` when a workflow runs nested under a mapPartitions Workflow Executor. Rebind should only run when that variable is actually set. ### Fix - Look up with `getVariable(...)` (returns `null` when unset) instead of `resolve(...)` - Centralize the variable name as `VAR_SPARK_TRANSFORM_OWNER_ID` (kept in sync with `SparkConst` in the spark plugin; engine cannot depend on that plugin) - Unit tests for unset / set / empty variable, and documenting the `resolve` vs `getVariable` distinction ### Expected after fix | Scenario | Before | After | |----------|--------|--------| | Top-level workflow | `parent_id = Internal.Spark.TransformOwnerId` | `null` | | Child workflow (local) | same literal | parent log-channel id | | Nested under Spark Workflow Executor | same literal | synthetic `pipeId\|name\|copy` | ## Test plan - [x] `LocalWorkflowEngineSparkParentRebindTest` (7 tests) - [ ] Repro from #7743 (`workflows.zip`): parent → child local workflows, Caching Database execution info location, hop-gui - Parent row: `parent_id` null/empty - Child row: `parent_id` = parent’s `id` - [ ] Optional: nested workflow under Spark Workflow Executor still parents to synthetic owner id -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
