GitHub user sivaguru-rajasekaran added a comment to the discussion: Oracle
Table Input with variables (${LAST_RUN_TS}, ${LAST_RUN_ID}) failing in Apache
Hop
Thanks for the detailed explanation — that helped clarify what’s really
happening under the hood with variable substitution.
Based on your suggestion, I moved the variable handling fully into a
**JavaScript step**, followed by **Generate Rows**, and then passed the
resolved values downstream instead of relying on variables inside the Table
Input. With this approach, the SQL is now fully concrete before execution, and
logging the parsed SQL confirms it’s exactly what I expect.
When I execute the steps **individually**, everything works fine:
* Variables are substituted correctly in JavaScript
* The generated rows contain the expected values
* Table Output inserts data without issues
However, when I execute the **full transformation**, the Table Output step
intermittently fails or behaves inconsistently, even though the upstream steps
still show correct data in preview.
At this point, I suspect one of the following:
* Row metadata mismatch between Generate Rows and Table Output
* Commit / batch size behavior when running end-to-end
* Execution order or row stream timing differences vs step-level execution
I’m currently validating field types, lengths, and commit settings on the Table
Output, but if there are any known gotchas around:
* Generate Rows → Table Output pipelines
* Or differences between preview vs full-run execution
I’d appreciate a nudge in the right direction. This feels close — just trying
to pin down that last edge case.
Thanks again...
<img width="814" height="457" alt="Image"
src="https://github.com/user-attachments/assets/a77bce1f-5109-47b4-ac6f-e57c766c86b0";
/>
<img width="824" height="542" alt="Image"
src="https://github.com/user-attachments/assets/479cb605-921e-4d91-8533-27ca9ad07cc0";
/>
GitHub link:
https://github.com/apache/hop/discussions/6250#discussioncomment-15324801
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
