sivaguru-rajasekaran commented on issue #6248: URL: https://github.com/apache/hop/issues/6248#issuecomment-3685328634
Thanks for the detailed explanation — that helped clarify what’s really happening under the hood with variable substitution. Based on your suggestion, I moved the variable handling fully into a **JavaScript step**, followed by **Generate Rows**, and then passed the resolved values downstream instead of relying on variables inside the Table Input. With this approach, the SQL is now fully concrete before execution, and logging the parsed SQL confirms it’s exactly what I expect. When I execute the steps **individually**, everything works fine: * Variables are substituted correctly in JavaScript * The generated rows contain the expected values * Table Output inserts data without issues However, when I execute the **full transformation**, the Table Output step intermittently fails or behaves inconsistently, even though the upstream steps still show correct data in preview. At this point, I suspect one of the following: * Row metadata mismatch between Generate Rows and Table Output * Commit / batch size behavior when running end-to-end * Execution order or row stream timing differences vs step-level execution I’m currently validating field types, lengths, and commit settings on the Table Output, but if there are any known gotchas around: * Generate Rows → Table Output pipelines * Or differences between preview vs full-run execution I’d appreciate a nudge in the right direction. This feels close — just trying to pin down that last edge case. Thanks again... <img width="814" height="457" alt="Image" src="https://github.com/user-attachments/assets/a77bce1f-5109-47b4-ac6f-e57c766c86b0"; /> <img width="824" height="542" alt="Image" src="https://github.com/user-attachments/assets/479cb605-921e-4d91-8533-27ca9ad07cc0"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
