dsanderbi opened a new issue, #9096:
URL: https://github.com/apache/hop/issues/9096
### Apache Hop version?
2.19.0
### Java version?
26.0.2
### Operating system
Windows
### What happened?
We use "Blocking until transforms finish" to run an SQL statement only after
a Table output has written all its rows:
Row generator (1 row) -> Blocking until transforms finish [waits for
"write to table", copy 0] -> Execute SQL
... input rows -> "write to table" (Table output, commit size 1000)
This works in almost every run, but in one of about 50 runs the blocking
transform passed its row on right at the start of the pipeline, while the Table
output was still running.
The SQL statement then ran at the same time as the Table output and failed
with a primary key violation, because both inserted the same keys.
Log of the failing run (basic log level):
10:20:37 - wait for write to table.0 - Transform write to table status:
Running
10:20:37 - wait for write to table.0 - Finished processing (I=0, O=0,
R=1, W=1, U=0, E=0)
10:20:38 - (rows start arriving at "write to table")
10:20:41 - Execute SQL.0 - ERROR: ... Violation of PRIMARY KEY
constraint ...
10:20:41 - write to table.0 - Finished processing (I=0, O=6000, R=6000,
W=5000, U=0, E=0)
The blocking transform finished at 10:20:37 and logged the status of "write
to table" as "Running" at that moment. The Table output only received its rows
after that and was stopped at 10:20:41 because of the error. In all other runs
the blocking transform waited until the Table output had finished.
We could not reproduce it on purpose. It looks like a timing issue at the
start of the pipeline, when the waited-for transform is just switching from
idle to running.
**Expected**: the blocking transform always waits until the listed transform
has finished.
### Issue Priority
Priority: 2
### Issue Component
Component: Transforms
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
