taoran92 opened a new issue, #11060:
URL: https://github.com/apache/paimon/issues/11060

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   ### Paimon version
   
   2.0 & master
   
   ### Compute Engine
   
   Spark 3.5.9 / JDK 11
   
   ### Minimal reproduce step
   
   The [`Paimon Metric: report output metric` 
test](https://github.com/apache/paimon/blob/5982b1241aef46d8da03205373e4600c16e213b0/paimon-spark/paimon-spark-ut/src/test/scala/org/apache/paimon/spark/sql/PaimonMetricTest.scala#L257-L285)
 collects output metrics in a `SparkListener.onTaskEnd` callback, but removes 
the listener immediately after `INSERT` returns:
   
   ```scala
   try {
     spark.sparkContext.addSparkListener(listener)
     sql("INSERT INTO T VALUES 1, 2, 3")
   } finally {
     spark.sparkContext.removeSparkListener(listener)
   }
   Assertions.assertEquals(3, recordsWritten)
   ```
   
   To reproduce deterministically, register an auxiliary listener whose 
`onJobStart` callback holds the shared listener queue with a latch. Run the 
existing test with either V1 or V2 writes, allowing the `INSERT` to finish 
while the queue is held. Release the latch and remove the auxiliary listener 
during test cleanup.
   
   ### What doesn't meet your expectations?
   
   Expected: the assertions observe the output metrics for the completed 
`INSERT`.
   
   Actual: pending `TaskEnd` events have not reached the metric listener when 
it is removed. The assertion can fail with `expected: <3> but was: <0>` even 
though the write succeeded.
   
   ### Anything else?
   
   Related: #8285. No failing CI run is currently linked to this report.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to