Technoboy- opened a new pull request #14615: URL: https://github.com/apache/pulsar/pull/14615
Cherry-pick https://github.com/apache/pulsar/pull/14379 ### Motivation 1. Sometimes when close PulsarRecordCursor, there may occur below exception: ``` 2021-12-15T11:40:58.328803210-05:00 2021-12-15T16:40:58.328Z INFO SplitRunner-6-8221 org.apache.pulsar.sql.presto.PulsarRecordCursor Closing cursor record 2021-12-15T11:40:58.330468168-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr Exception in thread "derserialize-thread-split-7" 2021-12-15T11:40:58.330560520-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1 2021-12-15T11:40:58.330567283-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.apache.bookkeeper.mledger.util.AbstractCASReferenceCounted.release0(AbstractCASReferenceCounted.java:99) 2021-12-15T11:40:58.330572615-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.apache.bookkeeper.mledger.util.AbstractCASReferenceCounted.release(AbstractCASReferenceCounted.java:87) 2021-12-15T11:40:58.330639859-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries$1.accept(PulsarRecordCursor.java:314) 2021-12-15T11:40:58.330650078-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries$1.accept(PulsarRecordCursor.java:254) 2021-12-15T11:40:58.330654136-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.jctools.queues.SpscArrayQueue.drain(SpscArrayQueue.java:266) 2021-12-15T11:40:58.330658199-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.jctools.queues.SpscArrayQueue.drain(SpscArrayQueue.java:239) 2021-12-15T11:40:58.330701030-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries.run(PulsarRecordCursor.java:254) 2021-12-15T11:40:58.330706984-05:00 2021-12-15T16:40:58.330Z INFO derserialize-thread-split-7 stderr at java.base/java.lang.Thread.run(Thread.java:829) ``` Because DeserializeEntries read entries from `entryQueue` and will release relative entries after processing. But when PulsarRecordCursor closes, it will also release entries in the `entryQueue`, so different threads have released the same entry to cause the above issue. 2. If DeserializeEntries occurs any uncaught exception, PulsarRecordCursor#advanceNextPosition could not retrieve any data from `messageQueue` and make the current thread sleep indefinitely. https://github.com/apache/pulsar/blob/0facd24e8eec2df56f6f241ce2cf87eb98590a7e/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarRecordCursor.java#L481-L503 So when no message from `messageQueue`, we should check if there is any uncaught exception. ### Modifications 1. Add `closeHandle` in DeserializeEntries and release entries in the `entryQueue` after DeserializeEntries close. 2. Add try/catch block in DeserializeEntries#run. ### Documentation - [x] `no-need-doc` (Please explain why) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
