JoshRosen commented on code in PR #42742:
URL: https://github.com/apache/spark/pull/42742#discussion_r1313513495
##########
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala:
##########
@@ -220,7 +220,8 @@ private[spark] class MemoryStore(
}
// Unroll this block safely, checking whether we have exceeded our
threshold periodically
- while (values.hasNext && keepUnrolling) {
+ // and if no thread interrupts have been received.
+ while (values.hasNext && keepUnrolling &&
!Thread.currentThread().isInterrupted) {
Review Comment:
> IIRC isInterrupted is a couple of orders more expensive than a boolean
check.
In older JDKs it is implemented via an intrinsic and in newer ones (14+)
it's just a volatile boolean check (see
https://bugs.openjdk.org/browse/JDK-8229516), so I don't think it's too
expensive. There's an old StackOverflow answer at
https://stackoverflow.com/a/5158441/590203 offering a plausible explanation for
why the intrinsic should be cheap.
On JDK 11 on my laptop, I ran a toy JMH benchmark and measured < 1ns per
call.
I think these costs are very small in comparison to the per-element
serialization costs.
Given this, I'm not too worried about performance regressions due to this
change and think it's probably okay to check on every element rather than every
16th.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
