HeartSaVioR commented on a change in pull request #32796:
URL: https://github.com/apache/spark/pull/32796#discussion_r647184751
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SymmetricHashJoinStateManager.scala
##########
@@ -269,18 +269,14 @@ class SymmetricHashJoinStateManager(
// The backing store is arraylike - we as the caller are responsible
for filling back in
// any hole. So we swap the last element into the hole and decrement
numValues to shorten.
// clean
- if (numValues > 1) {
+ if (index != numValues - 1) {
val valuePairAtMaxIndex = keyWithIndexToValue.get(currentKey,
numValues - 1)
if (valuePairAtMaxIndex != null) {
Review comment:
Just wondering, would `valuePairAtMaxIndex == null` happen in normal
case? I can only imagine the case the iterator is not fully consumed and
somehow numValues is not updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
