[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

Tue, 22 May 2018 02:28:43 -0700 

Github user cxzl25 commented on the issue:

    https://github.com/apache/spark/pull/21311
  
    @JoshRosen  @cloud-fan @gatorsmile 
    When introducing 
[SPARK-10399](https://issues.apache.org/jira/browse/SPARK-10399),UnsafeRow#getUTF8String
 check the size at this time.
    
[UnsafeRow#getUTF8String](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java#L420)
    
[OnHeapMemoryBlock](https://github.com/apache/spark/blob/master/common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OnHeapMemoryBlock.java#L34)
    >The sum of size 2097152 and offset 32 should not be larger than the size 
of the given memory space 2097168
    
![image](https://user-images.githubusercontent.com/3898450/40353741-cbd02616-5de4-11e8-9406-4fcb65a69294.png)
    
    But when this patch is not introduced, no error, get wrong value.
    
    ![spark-2 2 
0](https://user-images.githubusercontent.com/3898450/40353815-febe5494-5de4-11e8-9c43-e75a95a3e74f.png)
    
    
    



---

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to