[GitHub] [spark] wankunde commented on a diff in pull request #41782: [SPARK-44239][SQL] Free memory allocated by large vectors when vectors are reset

Sun, 20 Aug 2023 23:43:27 -0700 


wankunde commented on code in PR #41782:
URL: https://github.com/apache/spark/pull/41782#discussion_r1299672594


##########
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java:
##########
@@ -846,7 +849,14 @@ public final void addElementsAppended(int num) {
   /**
    * Marks this column as being constant.
    */
-  public final void setIsConstant() { isConstant = true; }
+  public final void setIsConstant() {
+    if (childColumns != null) {
+      for (WritableColumnVector c : childColumns) {
+        c.setIsConstant();

Review Comment:
   For example, `alter table t add column s array<int> default array(1, 2)` , 
spark will create a vector for column s and , and a vector for the items of 
this column.
   Before this PR, both those two vectors will not be reset.
   After this PR, the second vector will be reset without this change. 
   
   
![image](https://github.com/apache/spark/assets/3626747/e77926f3-6ac0-4cf5-a776-6e7b3f0957ef)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to