(incubator-celeborn) branch main updated: [CELEBORN-1078] Log info to indicate columnar shuffle writer take effect

[zhouky]

This is an automated email from the ASF dual-hosted git repository.

zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 73707a9d4 [CELEBORN-1078] Log info to indicate columnar shuffle writer 
take effect
73707a9d4 is described below

commit 73707a9d47393045f4fc05be22c5b8242f4734d9
Author: Fei Wang <fwan...@ebay.com>
AuthorDate: Fri Feb 16 22:03:31 2024 +0800

    [CELEBORN-1078] Log info to indicate columnar shuffle writer take effect
    
    ### What changes were proposed in this pull request?
    Log info when using columnar hash shuffle writer.
    
    ### Why are the changes needed?
    
    To close CELEBORN-1078
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Existing UT.
    
    Closes #2294 from turboFei/CELEBORN_1078_columnar_shuffle.
    
    Authored-by: Fei Wang <fwan...@ebay.com>
    Signed-off-by: zky.zhoukeyong <zky.zhoukey...@alibaba-inc.com>
---
 .../main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java | 1 +
 1 file changed, 1 insertion(+)

diff --git 
a/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java
 
b/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java
index 8064a8bab..fe29e9646 100644
--- 
a/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java
+++ 
b/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java
@@ -280,6 +280,7 @@ public class SparkShuffleManager implements ShuffleManager {
           SendBufferPool pool =
               SendBufferPool.get(cores, sendBufferPoolCheckInterval, 
sendBufferPoolExpireTimeout);
           if (COLUMNAR_SHUFFLE_CLASSES_PRESENT && 
celebornConf.columnarShuffleEnabled()) {
+            logger.info("Creating columnar hash shuffle writer for shuffle 
{}", shuffleId);
             return SparkUtils.createColumnarHashBasedShuffleWriter(
                 shuffleId, h, context, celebornConf, shuffleClient, metrics, 
pool);
           } else {