(celeborn) branch main updated: [CELEBORN-1372] Update ControlMessages to handle ApplicationMeta and ApplicationMetaRequest

[chengpan]

This is an automated email from the ASF dual-hosted git repository.

chengpan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 5065dbed5 [CELEBORN-1372] Update ControlMessages to handle 
ApplicationMeta and ApplicationMetaRequest
5065dbed5 is described below

commit 5065dbed59fc85c0304601e93d3530f8208d8ccc
Author: Mridul Muralidharan <mri...@gmail.com>
AuthorDate: Fri Apr 5 08:49:32 2024 +0800

    [CELEBORN-1372] Update ControlMessages to handle ApplicationMeta and 
ApplicationMetaRequest
    
    ### What changes were proposed in this pull request?
    
    With authentication enabled, rpc requests from executors to driver and vice 
versa fail when trying to send/receive `ApplicationMeta` and 
`ApplicationMetaRequest`.
    
    ### Why are the changes needed?
    
    Executor is unable to communicate with the driver with auth enabled.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, bug fix for feature introduced in 0.5
    
    ### How was this patch tested?
    
    Tested with spark 3.1 against a patched version of Celeborn (pre-reqs: 
#2445, #2446)
    
    Closes #2447 from mridulm/CELEBORN-1372-fix-ApplicationMeta-serde.
    
    Lead-authored-by: Mridul Muralidharan <mri...@gmail.com>
    Co-authored-by: Mridul Muralidharan <mridulatgmail.com>
    Signed-off-by: Cheng Pan <cheng...@apache.org>
---
 .../celeborn/common/protocol/message/ControlMessages.scala   | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git 
a/common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala
 
b/common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala
index 82946dfb7..9bd9469b0 100644
--- 
a/common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala
+++ 
b/common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala
@@ -891,6 +891,12 @@ object ControlMessages extends Logging {
 
     case pb: PbCheckWorkersAvailableResponse =>
       new TransportMessage(MessageType.CHECK_WORKERS_AVAILABLE_RESPONSE, 
pb.toByteArray)
+
+    case pb: PbApplicationMeta =>
+      new TransportMessage(MessageType.APPLICATION_META, pb.toByteArray)
+
+    case pb: PbApplicationMetaRequest =>
+      new TransportMessage(MessageType.APPLICATION_META_REQUEST, 
pb.toByteArray)
   }
 
   // TODO change return type to GeneratedMessageV3
@@ -1238,6 +1244,12 @@ object ControlMessages extends Logging {
 
       case CHECK_WORKERS_AVAILABLE_RESPONSE_VALUE =>
         PbCheckWorkersAvailableResponse.parseFrom(message.getPayload)
+
+      case APPLICATION_META_VALUE =>
+        PbApplicationMeta.parseFrom(message.getPayload)
+
+      case APPLICATION_META_REQUEST_VALUE =>
+        PbApplicationMetaRequest.parseFrom(message.getPayload)
     }
   }
 }