Alirezs Asgarian, Thanks for reaching out here. After checking protocol definition of 2.10.2 and 3.2.4 and IO package format, I think it could be feasible if we don't use the new RPC method like `getLocatedFileInfo`. Anyway, regression testing is necessary before deploying to the prod environment.
TBH, I didn't practice using 3.2.* client to access 2.10.* cluster. Please correct me if I'm wrong. cc common-dev and hdfs-dev. Best Regards, - He Xiaoqiao On Sun, Aug 17, 2025 at 10:46 AM Alireza Asgarian <ar.asgarian1...@gmail.com> wrote: > Dear Xiaoqiao > > I hope this email finds you well. > > I am currently working on a project where the HDFS server is version > 2.10.2. Due to certain issues, I need to upgrade Spark to version 4, which > no longer supports Hadoop client 2. Upgrading our project cluster to Hadoop > 3 would be extremely time-consuming. These challenges have prompted me to > investigate whether the HDFS client 3.2.4 can be compatible with the HDFS > server 2.10.2. > > I have reviewed the code for client 3.2.4 and found that almost all RPC > methods remain the same. For the new RPC methods, as long as we avoid > calling the new methods and APIs, they haven't changed fundamentally—only a > few new fields have been added. Since it's based on Protobuf, the server > side should ignore these extra fields without causing any issues. > > The only potential problematic RPC in this scenario is the > getLocatedFileInfo method, which was added in HDFS 3 and does not exist in > HDFS 2. However, my investigations indicate that if we call the old methods > in the version 2 format, this RPC method will definitely not be invoked in > version 3.2.4. > > With all these checks in mind, I would greatly appreciate it if you could > provide guidance on this matter. If there's any point I've overlooked or > any compatibility issue that exists, please let me know. I truly need the > confirmation and opinion of a professional contributor in this area right > now. I would be extremely grateful for your help. > > Thank you very much in advance. > > Best regards, > Alireza Asgarian >
