(spark) branch master updated: [SPARK-48219][CORE] StreamReader Charset fix with UTF8

[yao]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 5e8322150a05 [SPARK-48219][CORE] StreamReader Charset fix with UTF8
5e8322150a05 is described below

commit 5e8322150a050ad4d0c3962d62c9a2b3e9a937c1
Author: xuyu <11161...@vivo.com>
AuthorDate: Thu May 16 12:11:44 2024 +0800

    [SPARK-48219][CORE] StreamReader Charset fix with UTF8
    
    ### What changes were proposed in this pull request?
    Fix some StreamReader not set with UTF8，if we actually default charset not 
support Chinese chars such as latin and conf contain Chinese chars，it would not 
resolve success，so we need set it as utf8 in StreamReader，we can find all 
StreamReader with utf8 charset in other compute framework，such as 
Calcite、Hive、Hudi and so on.
    
    ### Why are the changes needed?
    May cause string decode not as expected
    
    ### Does this PR introduce _any_ user-facing change?
    Yes
    
    ### How was this patch tested?
    Not need
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #46509 from xuzifu666/SPARK-48219.
    
    Authored-by: xuyu <11161...@vivo.com>
    Signed-off-by: Kent Yao <y...@apache.org>
---
 .../main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git 
a/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
 
b/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
index 410d010a79bd..4b55453ec7a8 100644
--- 
a/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
+++ 
b/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
@@ -22,6 +22,7 @@ import java.io.File;
 import java.io.FileInputStream;
 import java.io.IOException;
 import java.io.InputStreamReader;
+import java.nio.charset.StandardCharsets;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Map;
@@ -171,7 +172,7 @@ public class HiveSessionImpl implements HiveSession {
       FileInputStream initStream = null;
       BufferedReader bufferedReader = null;
       initStream = new FileInputStream(fileName);
-      bufferedReader = new BufferedReader(new InputStreamReader(initStream));
+      bufferedReader = new BufferedReader(new InputStreamReader(initStream, 
StandardCharsets.UTF_8));
       return bufferedReader;
     }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org