Bui Bao Anh created SPARK-32961: ----------------------------------- Summary: PySpark CSV read with UTF-16 encoding is not working correctly Key: SPARK-32961 URL: https://issues.apache.org/jira/browse/SPARK-32961 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 3.0.1, 2.4.4 Environment: both spark local and cluster mode Reporter: Bui Bao Anh
There are weird characters in the output when printing out to console or writing to files. Below is how it look in Spark Dataframe: !image-2020-09-22-10-44-40-984.png! However, this is not the case in Pandas Dataframe: !image-2020-09-22-10-51-28-294.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org