srowen commented on issue #26831: [SPARK-30201][SQL] HiveOutputWriter standardOI should use ObjectInspectorCopyOption.DEFAULT URL: https://github.com/apache/spark/pull/26831#issuecomment-564820775 The second pipeline is definitely wrong. Construing bytes that encode a string in some non-UTF8 encoding as UTF8, then getting the bytes back, is going to be incorrect in some cases. I am still not clear how you supply bytes that encode a string in something other than UTF8 to Spark, but tell it to interpret as UTF8?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
