[ https://issues.apache.org/jira/browse/FLINK-33197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773536#comment-17773536 ]
Liu Chong commented on FLINK-33197: ----------------------------------- we've got the code ready for adding ByteArraySchema to pyflink, would you like to give some comments if we should move forward and submit the PR? [~dianfu] > PyFlink support for ByteArraySchema > ----------------------------------- > > Key: FLINK-33197 > URL: https://issues.apache.org/jira/browse/FLINK-33197 > Project: Flink > Issue Type: New Feature > Components: API / Python > Affects Versions: 1.17.0 > Reporter: Liu Chong > Priority: Minor > > Currently in Python Flink API, when reading messages from a Kafka source, > only SimpleStringSchema is available. > If the data is in arbitary binary format(e.g. marshalled Protocol Buffer msg) > it may not be decodable with the default 'utf-8' encoding. > There's currently a workaround which is to manually set the encoding to > 'ISO-8859-1' which supports all possible byte combinations. > However this is not an elegant solution. > We should support ByteArraySchema which outputs a raw byte array for > subsequent unmarshalling. -- This message was sent by Atlassian Jira (v8.20.10#820010)