[
https://issues.apache.org/jira/browse/FLINK-14356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-14356:
----------------------------
Description:
I want to use flink sql to write kafka messages directly to hdfs. The
serialization and deserialization of messages are not involved in the middle.
The bytes of the message directly convert the first field of Row. However, the
current RowSerializationSchema does not support the conversion of bytes to
VARBINARY. Can we add some special RowSerializationSchema and
RowDerializationSchema ?
========================================================================
Copied from FLINK-9963:
Sometimes it might be useful to just read or write a single value into Kafka or
other connectors. We should add a single-value SerializationSchemaFactory and
single-value DeserializationSchemaFactory, the types below and their array
types shall be considered.
byte, short, int, long, float, double, string
For the numeric types, we might want to specify the endian format.
A string type single-value format will be added with this issue for future
reference.
was:I want to use flink sql to write kafka messages directly to hdfs. The
serialization and deserialization of messages are not involved in the middle.
The bytes of the message directly convert the first field of Row. However, the
current RowSerializationSchema does not support the conversion of bytes to
VARBINARY. Can we add some special RowSerializationSchema and
RowDerializationSchema ?
> Introduce "single-field" format to (de)serialize message to a single field
> --------------------------------------------------------------------------
>
> Key: FLINK-14356
> URL: https://issues.apache.org/jira/browse/FLINK-14356
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / API
> Reporter: jinfeng
> Assignee: jinfeng
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> I want to use flink sql to write kafka messages directly to hdfs. The
> serialization and deserialization of messages are not involved in the middle.
> The bytes of the message directly convert the first field of Row. However,
> the current RowSerializationSchema does not support the conversion of bytes
> to VARBINARY. Can we add some special RowSerializationSchema and
> RowDerializationSchema ?
> ========================================================================
> Copied from FLINK-9963:
> Sometimes it might be useful to just read or write a single value into Kafka
> or other connectors. We should add a single-value SerializationSchemaFactory
> and single-value DeserializationSchemaFactory, the types below and their
> array types shall be considered.
> byte, short, int, long, float, double, string
> For the numeric types, we might want to specify the endian format.
> A string type single-value format will be added with this issue for future
> reference.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
