[
https://issues.apache.org/jira/browse/FLINK-12410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043641#comment-17043641
]
Dominique De Vito edited comment on FLINK-12410 at 2/24/20 4:18 PM:
--------------------------------------------------------------------
Note : I am using Flink 1.8.1.
And I have got the same lines in my logs:
{code:java}
2020-02-24 12:55:36,137 INFO org.apache.flink.api.java.typeutils.TypeExtractor
- class
org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition does
not contain a setter for field topic
2020-02-24 12:55:36,137 INFO org.apache.flink.api.java.typeutils.TypeExtractor
- Class class
org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
cannot be used as a POJO type because not all fields are valid POJO fields, and
must be processed as GenericType. Please read the Flink documentation on "Data
Types & Serialization" for details of the effect on performance.
{code}
However, I am NOT doing myself any use of serialization involving
KafkaTopicPartition type.
Indeed, the string "KafkaTopicPartition" does NOT even appear in my source code.
It looks like, as said above, that this log is produced due to Flink inner
("FlinkKafkaConsumerBase" ?) class:
{code:java}
2020-02-24 12:55:36,173 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer: {KafkaTopicPartition{topic='<SOME
TOPIC>', partition=0}=1234}
2020-02-24 12:55:36,231 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer: {KafkaTopicPartition{topic='<YET
ANOTHER TOPIC>', partition=0}=3407165}
2020-02-24 12:55:36,255 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer: {KafkaTopicPartition{topic='<SOME
ANOTHER ONE TOPIC>', partition=0}=7, KafkaTopicPartition{topic='<SOME ANOTHER
ONE TOPIC>', partition=1}=7, KafkaTopicPartition{topic='<SOME ANOTHER ONE
TOPIC>', partition=2}=6, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=3}=4, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=4}=3, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=5}=8, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=6}=5, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=7}=3}
2020-02-24 12:55:36,256 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer:
{KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=0}=45,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=1}=69,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=2}=61,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=3}=53,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=4}=41,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=5}=69,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=6}=57,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=7}=81,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=8}=41,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=9}=53}
{code}
was (Author: dmdevito):
I have got the same lines in my logs:
{code:java}
2020-02-24 12:55:36,137 INFO org.apache.flink.api.java.typeutils.TypeExtractor
- class
org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition does
not contain a setter for field topic
2020-02-24 12:55:36,137 INFO org.apache.flink.api.java.typeutils.TypeExtractor
- Class class
org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
cannot be used as a POJO type because not all fields are valid POJO fields, and
must be processed as GenericType. Please read the Flink documentation on "Data
Types & Serialization" for details of the effect on performance.
{code}
However, I am NOT doing myself any use of serialization involving
KafkaTopicPartition type.
Indeed, the string "KafkaTopicPartition" does NOT even appear in my source code.
It looks like, as said above, that this log is produced due to Flink inner
("FlinkKafkaConsumerBase" ?) class:
{code:java}
2020-02-24 12:55:36,173 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer: {KafkaTopicPartition{topic='<SOME
TOPIC>', partition=0}=1234}
2020-02-24 12:55:36,231 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer: {KafkaTopicPartition{topic='<YET
ANOTHER TOPIC>', partition=0}=3407165}
2020-02-24 12:55:36,255 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer: {KafkaTopicPartition{topic='<SOME
ANOTHER ONE TOPIC>', partition=0}=7, KafkaTopicPartition{topic='<SOME ANOTHER
ONE TOPIC>', partition=1}=7, KafkaTopicPartition{topic='<SOME ANOTHER ONE
TOPIC>', partition=2}=6, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=3}=4, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=4}=3, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=5}=8, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=6}=5, KafkaTopicPartition{topic='<SOME ANOTHER ONE TOPIC>',
partition=7}=3}
2020-02-24 12:55:36,256 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - Setting
restore state in the FlinkKafkaConsumer:
{KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=0}=45,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=1}=69,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=2}=61,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=3}=53,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=4}=41,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=5}=69,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=6}=57,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=7}=81,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=8}=41,
KafkaTopicPartition{topic='<SOME-OTHER-TOPIC>', partition=9}=53}
{code}
> KafkaTopicPartition cannot be used as a POJO type because not all fields are
> valid POJO fields
> ----------------------------------------------------------------------------------------------
>
> Key: FLINK-12410
> URL: https://issues.apache.org/jira/browse/FLINK-12410
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.8.0
> Reporter: Chethan UK
> Priority: Minor
>
> While Pushing data to Kafka Topic using Pojo, there is an error: "INFO
> org.apache.flink.api.java.typeutils.TypeExtractor - Class class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> cannot be used as a POJO type because not all fields are valid POJO fields,
> and must be processed as GenericType"
> {code:java}
> // code placeholder
> 09:18:27,633 INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> does not contain a setter for field topic
> 09:18:27,634 INFO org.apache.flink.api.java.typeutils.TypeExtractor - Class
> class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> cannot be used as a POJO type because not all fields are valid POJO fields,
> and must be processed as GenericType. Please read the Flink documentation on
> "Data Types & Serialization" for details of the effect on performance.
> 09:18:27,633 INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> does not contain a setter for field topic
> 09:18:27,635 INFO org.apache.flink.api.java.typeutils.TypeExtractor - Class
> class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> cannot be used as a POJO type because not all fields are valid POJO fields,
> and must be processed as GenericType. Please read the Flink documentation on
> "Data Types & Serialization" for details of the effect on performance.
> 09:18:27,635 INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> does not contain a setter for field topic
> 09:18:27,636 INFO org.apache.flink.api.java.typeutils.TypeExtractor - Class
> class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> cannot be used as a POJO type because not all fields are valid POJO fields,
> and must be processed as GenericType. Please read the Flink documentation on
> "Data Types & Serialization" for details of the effect on performance.
> 09:18:27,639 INFO org.apache.flink.api.java.typeutils.TypeExtractor - class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> does not contain a setter for field topic
> 09:18:27,642 INFO org.apache.flink.api.java.typeutils.TypeExtractor - Class
> class
> org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> cannot be used as a POJO type because not all fields are valid POJO fields,
> and must be processed as GenericType. Please read the Flink documentation on
> "Data Types & Serialization" for details of the effect on performance.
> 09:18:27,654 INFO
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - No
> restore state for FlinkKafkaConsumer.
> 09:18:27,655 INFO
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - No
> restore state for FlinkKafkaConsumer.
> 09:18:27,655 INFO
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - No
> restore state for FlinkKafkaConsumer.
> 09:18:27,654 INFO
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase - No
> restore state for FlinkKafkaConsumer.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
