[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-465594547 @aljoscha, sure. Let me know if I can help This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-457051271 The compile problem was due master flink-connector-kafka-base uses Kafka 0.10.2.1, where ConsumerRecord constructor has different signature from Kafka 0.8, which was used flink-connector-kafka-base when I branched PR branch. To be signature independent, changed test to use mock instead for creating ConsumerRecord via constructor. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-456910348 Weird build is fine locally using 1.8.0_181-b13, but fails on Travis with 1.8.0_151-b12: >17:38:32.202 [ERROR] /home/travis/build/apache/flink/flink-connectors/flink-connector-kafka-base/src/test/java/org/apache/flink/streaming/connectors/kafka/JSONKeyValueDeserializationSchemaTest.java:[105,71] cannot infer type arguments for org.apache.kafka.clients.consumer.ConsumerRecord<> for following lines: ``` final ConsumerRecord consumerRecord = new ConsumerRecord<>( "topic#1", 3, serializedKey, serializedValue, 4L); ``` Any ideas? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-432846236 @yanghua, @aljoscha is anything else need to be done with this PR ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-431860212 Rebased on top of #6703 and implemented KeyedDeserializationSchema.Record for Kafka 2.0 (had to keep KeyedDeserializationSchema.Record because KeyedDeserializationSchema is used by legacy connectors as well. I will create PR for [FLINK-8500](https://issues.apache.org/jira/browse/FLINK-8500)(Timestamp) on top of this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-421014612 @tzulitai, using ```Record``` wrapping Kafka ```ConsumerRecord``` allows to add for example timestamp from PR #6105 w/o need to change client code, so it looks like more extensible approach. Not sure how it makes hard to reuse some already existing deserialization formats such as ```AvroDeserializationSchema```, at least not harder then now - ```AvroDeserializationSchema``` will be wrapped via ```KeyedDeserializationSchemaWrapper``` in exactly same way as know. Also ```KeyedDeserializationSchemaWrapper``` calls only ```Record.value()```, so it doesn't ties deserialization of byte with access to other metadata, not in terms of execution path (in logical terms it is always tied because underlying level - Kafka ```ConsumerRecord``` contains key, value and metadata) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers
alexeyt820 commented on issue #6615: [FLINK-8354] [flink-connectors] Add ability to access and provider Kafka headers URL: https://github.com/apache/flink/pull/6615#issuecomment-420812548 @yanghua, @aljoscha any other suggestions? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
