[jira] [Created] (KAFKA-7624) HOW TO FILETER NESTED THROUGH KAFKA CONNECT
Chenchu Lakshman kumar created KAFKA-7624:
-
Summary: HOW TO FILETER NESTED THROUGH KAFKA CONNECT
Key: KAFKA-7624
URL: https://issues.apache.org/jira/browse/KAFKA-7624
Project: Kafka
Issue Type: Test
Components: KafkaConnect
Reporter: Chenchu Lakshman kumar
Hi Team
please help
{"messageID":"ID:COPTW_B_SIT.1D815BC7447C2A20:3880","messageType":"text","timestamp":1542191173487,"deliveryMode":2,"correlationID":null,"replyTo":null,"destination":\{"destinationType":"queue","name":"test.queue"},"redelivered":false,"type":null,"expiration":0,"priority":4,"properties":{},"bytes":null,"map":null,{color:#f6c342}"text":"helo{color}"}
we need to get only "text":"helo" message while consuming , so that i can send
direclty to mango DB.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KAFKA-7623) SMT STRUCT to MASK or FILTER
[
https://issues.apache.org/jira/browse/KAFKA-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenchu Lakshman kumar resolved KAFKA-7623.
---
Resolution: Fixed
> SMT STRUCT to MASK or FILTER
>
>
> Key: KAFKA-7623
> URL: https://issues.apache.org/jira/browse/KAFKA-7623
> Project: Kafka
> Issue Type: Test
> Components: KafkaConnect
>Reporter: Chenchu Lakshman kumar
>Priority: Major
>
> {
> "schema": {
> "type": "struct",
> "fields": [{
> "type": "string",
> "optional": false,
> "doc": "This field stores the value of `Message.getJMSMessageID()
> `_.",
> "field": "messageID"
> }, {
> "type": "string",
> "optional": false,
> "doc": "This field stores the type of message that was received. This
> corresponds to the subinterfaces of `Message
> `_. `BytesMessage
> `_ =
> `bytes`, `MapMessage
> `_ = `map`,
> `ObjectMessage
> `_ =
> `object`, `StreamMessage
> `_ =
> `stream` and `TextMessage
> `_ = `text`.
> The corresponding field will be populated with the values from the respective
> Message subinterface.",
> "field": "messageType"
> }, {
> "type": "int64",
> "optional": false,
> "doc": "Data from the `getJMSTimestamp()
> `_
> method.",
> "field": "timestamp"
> }, {
> "type": "int32",
> "optional": false,
> "doc": "This field stores the value of `Message.getJMSDeliveryMode()
> `_.",
> "field": "deliveryMode"
> }, {
> "type": "string",
> "optional": true,
> "doc": "This field stores the value of `Message.getJMSCorrelationID()
> `_.",
> "field": "correlationID"
> }, {
> "type": "struct",
> "fields": [{
> "type": "string",
> "optional": false,
> "doc": "The type of JMS Destination, and either ``queue`` or ``topic``.",
> "field": "destinationType"
> }, {
> "type": "string",
> "optional": false,
> "doc": "The name of the destination. This will be the value of
> `Queue.getQueueName()
> `_
> or `Topic.getTopicName()
> `_.",
> "field": "name"
> }],
> "optional": true,
> "name": "io.confluent.connect.jms.Destination",
> "doc": "This schema is used to represent a JMS Destination, and is either
> `queue `_ or `topic
> `_.",
> "field": "replyTo"
> }, {
> "type": "struct",
> "fields": [{
> "type": "string",
> "optional": false,
> "doc": "The type of JMS Destination, and either ``queue`` or ``topic``.",
> "field": "destinationType"
> }, {
> "type": "string",
> "optional": false,
> "doc": "The name of the destination. This will be the value of
> `Queue.getQueueName()
> `_
> or `Topic.getTopicName()
> `_.",
> "field": "name"
> }],
> "optional": true,
> "name": "io.confluent.connect.jms.Destination",
> "doc": "This schema is used to represent a JMS Destination, and is either
> `queue `_ or `topic
> `_.",
> "field": "destination"
> }, {
> "type": "boolean",
> "optional": false,
> "doc": "This field stores the value of `Message.getJMSRedelivered()
> `_.",
> "field": "redelivered"
> }, {
> "type": "string",
> "optional": true,
> "doc": "This field stores the value of `Message.getJMSType()
> `_.",
> "field": "type"
> }, {
> "type": "int64",
> "optional": false,
> "doc": "This field stores the value of `Message.getJMSExpiration()
> `_.",
> "field": "expiration"
> }, {
> "type": "int32",
> "optional": false,
> "doc": "This field stores the value of `Message.getJMSPriority()
>
[jira] [Created] (KAFKA-7623) SMT STRUCT to MASK or FILTER
Chenchu Lakshman kumar created KAFKA-7623:
-
Summary: SMT STRUCT to MASK or FILTER
Key: KAFKA-7623
URL: https://issues.apache.org/jira/browse/KAFKA-7623
Project: Kafka
Issue Type: Test
Components: KafkaConnect
Reporter: Chenchu Lakshman kumar
{
"schema": {
"type": "struct",
"fields": [{
"type": "string",
"optional": false,
"doc": "This field stores the value of `Message.getJMSMessageID()
`_.",
"field": "messageID"
}, {
"type": "string",
"optional": false,
"doc": "This field stores the type of message that was received. This
corresponds to the subinterfaces of `Message
`_. `BytesMessage
`_ = `bytes`,
`MapMessage `_ =
`map`, `ObjectMessage
`_ =
`object`, `StreamMessage
`_ = `stream`
and `TextMessage
`_ = `text`.
The corresponding field will be populated with the values from the respective
Message subinterface.",
"field": "messageType"
}, {
"type": "int64",
"optional": false,
"doc": "Data from the `getJMSTimestamp()
`_
method.",
"field": "timestamp"
}, {
"type": "int32",
"optional": false,
"doc": "This field stores the value of `Message.getJMSDeliveryMode()
`_.",
"field": "deliveryMode"
}, {
"type": "string",
"optional": true,
"doc": "This field stores the value of `Message.getJMSCorrelationID()
`_.",
"field": "correlationID"
}, {
"type": "struct",
"fields": [{
"type": "string",
"optional": false,
"doc": "The type of JMS Destination, and either ``queue`` or ``topic``.",
"field": "destinationType"
}, {
"type": "string",
"optional": false,
"doc": "The name of the destination. This will be the value of
`Queue.getQueueName()
`_ or
`Topic.getTopicName()
`_.",
"field": "name"
}],
"optional": true,
"name": "io.confluent.connect.jms.Destination",
"doc": "This schema is used to represent a JMS Destination, and is either
`queue `_ or `topic
`_.",
"field": "replyTo"
}, {
"type": "struct",
"fields": [{
"type": "string",
"optional": false,
"doc": "The type of JMS Destination, and either ``queue`` or ``topic``.",
"field": "destinationType"
}, {
"type": "string",
"optional": false,
"doc": "The name of the destination. This will be the value of
`Queue.getQueueName()
`_ or
`Topic.getTopicName()
`_.",
"field": "name"
}],
"optional": true,
"name": "io.confluent.connect.jms.Destination",
"doc": "This schema is used to represent a JMS Destination, and is either
`queue `_ or `topic
`_.",
"field": "destination"
}, {
"type": "boolean",
"optional": false,
"doc": "This field stores the value of `Message.getJMSRedelivered()
`_.",
"field": "redelivered"
}, {
"type": "string",
"optional": true,
"doc": "This field stores the value of `Message.getJMSType()
`_.",
"field": "type"
}, {
"type": "int64",
"optional": false,
"doc": "This field stores the value of `Message.getJMSExpiration()
`_.",
"field": "expiration"
}, {
"type": "int32",
"optional": false,
"doc": "This field stores the value of `Message.getJMSPriority()
`_.",
"field": "priority"
}, {
"type": "map",
"keys": {
"type": "string",
"optional": false
},
"values": {
"type": "struct",
"fields": [{
"type": "string",
"optional": false,
"doc": "The java type of the property on the Message. One of ``boolean``,
``byte``, ``short``, ``integer``, ``long``, ``float``, ``double``, or
``string``.",
"field": "propertyType"
},
[jira] [Updated] (KAFKA-7590) GETTING HUGE MESSAGE STRUCTURE THROUGH JMS CONNECTOR
[
https://issues.apache.org/jira/browse/KAFKA-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenchu Lakshman kumar updated KAFKA-7590:
--
Description:
Message
{"schema":{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"This field stores the value of
`Message.getJMSMessageID()
`_.","field":"messageID"}
,
{"type":"string","optional":false,"doc":"This field stores the type of message
that was received. This corresponds to the subinterfaces of `Message
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html]>`_. `BytesMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/BytesMessage.html]>`_ =
`bytes`, `MapMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/MapMessage.html]>`_ = `map`,
`ObjectMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/ObjectMessage.html]>`_ =
`object`, `StreamMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/StreamMessage.html]>`_ =
`stream` and `TextMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/TextMessage.html]>`_ = `text`.
The corresponding field will be populated with the values from the respective
Message subinterface.","field":"messageType"}
,
{"type":"int64","optional":false,"doc":"Data from the `getJMSTimestamp()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSTimestamp(])>`_
method.","field":"timestamp"}
,
{"type":"int32","optional":false,"doc":"This field stores the value of
`Message.getJMSDeliveryMode()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSDeliveryMode(])>`_.","field":"deliveryMode"}
,
{"type":"string","optional":true,"doc":"This field stores the value of
`Message.getJMSCorrelationID()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSCorrelationID(])>`_.","field":"correlationID"}
,{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"The type of JMS Destination, and
either ``queue`` or ``topic``.","field":"destinationType"}
,
{"type":"string","optional":false,"doc":"The name of the destination. This will
be the value of `Queue.getQueueName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html#getQueueName(])>`_
or `Topic.getTopicName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html#getTopicName(])>`_.","field":"name"}
],"optional":true,"name":"io.confluent.connect.jms.Destination","doc":"This
schema is used to represent a JMS Destination, and is either `queue
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html]>`_ or `topic
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html]>`_.","field":"replyTo"},{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"The type of JMS Destination, and
either ``queue`` or ``topic``.","field":"destinationType"}
,
{"type":"string","optional":false,"doc":"The name of the destination. This will
be the value of `Queue.getQueueName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html#getQueueName(])>`_
or `Topic.getTopicName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html#getTopicName(])>`_.","field":"name"}
],"optional":true,"name":"io.confluent.connect.jms.Destination","doc":"This
schema is used to represent a JMS Destination, and is either `queue
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html]>`_ or `topic
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html]>`_.","field":"destination"},
{"type":"boolean","optional":false,"doc":"This field stores the value of
`Message.getJMSRedelivered()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSRedelivered(])>`_.","field":"redelivered"}
,
{"type":"string","optional":true,"doc":"This field stores the value of
`Message.getJMSType()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSType(])>`_.","field":"type"}
,
{"type":"int64","optional":false,"doc":"This field stores the value of
`Message.getJMSExpiration()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSExpiration(])>`_.","field":"expiration"}
,
{"type":"int32","optional":false,"doc":"This field stores the value of
`Message.getJMSPriority()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSPriority(])>`_.","field":"priority"}
,{"type":"map","keys":
{"type":"string","optional":false}
,"values":{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"The java type of the property on the
Message. One of ``boolean``, ``byte``, ``short``, ``integer``, ``long``,
``float``, ``double``, or ``string``.","field":"propertyType"}
,\{"type":"boolean","optional":true,"doc":"The value stored as a boolean. Null
unless ``propertyType`` is set to
``boolean``.","field":"boolean"},\{"type":"int8","optional":true,"doc":"The
value stored as a byte. Null unless
[jira] [Resolved] (KAFKA-7590) GETTING HUGE MESSAGE STRUCTURE THROUGH JMS CONNECTOR
[
https://issues.apache.org/jira/browse/KAFKA-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenchu Lakshman kumar resolved KAFKA-7590.
---
Resolution: Fixed
> GETTING HUGE MESSAGE STRUCTURE THROUGH JMS CONNECTOR
>
>
> Key: KAFKA-7590
> URL: https://issues.apache.org/jira/browse/KAFKA-7590
> Project: Kafka
> Issue Type: Test
> Components: config, KafkaConnect
>Affects Versions: 2.0.0
>Reporter: Chenchu Lakshman kumar
>Priority: Major
>
> Message
>
> {"schema":{"type":"struct","fields":[
> {"type":"string","optional":false,"doc":"This field stores the value of
> `Message.getJMSMessageID()
> `_.","field":"messageID"}
> ,
> {"type":"string","optional":false,"doc":"This field stores the type of
> message that was received. This corresponds to the subinterfaces of `Message
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html]>`_.
> `BytesMessage
> <[http://docs.oracle.com/javaee/6/api/javax/jms/BytesMessage.html]>`_ =
> `bytes`, `MapMessage
> <[http://docs.oracle.com/javaee/6/api/javax/jms/MapMessage.html]>`_ = `map`,
> `ObjectMessage
> <[http://docs.oracle.com/javaee/6/api/javax/jms/ObjectMessage.html]>`_ =
> `object`, `StreamMessage
> <[http://docs.oracle.com/javaee/6/api/javax/jms/StreamMessage.html]>`_ =
> `stream` and `TextMessage
> <[http://docs.oracle.com/javaee/6/api/javax/jms/TextMessage.html]>`_ =
> `text`. The corresponding field will be populated with the values from the
> respective Message subinterface.","field":"messageType"}
> ,
> {"type":"int64","optional":false,"doc":"Data from the `getJMSTimestamp()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSTimestamp(])>`_
> method.","field":"timestamp"}
> ,
> {"type":"int32","optional":false,"doc":"This field stores the value of
> `Message.getJMSDeliveryMode()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSDeliveryMode(])>`_.","field":"deliveryMode"}
> ,
> {"type":"string","optional":true,"doc":"This field stores the value of
> `Message.getJMSCorrelationID()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSCorrelationID(])>`_.","field":"correlationID"}
> ,{"type":"struct","fields":[
> {"type":"string","optional":false,"doc":"The type of JMS Destination, and
> either ``queue`` or ``topic``.","field":"destinationType"}
> ,
> {"type":"string","optional":false,"doc":"The name of the destination. This
> will be the value of `Queue.getQueueName()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html#getQueueName(])>`_
> or `Topic.getTopicName()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html#getTopicName(])>`_.","field":"name"}
> ],"optional":true,"name":"io.confluent.connect.jms.Destination","doc":"This
> schema is used to represent a JMS Destination, and is either `queue
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html]>`_ or `topic
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html]>`_.","field":"replyTo"},{"type":"struct","fields":[
> {"type":"string","optional":false,"doc":"The type of JMS Destination, and
> either ``queue`` or ``topic``.","field":"destinationType"}
> ,
> {"type":"string","optional":false,"doc":"The name of the destination. This
> will be the value of `Queue.getQueueName()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html#getQueueName(])>`_
> or `Topic.getTopicName()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html#getTopicName(])>`_.","field":"name"}
> ],"optional":true,"name":"io.confluent.connect.jms.Destination","doc":"This
> schema is used to represent a JMS Destination, and is either `queue
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html]>`_ or `topic
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html]>`_.","field":"destination"},
> {"type":"boolean","optional":false,"doc":"This field stores the value of
> `Message.getJMSRedelivered()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSRedelivered(])>`_.","field":"redelivered"}
> ,
> {"type":"string","optional":true,"doc":"This field stores the value of
> `Message.getJMSType()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSType(])>`_.","field":"type"}
> ,
> {"type":"int64","optional":false,"doc":"This field stores the value of
> `Message.getJMSExpiration()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSExpiration(])>`_.","field":"expiration"}
> ,
> {"type":"int32","optional":false,"doc":"This field stores the value of
> `Message.getJMSPriority()
> <[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSPriority(])>`_.","field":"priority"}
>
[jira] [Updated] (KAFKA-7590) GETTING HUGE MESSAGE STRUCTURE THROUGH JMS CONNECTOR
[
https://issues.apache.org/jira/browse/KAFKA-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenchu Lakshman kumar updated KAFKA-7590:
--
Description:
Message
{"schema":{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"This field stores the value of
`Message.getJMSMessageID()
`_.","field":"messageID"}
,{"type":"string","optional":false,"doc":"This field stores the type of message
that was received. This corresponds to the subinterfaces of `Message
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html]>`_. `BytesMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/BytesMessage.html]>`_ =
`bytes`, `MapMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/MapMessage.html]>`_ = `map`,
`ObjectMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/ObjectMessage.html]>`_ =
`object`, `StreamMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/StreamMessage.html]>`_ =
`stream` and `TextMessage
<[http://docs.oracle.com/javaee/6/api/javax/jms/TextMessage.html]>`_ = `text`.
The corresponding field will be populated with the values from the respective
Message
subinterface.","field":"messageType"},{"type":"int64","optional":false,"doc":"Data
from the `getJMSTimestamp()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSTimestamp(])>`_
method.","field":"timestamp"},{"type":"int32","optional":false,"doc":"This
field stores the value of `Message.getJMSDeliveryMode()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSDeliveryMode(])>`_.","field":"deliveryMode"},{"type":"string","optional":true,"doc":"This
field stores the value of `Message.getJMSCorrelationID()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSCorrelationID(])>`_.","field":"correlationID"},{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"The type of JMS Destination, and
either ``queue`` or ``topic``.","field":"destinationType"}
,{"type":"string","optional":false,"doc":"The name of the destination. This
will be the value of `Queue.getQueueName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html#getQueueName(])>`_
or `Topic.getTopicName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html#getTopicName(])>`_.","field":"name"}],"optional":true,"name":"io.confluent.connect.jms.Destination","doc":"This
schema is used to represent a JMS Destination, and is either `queue
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html]>`_ or `topic
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html]>`_.","field":"replyTo"},{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"The type of JMS Destination, and
either ``queue`` or ``topic``.","field":"destinationType"}
,{"type":"string","optional":false,"doc":"The name of the destination. This
will be the value of `Queue.getQueueName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html#getQueueName(])>`_
or `Topic.getTopicName()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html#getTopicName(])>`_.","field":"name"}],"optional":true,"name":"io.confluent.connect.jms.Destination","doc":"This
schema is used to represent a JMS Destination, and is either `queue
<[http://docs.oracle.com/javaee/6/api/javax/jms/Queue.html]>`_ or `topic
<[http://docs.oracle.com/javaee/6/api/javax/jms/Topic.html]>`_.","field":"destination"},{"type":"boolean","optional":false,"doc":"This
field stores the value of `Message.getJMSRedelivered()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSRedelivered(])>`_.","field":"redelivered"},{"type":"string","optional":true,"doc":"This
field stores the value of `Message.getJMSType()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSType(])>`_.","field":"type"},{"type":"int64","optional":false,"doc":"This
field stores the value of `Message.getJMSExpiration()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSExpiration(])>`_.","field":"expiration"},{"type":"int32","optional":false,"doc":"This
field stores the value of `Message.getJMSPriority()
<[http://docs.oracle.com/javaee/6/api/javax/jms/Message.html#getJMSPriority(])>`_.","field":"priority"},{"type":"map","keys":
{"type":"string","optional":false}
,"values":{"type":"struct","fields":[
{"type":"string","optional":false,"doc":"The java type of the property on the
Message. One of ``boolean``, ``byte``, ``short``, ``integer``, ``long``,
``float``, ``double``, or ``string``.","field":"propertyType"}
,\{"type":"boolean","optional":true,"doc":"The value stored as a boolean. Null
unless ``propertyType`` is set to
``boolean``.","field":"boolean"},\{"type":"int8","optional":true,"doc":"The
value stored as a byte. Null unless ``propertyType`` is set to
