Github user gemmellr commented on a diff in the pull request:

    https://github.com/apache/activemq-artemis/pull/1607#discussion_r146821818
  
    --- Diff: docs/user-manual/en/kafka-bridges.md ---
    @@ -0,0 +1,180 @@
    +# Apache ActiveMQ Kafka Bridge
    +
    +The function of a bridge is to consume messages from a source queue in 
Apache ActiveMQ Artemis, 
    +and forward them to a target topic, on a remote Apache Kafka server.
    +
    +By pairing Apache ActiveMQ Artemis and Apache Kafka with the bridge you 
could have a hybrid broker setup, 
    +having a data flow with CORE, AMQP, MQTT clients, as well as now Kafka 
clients also. 
    +Taking and giving the best features of both broker technologies when and 
where needed for a flow of data.
    +
    +![ActiveMQ Artemis Kafka Bridge 
Clients](images/activemq-kafka-bridge-clients.png)
    +
    +The intent is this will be a two way bridge, but currently the flow is a 
single direction 
    +from Apache ActiveMQ Artemis to Apache Kafka
    +
    +
    +The source and target servers are remote making bridging suitable
    +for reliably sending messages from one artemis cluster to kafka, 
    +for instance across a WAN, to the cloud, or internet and where the 
connection may be unreliable.
    +
    +The bridge has built in resilience to failure so if the target server
    +connection is lost, e.g. due to network failure, the bridge will retry
    +connecting to the target until it comes back online. When it comes back
    +online it will resume operation as normal.
    +
    +![ActiveMQ Artemis Kafka Bridge 
Diagram](images/activemq-kafka-bridge-diagram.png)
    +
    +In summary, Apache ActiveMQ Kafka Bridge is a way to reliably connect 
separate 
    +Apache ActiveMQ Artemis server and Apache Kafka server together.
    +
    +## Configuring Kakfa Bridges
    +
    +Bridges are configured in `broker.xml`.  
    +Let's kick off
    +with an example (this is actually from the kafka bridge test example):
    +
    +
    +    <connector-services>
    +         <connector-service name="my-kafka-bridge">
    +            
<factory-class>org.apache.activemq.artemis.integration.kafka.bridge.KafkaProducerBridgeFactory</factory-class>
    +            <param key="bootstrap.servers" 
value="kafka-1.domain.local:9092,kafka-2.domain.local:9092,kafka-3.domain.local:9092"
 />
    +            <param key="queue-name" value="my.artemis.queue" />
    +            <param key="kafka-topic" value="my_kafka_topic" />
    +         </connector-service>
    +    </connector-services>
    +
    +In the above example we have shown the required parameters to
    +configure for a kakfa bridge. See below for a complete list of available 
configuration options. 
    +
    +### Serialization
    +By default the CoreMessageSerializer is used.
    +
    +#### CoreMessageSerializer
    +Default but can be explicitly set using
    +           
    +    <param key="value.serializer" 
value="org.apache.activemq.artemis.integration.kafka.protocol.core.CoreMessageSerializer"
 />
    +
    +This maps the Message properties to Record headers.
    +And then maps the payload binary as the Record value, encoding TextMessage
    +
    +This makes it easy to consume from Kafka using default deserializers
    +TextMessage using StringDeserializer and
    +ByteMessage using BytesDeserializer.
    +
    +Also to note, if you serialized your objects using binary serialization 
like Apache Avro, Apache Thrift etc. into a byte array for the ByteMessage, 
    +as that byte array is preserved in the record value part as is, you could 
deserialize using an equivalent deserializer 
    +direct to back into Avro Record / Thrift Struct. 
    +
    +Also supplied are some Apache Kafka deserializers allowing you to consume 
from Apache Kafka 
    +and get a more familiar CoreMessage or JMSMessage, that your consumers can 
use:
    +
    
+`org.apache.activemq.artemis.integration.kafka.protocol.core.jms.CoreJmsMessageDeserializer`
    
+`org.apache.activemq.artemis.integration.kafka.protocol.core.CoreMessageDeserializer`
    +
    +You can get these Apache Kafka, Serializers/Deserializers via maven using 
the following GAV coordinates:
    +
    +    <depedency>
    +       <groupId>org.apache.activemq</groupId>
    +       <artifactId>artemis-kafka-core-protocol</artifactId>
    +       <version>2.4.0-SNAPSHOT</version>
    +    </depedency>
    +
    +
    +#### AMQPMessageSerializer
    +Can be set by using:
    +    
    +    <param key="value.serializer" 
value="org.apache.activemq.artemis.integration.kafka.protocol.amqp.AMQPMessageSerializer"
 />
    +
    +
    +This encodes the whole message into amqp binary protocol into the Record 
value.
    +
    +This is useful if you have something like
    +
    +   https://github.com/EnMasseProject/amqp-kafka-bridge
    +
    +As it allows messages to pump into kafka in amqp binary, to then consumed 
from kafka by clients using amqp binary protocol.
    +
    +Also provided are Apache Qpid Proton based Apache Kafka deserializers 
allowing you to consume from 
    +Apache Kafka and get a ProtonMessage.
    +
    
+`org.apache.activemq.artemis.integration.kafka.protocol.amqp.proton.ProtonMessageDeserializer`
    +
    +You can get these Apache Kafka, Serializers/Deserializers via maven using 
the following GAV coordinates:
    +
    +    <depedency>
    +       <groupId>org.apache.activemq</groupId>
    +       <artifactId>artemis-kafka-amqp-protocol</artifactId>
    +       <version>2.4.0-SNAPSHOT</version>
    --- End diff --
    
    As earlier version comment.


---

Reply via email to