Github user gemmellr commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/1607#discussion_r146824495 --- Diff: docs/user-manual/en/kafka-bridges.md --- @@ -0,0 +1,180 @@ +# Apache ActiveMQ Kafka Bridge + +The function of a bridge is to consume messages from a source queue in Apache ActiveMQ Artemis, +and forward them to a target topic, on a remote Apache Kafka server. + +By pairing Apache ActiveMQ Artemis and Apache Kafka with the bridge you could have a hybrid broker setup, +having a data flow with CORE, AMQP, MQTT clients, as well as now Kafka clients also. +Taking and giving the best features of both broker technologies when and where needed for a flow of data. + +![ActiveMQ Artemis Kafka Bridge Clients](images/activemq-kafka-bridge-clients.png) + +The intent is this will be a two way bridge, but currently the flow is a single direction +from Apache ActiveMQ Artemis to Apache Kafka + + +The source and target servers are remote making bridging suitable +for reliably sending messages from one artemis cluster to kafka, +for instance across a WAN, to the cloud, or internet and where the connection may be unreliable. + +The bridge has built in resilience to failure so if the target server +connection is lost, e.g. due to network failure, the bridge will retry +connecting to the target until it comes back online. When it comes back +online it will resume operation as normal. + +![ActiveMQ Artemis Kafka Bridge Diagram](images/activemq-kafka-bridge-diagram.png) + +In summary, Apache ActiveMQ Kafka Bridge is a way to reliably connect separate +Apache ActiveMQ Artemis server and Apache Kafka server together. + +## Configuring Kakfa Bridges + +Bridges are configured in `broker.xml`. +Let's kick off +with an example (this is actually from the kafka bridge test example): + + + <connector-services> + <connector-service name="my-kafka-bridge"> + <factory-class>org.apache.activemq.artemis.integration.kafka.bridge.KafkaProducerBridgeFactory</factory-class> + <param key="bootstrap.servers" value="kafka-1.domain.local:9092,kafka-2.domain.local:9092,kafka-3.domain.local:9092" /> + <param key="queue-name" value="my.artemis.queue" /> + <param key="kafka-topic" value="my_kafka_topic" /> + </connector-service> + </connector-services> + +In the above example we have shown the required parameters to +configure for a kakfa bridge. See below for a complete list of available configuration options. + +### Serialization +By default the CoreMessageSerializer is used. + +#### CoreMessageSerializer +Default but can be explicitly set using + + <param key="value.serializer" value="org.apache.activemq.artemis.integration.kafka.protocol.core.CoreMessageSerializer" /> + +This maps the Message properties to Record headers. +And then maps the payload binary as the Record value, encoding TextMessage + +This makes it easy to consume from Kafka using default deserializers +TextMessage using StringDeserializer and +ByteMessage using BytesDeserializer. + +Also to note, if you serialized your objects using binary serialization like Apache Avro, Apache Thrift etc. into a byte array for the ByteMessage, +as that byte array is preserved in the record value part as is, you could deserialize using an equivalent deserializer +direct to back into Avro Record / Thrift Struct. + +Also supplied are some Apache Kafka deserializers allowing you to consume from Apache Kafka +and get a more familiar CoreMessage or JMSMessage, that your consumers can use: + +`org.apache.activemq.artemis.integration.kafka.protocol.core.jms.CoreJmsMessageDeserializer` +`org.apache.activemq.artemis.integration.kafka.protocol.core.CoreMessageDeserializer` + +You can get these Apache Kafka, Serializers/Deserializers via maven using the following GAV coordinates: + + <depedency> + <groupId>org.apache.activemq</groupId> + <artifactId>artemis-kafka-core-protocol</artifactId> + <version>2.4.0-SNAPSHOT</version> + </depedency> + + +#### AMQPMessageSerializer +Can be set by using: + + <param key="value.serializer" value="org.apache.activemq.artemis.integration.kafka.protocol.amqp.AMQPMessageSerializer" /> + + +This encodes the whole message into amqp binary protocol into the Record value. --- End diff -- Various instances of 'amqp', such as those here, should probably be AMQP instead.
---