Im 0.5 this idea atm as there isn’t such a page. So I can become more sold on this to become a +1, being a committer how would I go about creating such an ecosystem page within ActiveMQ Artemis site.
Cheers Mike Sent from my iPhone > On 27 Oct 2017, at 11:05, Martyn Taylor <[email protected]> wrote: > > Looking at the Kafka Connect example. I see that the integration > components are all maintained outside of the Kafka project. They're listed > here: https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem > > How about we do something similar and create a wiki/web page linking the > integration projects. I realise our website isn't the most accessible atm > but going forward this should be improved. > > On Thu, Oct 26, 2017 at 5:42 PM, Robbie Gemmell <[email protected]> > wrote: > >> The related PR is https://github.com/apache/activemq-artemis/pull/1607 >> for anyone not following along thus far, or later reference. >> >> My main view there was that such an extension would be better >> maintained seperate from the main broker repository+distribution for a >> variety of reasons. After that I didn't have a strong view on where it >> lives, but I would say that I don't actually see any issue at all with >> it being outwith the ActiveMQ project. To me thats just a key thing >> enabled by such plugin points existing, and not uncommon with other >> projects/components. >> >> Robbie >> >> On 26 October 2017 at 16:46, Michael André Pearce >> <[email protected]> wrote: >>> As noted on the PR part of the PR discussion is should Service >> Connectors exist with the broker code. >>> >>> Service Connectors / Extensions are ActiveMQ Artemis specific. >>> >>> I think having them within the project space is good so we can grow an >> eco system of some, people can just use will aid in having better broker >> adoption as tools people look for would be available if contributed. >>> >>> As stated above it seems the general consensus on the PR is though liked >> is the broker code base best for them to live? >>> >>> If we don’t wish them to exist with broker code so they can have their >> own release cycle, and don’t murky the water of the broker, I would like to >> discuss if we could make a sub project for these to live, and possibly >> propose a sub project be setup called “ActiveMQ Artemis Integrations” ? >>> >>> If setup I would also propose maybe moving the spring integration there >> also. >>> >>> Cheers >>> Mike >>> >>> >>> >>> >>> >>> >>> >>> Sent from my iPhone >>> >>>> On 6 Oct 2017, at 12:20, Michael André Pearce < >> [email protected]> wrote: >>>> >>>> Yes and no, it would be same as arguing that core bridge between >> artemis clusters could also be separated components. >>>> >>>> The idea is to effectively have similar to a core bridge but to kafka. >>>> >>>> Some key parts why are (and probably similar to why core bridge is >> implemented within the broker) >>>> >>>> Latency >>>> Extra processes and hops >>>> Out the box in broker support >>>> >>>> Cheers >>>> Mike >>>> >>>> >>>> Sent from my iPhone >>>> >>>>> On 6 Oct 2017, at 12:14, Andy Taylor <[email protected]> wrote: >>>>> >>>>> I think https://github.com/ppatierno/kafka-connect-amqp is an example >> of >>>>> this >>>>> >>>>> On 6 October 2017 at 12:07, Christopher Shannon < >>>>> [email protected]> wrote: >>>>> >>>>>> Why not use Kafka's Connect framework? Seems like building a source >>>>>> connector here would make the most sense. This is not to say I'm >> against >>>>>> the idea I'm just wondering the benefits of doing it from the Artemis >> side >>>>>> of things when Kafka already has a framework that can be used to >>>>>> import/export data. >>>>>> >>>>>> On Fri, Oct 6, 2017 at 6:16 AM, Michael André Pearce < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Andy, >>>>>>> >>>>>>> As you may be aware or not we run both activemq and kafka each has a >>>>>>> specific plus point to it, use the right tool for the right job. >>>>>>> >>>>>>> We use activemq as our feature rich, fully jms compliant and >> performant >>>>>>> broker. >>>>>>> >>>>>>> And use kafka more for post processing and data streaming, with >>>>>> historical >>>>>>> feed also. >>>>>>> >>>>>>> Idea here is that producers and consumers bind into our activemq >> where >>>>>> jms >>>>>>> api is wanted or needed, lower per message latencies etc. but then >> we can >>>>>>> bridge messages from a core queue into kafka so that the data can >> flow to >>>>>>> it as fast as possible and consumed from there also. >>>>>>> >>>>>>> Idea is to try make a hybrid taking best of both. >>>>>>> >>>>>>> Cheers >>>>>>> Mike >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>>> On 6 Oct 2017, at 09:37, Andy Taylor <[email protected]> >> wrote: >>>>>>>> >>>>>>>> Whats the use case here Michael, sounds like a cool idea and >> something >>>>>>> that >>>>>>>> has been mentioned by different people but I am yet to understand >> how >>>>>>> this >>>>>>>> could be used. >>>>>>>> >>>>>>>> On 6 October 2017 at 07:48, Michael André Pearce < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> >>>>>>>>> I am looking to contribute an artemis ServiceConnector that would >>>>>>> leverage >>>>>>>>> the existing service connector interface to bridge from a queue >> into a >>>>>>>>> kafka topic similar to bridge between artemis clusters. >>>>>>>>> >>>>>>>>> Idea is to make it configurable how to transform and encode the >>>>>> message >>>>>>> in >>>>>>>>> kafka but as amqp is becoming very well supported by artemis >> provide >>>>>> out >>>>>>>>> the box that the message be encoded in amqp byte protocol. >>>>>>>>> >>>>>>>>> It something I’ve been working on privately for a bit, but since >>>>>>> becoming >>>>>>>>> a committer here is really like to share it. >>>>>>>>> >>>>>>>>> I hope if I get a positive response here to get a branch ready >> soon, >>>>>> but >>>>>>>>> feedback if this is welcomed would be great. >>>>>>>>> >>>>>>>>> Cheers >>>>>>>>> Mike >>>>>>>>> >>>>>>>>> >>>>>>>>> Sent from my iPhone >>>>>>> >>>>>> >>
