Re: Kafka mirror maker help

[Peter Bukowinski]

I run instances of Mirror Maker as supervisord tasks (http://supervisord.org 
). I’d recommend looking into it. In addition to 
letting you sidestep the service issue, supervisord watches the processes and 
can auto-restart them if they stop for any reason.

—
Peter Bukowinski

> On Apr 27, 2018, at 11:58 AM, Hans Jespersen  wrote:
> 
> Sorry I hit send a bit too soon. I was so focused on the systemd part of
> the email and not the Mirror Maker part.
> Confluent packages include Mirror Maker but the systemd scripts are setup
> to use Confluent Replicator rather than Mirror Maker.
> My apologies.
> 
> -hans
> 
> /**
> * Hans Jespersen, Director Systems Engineering, Confluent Inc.
> * h...@confluent.io (650)924-2670
> */
> 
> On Fri, Apr 27, 2018 at 11:56 AM, Hans Jespersen  wrote:
> 
>> The latest Confluent packages now ship with systemd scripts. That is since
>> Confluent Version 4.1 - which included Apache Kafka 1.1
>> 
>> -hans
>> 
>> /**
>> * Hans Jespersen, Director Systems Engineering, Confluent Inc.
>> * h...@confluent.io (650)924-2670
>> */
>> 
>> On Fri, Apr 27, 2018 at 11:15 AM, Andrew Otto  wrote:
>> 
>>> Hiya,
>>> 
>>> Saravanan, I saw you emailed my colleague Alex about WMF’s old debian
>>> packaging.  I’ll reply here.
>>> 
>>> We now use Confluent’s Kafka debian packaging which does not (or did not?)
>>> ship with init scripts.  We don’t use Sys V init.d scripts anymore either,
>>> but use systemd instead.  Our systemd service unit (ERB template format)
>>> is
>>> here:
>>> 
>>> https://github.com/wikimedia/puppet/blob/production/modules/
>>> confluent/templates/initscripts/kafka-mirror-instance.systemd.erb
>>> 
>>> 
>>> 
>>> On Fri, Apr 27, 2018 at 1:35 AM, Amrit Jangid 
>>> wrote:
>>> 
 You should share related info, such source-destination Kafka versions,
 sample Config or error if any.
 
 FYI,  Go through
 https://kafka.apache.org/documentation/#basic_ops_mirror_maker
 
>>> 
>> 
>> 

Re: Kafka mirror maker help

[Hans Jespersen]

Sorry I hit send a bit too soon. I was so focused on the systemd part of
the email and not the Mirror Maker part.
Confluent packages include Mirror Maker but the systemd scripts are setup
to use Confluent Replicator rather than Mirror Maker.
My apologies.

-hans

/**
 * Hans Jespersen, Director Systems Engineering, Confluent Inc.
 * h...@confluent.io (650)924-2670
 */

On Fri, Apr 27, 2018 at 11:56 AM, Hans Jespersen  wrote:

> The latest Confluent packages now ship with systemd scripts. That is since
> Confluent Version 4.1 - which included Apache Kafka 1.1
>
> -hans
>
> /**
>  * Hans Jespersen, Director Systems Engineering, Confluent Inc.
>  * h...@confluent.io (650)924-2670
>  */
>
> On Fri, Apr 27, 2018 at 11:15 AM, Andrew Otto  wrote:
>
>> Hiya,
>>
>> Saravanan, I saw you emailed my colleague Alex about WMF’s old debian
>> packaging.  I’ll reply here.
>>
>> We now use Confluent’s Kafka debian packaging which does not (or did not?)
>> ship with init scripts.  We don’t use Sys V init.d scripts anymore either,
>> but use systemd instead.  Our systemd service unit (ERB template format)
>> is
>> here:
>>
>> https://github.com/wikimedia/puppet/blob/production/modules/
>> confluent/templates/initscripts/kafka-mirror-instance.systemd.erb
>>
>>
>>
>> On Fri, Apr 27, 2018 at 1:35 AM, Amrit Jangid 
>> wrote:
>>
>> > You should share related info, such source-destination Kafka versions,
>> > sample Config or error if any.
>> >
>> > FYI,  Go through
>> > https://kafka.apache.org/documentation/#basic_ops_mirror_maker
>> >
>>
>
>

Re: Kafka mirror maker help

[Hans Jespersen]

The latest Confluent packages now ship with systemd scripts. That is since
Confluent Version 4.1 - which included Apache Kafka 1.1

-hans

/**
 * Hans Jespersen, Director Systems Engineering, Confluent Inc.
 * h...@confluent.io (650)924-2670
 */

On Fri, Apr 27, 2018 at 11:15 AM, Andrew Otto  wrote:

> Hiya,
>
> Saravanan, I saw you emailed my colleague Alex about WMF’s old debian
> packaging.  I’ll reply here.
>
> We now use Confluent’s Kafka debian packaging which does not (or did not?)
> ship with init scripts.  We don’t use Sys V init.d scripts anymore either,
> but use systemd instead.  Our systemd service unit (ERB template format) is
> here:
>
> https://github.com/wikimedia/puppet/blob/production/
> modules/confluent/templates/initscripts/kafka-mirror-instance.systemd.erb
>
>
>
> On Fri, Apr 27, 2018 at 1:35 AM, Amrit Jangid 
> wrote:
>
> > You should share related info, such source-destination Kafka versions,
> > sample Config or error if any.
> >
> > FYI,  Go through
> > https://kafka.apache.org/documentation/#basic_ops_mirror_maker
> >
>

Re: Kafka mirror maker help

[Andrew Otto]

Hiya,

Saravanan, I saw you emailed my colleague Alex about WMF’s old debian
packaging.  I’ll reply here.

We now use Confluent’s Kafka debian packaging which does not (or did not?)
ship with init scripts.  We don’t use Sys V init.d scripts anymore either,
but use systemd instead.  Our systemd service unit (ERB template format) is
here:

https://github.com/wikimedia/puppet/blob/production/modules/confluent/templates/initscripts/kafka-mirror-instance.systemd.erb



On Fri, Apr 27, 2018 at 1:35 AM, Amrit Jangid 
wrote:

> You should share related info, such source-destination Kafka versions,
> sample Config or error if any.
>
> FYI,  Go through
> https://kafka.apache.org/documentation/#basic_ops_mirror_maker
>

Re: Kafka mirror maker help

[Amrit Jangid]

You should share related info, such source-destination Kafka versions,
sample Config or error if any.

FYI,  Go through
https://kafka.apache.org/documentation/#basic_ops_mirror_maker