Thanks Andrea.
On 3/7/21, 3:08 AM, "Andrea Cosentino" <[email protected]> wrote:
Those files in confluent platform are under the
$CONFLUENT_PLATFORM_HOME/etc/kafka
in particular:
- connect-standalone.properties
- connect-distributed.properties
- producer.properties
- consumer.properties
Il giorno sab 6 mar 2021 alle ore 17:20 Andrea Cosentino <[email protected]>
ha scritto:
> I know what confluent platform is, it is just I'm not familiar with its
> configuration files location.
>
> Strimzi is a project to deploy and manage a kafka cluster on top of
> kubernetes/openshift
>
> Il sab 6 mar 2021, 17:10 Arundhati Bhende
<[email protected]>
> ha scritto:
>
>> Thanks. Yes, Confluent is some additional components on top of kafka
>> components, I believe schema registry is one of those additional
components
>> that it provids , but it is still apache kafka. And unfortunately, I
don't
>> know what Strimzi is. Maybe, a wrapper on Kafka, similar to Confluent.
>>
>> Does anyone else have idea about where do I need to make these changes
>> wrt Confluent?
>>
>> Regards
>>
>> On 3/6/21, 10:11 AM, "Andrea Cosentino" <[email protected]> wrote:
>>
>> I'm not expert of Confluent platform, I was talking about Apache
>> Kafka.
>>
>> Il giorno sab 6 mar 2021 alle ore 15:54 Arundhati Bhende <
>> [email protected]> ha scritto:
>>
>> > Thank you.
>> >
>> > It is a Confluent implementation ( Cluster ).
>> >
>> > So, do I need to make this change on all connect servers ? ( 2 in
>> our
>> > case )
>> >
>> > And which files do I update? Somewhere I was reading we have to
>> update 3
>> > files, connect properties, producer and consumer configs, but when
>> I looked
>> > I did not find the connect properties file on those seervers.
>> >
>> > Am I looking in the wrong place?
>> >
>> > Regards
>> >
>> >
>> > On 3/6/21, 8:49 AM, "Andrea Cosentino" <[email protected]> wrote:
>> >
>> > Hello
>> >
>> > 1 yes, the max result poll number of files will be picked up
>> > immediately.
>> >
>> > 2. If you are using the kafka connect in standalone mode,
>> you'll need
>> > to
>> > modify the config/connect-standalone.properties and add the
>> following
>> > prop
>> > producer.max.request.size and set the value
>> >
>> > In Strimzi you have to specify this property for the kafka
>> connect
>> > cluster.
>> >
>> > If it's a docker container you'll need to modify the connect
>> conf as
>> > well
>> >
>> > 3. Yes, it happens because the bucket contains files bigger
>> than the
>> > default size.
>> >
>> >
>> >
>> > Il sab 6 mar 2021, 14:13 Arundhati Bhende <
>> > [email protected]>
>> > ha scritto:
>> >
>> > > Hi, where can I find thee below information? Trying to
>> > understand how
>> > > the s3 source connector works and if I can make changes in
the
>> > config if I
>> > > can't make changes on the server tp add max.request.sizee?
>> > >
>> > > Thanks
>> > >
>> > >
>> > > On 3/4/21, 5:12 PM, "Arundhati Bhende" <
>> > [email protected]>
>> > > wrote:
>> > >
>> > > I am trying to set up a Camel S3 Source connector. The
>> S3 bucket
>> > > already exists.
>> > >
>> > > I have 2 questions
>> > >
>> > > 1. If the bucket already exists and has files in the
>> > bucket, when
>> > > we define the new connector, are those files immediately read
>> into
>> > the
>> > > topic?
>> > >
>> > > 2. The error I am getting is
>> > >
>> > > The message is 57139013 bytes when serialized which
>> is
>> > larger
>> > > than 1048576, which is the value of the max.request.size
>> > configuration
>> > >
>> > > Where should I change the max.request.size Value?
>> Do I
>> > add that
>> > > to the connector config?
>> > >
>> > > 3. Why am I getting this error - right after I
>> create the
>> > > connector. Is it because 1 happens and that the bucket has
>> files
>> > larger
>> > > than the default size?
>> > >
>> > > Thank you
>> > > Aru
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>>
>>
