Those files in confluent platform are under the $CONFLUENT_PLATFORM_HOME/etc/kafka
in particular: - connect-standalone.properties - connect-distributed.properties - producer.properties - consumer.properties Il giorno sab 6 mar 2021 alle ore 17:20 Andrea Cosentino <[email protected]> ha scritto: > I know what confluent platform is, it is just I'm not familiar with its > configuration files location. > > Strimzi is a project to deploy and manage a kafka cluster on top of > kubernetes/openshift > > Il sab 6 mar 2021, 17:10 Arundhati Bhende <[email protected]> > ha scritto: > >> Thanks. Yes, Confluent is some additional components on top of kafka >> components, I believe schema registry is one of those additional components >> that it provids , but it is still apache kafka. And unfortunately, I don't >> know what Strimzi is. Maybe, a wrapper on Kafka, similar to Confluent. >> >> Does anyone else have idea about where do I need to make these changes >> wrt Confluent? >> >> Regards >> >> On 3/6/21, 10:11 AM, "Andrea Cosentino" <[email protected]> wrote: >> >> I'm not expert of Confluent platform, I was talking about Apache >> Kafka. >> >> Il giorno sab 6 mar 2021 alle ore 15:54 Arundhati Bhende < >> [email protected]> ha scritto: >> >> > Thank you. >> > >> > It is a Confluent implementation ( Cluster ). >> > >> > So, do I need to make this change on all connect servers ? ( 2 in >> our >> > case ) >> > >> > And which files do I update? Somewhere I was reading we have to >> update 3 >> > files, connect properties, producer and consumer configs, but when >> I looked >> > I did not find the connect properties file on those seervers. >> > >> > Am I looking in the wrong place? >> > >> > Regards >> > >> > >> > On 3/6/21, 8:49 AM, "Andrea Cosentino" <[email protected]> wrote: >> > >> > Hello >> > >> > 1 yes, the max result poll number of files will be picked up >> > immediately. >> > >> > 2. If you are using the kafka connect in standalone mode, >> you'll need >> > to >> > modify the config/connect-standalone.properties and add the >> following >> > prop >> > producer.max.request.size and set the value >> > >> > In Strimzi you have to specify this property for the kafka >> connect >> > cluster. >> > >> > If it's a docker container you'll need to modify the connect >> conf as >> > well >> > >> > 3. Yes, it happens because the bucket contains files bigger >> than the >> > default size. >> > >> > >> > >> > Il sab 6 mar 2021, 14:13 Arundhati Bhende < >> > [email protected]> >> > ha scritto: >> > >> > > Hi, where can I find thee below information? Trying to >> > understand how >> > > the s3 source connector works and if I can make changes in the >> > config if I >> > > can't make changes on the server tp add max.request.sizee? >> > > >> > > Thanks >> > > >> > > >> > > On 3/4/21, 5:12 PM, "Arundhati Bhende" < >> > [email protected]> >> > > wrote: >> > > >> > > I am trying to set up a Camel S3 Source connector. The >> S3 bucket >> > > already exists. >> > > >> > > I have 2 questions >> > > >> > > 1. If the bucket already exists and has files in the >> > bucket, when >> > > we define the new connector, are those files immediately read >> into >> > the >> > > topic? >> > > >> > > 2. The error I am getting is >> > > >> > > The message is 57139013 bytes when serialized which >> is >> > larger >> > > than 1048576, which is the value of the max.request.size >> > configuration >> > > >> > > Where should I change the max.request.size Value? >> Do I >> > add that >> > > to the connector config? >> > > >> > > 3. Why am I getting this error - right after I >> create the >> > > connector. Is it because 1 happens and that the bucket has >> files >> > larger >> > > than the default size? >> > > >> > > Thank you >> > > Aru >> > > >> > > >> > > >> > > >> > >> > >> >>
