Thank you. It is a Confluent implementation ( Cluster ).
So, do I need to make this change on all connect servers ? ( 2 in our case ) And which files do I update? Somewhere I was reading we have to update 3 files, connect properties, producer and consumer configs, but when I looked I did not find the connect properties file on those seervers. Am I looking in the wrong place? Regards On 3/6/21, 8:49 AM, "Andrea Cosentino" <[email protected]> wrote: Hello 1 yes, the max result poll number of files will be picked up immediately. 2. If you are using the kafka connect in standalone mode, you'll need to modify the config/connect-standalone.properties and add the following prop producer.max.request.size and set the value In Strimzi you have to specify this property for the kafka connect cluster. If it's a docker container you'll need to modify the connect conf as well 3. Yes, it happens because the bucket contains files bigger than the default size. Il sab 6 mar 2021, 14:13 Arundhati Bhende <[email protected]> ha scritto: > Hi, where can I find thee below information? Trying to understand how > the s3 source connector works and if I can make changes in the config if I > can't make changes on the server tp add max.request.sizee? > > Thanks > > > On 3/4/21, 5:12 PM, "Arundhati Bhende" <[email protected]> > wrote: > > I am trying to set up a Camel S3 Source connector. The S3 bucket > already exists. > > I have 2 questions > > 1. If the bucket already exists and has files in the bucket, when > we define the new connector, are those files immediately read into the > topic? > > 2. The error I am getting is > > The message is 57139013 bytes when serialized which is larger > than 1048576, which is the value of the max.request.size configuration > > Where should I change the max.request.size Value? Do I add that > to the connector config? > > 3. Why am I getting this error - right after I create the > connector. Is it because 1 happens and that the bucket has files larger > than the default size? > > Thank you > Aru > > > >
