Hello 1 yes, the max result poll number of files will be picked up immediately.
2. If you are using the kafka connect in standalone mode, you'll need to modify the config/connect-standalone.properties and add the following prop producer.max.request.size and set the value In Strimzi you have to specify this property for the kafka connect cluster. If it's a docker container you'll need to modify the connect conf as well 3. Yes, it happens because the bucket contains files bigger than the default size. Il sab 6 mar 2021, 14:13 Arundhati Bhende <[email protected]> ha scritto: > Hi, where can I find thee below information? Trying to understand how > the s3 source connector works and if I can make changes in the config if I > can't make changes on the server tp add max.request.sizee? > > Thanks > > > On 3/4/21, 5:12 PM, "Arundhati Bhende" <[email protected]> > wrote: > > I am trying to set up a Camel S3 Source connector. The S3 bucket > already exists. > > I have 2 questions > > 1. If the bucket already exists and has files in the bucket, when > we define the new connector, are those files immediately read into the > topic? > > 2. The error I am getting is > > The message is 57139013 bytes when serialized which is larger > than 1048576, which is the value of the max.request.size configuration > > Where should I change the max.request.size Value? Do I add that > to the connector config? > > 3. Why am I getting this error - right after I create the > connector. Is it because 1 happens and that the bucket has files larger > than the default size? > > Thank you > Aru > > > >
