I'm not expert of Confluent platform, I was talking about Apache Kafka. Il giorno sab 6 mar 2021 alle ore 15:54 Arundhati Bhende < [email protected]> ha scritto:
> Thank you. > > It is a Confluent implementation ( Cluster ). > > So, do I need to make this change on all connect servers ? ( 2 in our > case ) > > And which files do I update? Somewhere I was reading we have to update 3 > files, connect properties, producer and consumer configs, but when I looked > I did not find the connect properties file on those seervers. > > Am I looking in the wrong place? > > Regards > > > On 3/6/21, 8:49 AM, "Andrea Cosentino" <[email protected]> wrote: > > Hello > > 1 yes, the max result poll number of files will be picked up > immediately. > > 2. If you are using the kafka connect in standalone mode, you'll need > to > modify the config/connect-standalone.properties and add the following > prop > producer.max.request.size and set the value > > In Strimzi you have to specify this property for the kafka connect > cluster. > > If it's a docker container you'll need to modify the connect conf as > well > > 3. Yes, it happens because the bucket contains files bigger than the > default size. > > > > Il sab 6 mar 2021, 14:13 Arundhati Bhende < > [email protected]> > ha scritto: > > > Hi, where can I find thee below information? Trying to > understand how > > the s3 source connector works and if I can make changes in the > config if I > > can't make changes on the server tp add max.request.sizee? > > > > Thanks > > > > > > On 3/4/21, 5:12 PM, "Arundhati Bhende" < > [email protected]> > > wrote: > > > > I am trying to set up a Camel S3 Source connector. The S3 bucket > > already exists. > > > > I have 2 questions > > > > 1. If the bucket already exists and has files in the > bucket, when > > we define the new connector, are those files immediately read into > the > > topic? > > > > 2. The error I am getting is > > > > The message is 57139013 bytes when serialized which is > larger > > than 1048576, which is the value of the max.request.size > configuration > > > > Where should I change the max.request.size Value? Do I > add that > > to the connector config? > > > > 3. Why am I getting this error - right after I create the > > connector. Is it because 1 happens and that the bucket has files > larger > > than the default size? > > > > Thank you > > Aru > > > > > > > > > >
