umehrot2 commented on issue #1847: URL: https://github.com/apache/hudi/issues/1847#issuecomment-664709899
@zuyanton yes `fs.s3.cse.enabled` is required for client side encryption to kick in. I wonder why you still have the `fs.s3.cse..kms.keyId` there. Also you don't use EmrFS consistent view right ? At this point I don't see a reason for `getLen()` taking time, since like @bvaradar mentioned its just cached when the FileStatus is created. However, I would still suggest trying by removing the unnecessary configurations that you have for EmrFS. Another thing that I would like you to do is enable EmrFS debug logs, by going to `/etc/spark/conf/log4j.properties` and add an entry with `DEBUG` log level for `com.amazon.ws.emr.hadoop.fs` namespace. This should give more information if there are any S3 calls being made during that time of 100ms. If it does not reveal anything, I will try to work with you internally to reproduce the issue. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
