quantranhong1999 commented on code in PR #1705:
URL: https://github.com/apache/james-project/pull/1705#discussion_r1308652706
##########
server/data/data-jmap/src/main/java/org/apache/james/jmap/api/upload/UploadServiceDefaultImpl.java:
##########
@@ -90,36 +88,26 @@ private Mono<Void> cleanupUpload(Username username, long
currentUploadUsage) {
long minimumSpaceToClean = currentUploadUsage -
jmapUploadQuotaConfiguration.getUploadQuotaLimitInBytes() / 2;
AtomicLong cleanedSpace = new AtomicLong(0L);
- if (shouldPerformReadRepair()) {
- return Flux.from(uploadRepository.listUploads(username))
- .sort(Comparator.comparing(UploadMetaData::uploadDate))
- .flatMap(upload -> {
- if (cleanedSpace.get() < minimumSpaceToClean) {
- return
Mono.from(uploadRepository.delete(upload.uploadId(), username))
-
.then(Mono.from(uploadUsageRepository.decreaseSpace(username,
QuotaSizeUsage.size(upload.sizeAsLong()))))
- .then(Mono.fromCallable(() ->
cleanedSpace.addAndGet(upload.sizeAsLong())))
- .then(Mono.empty());
- } else {
- return Mono.fromCallable(() -> upload);
- }
- }, LOW_CONCURRENCY)
- .map(UploadMetaData::sizeAsLong)
- .reduce(Long::sum)
- .filter(totalSpaceUsed -> totalSpaceUsed != currentUploadUsage)
- .flatMap(totalSpaceUsed ->
Mono.from(uploadUsageRepository.resetSpace(username,
- QuotaSizeUsage.size(totalSpaceUsed))))
- .then();
- } else {
- return Flux.from(uploadRepository.listUploads(username))
- .sort(Comparator.comparing(UploadMetaData::uploadDate))
- .takeWhile(upload ->
cleanedSpace.addAndGet(upload.sizeAsLong()) <= minimumSpaceToClean)
- .flatMap(upload ->
Mono.from(uploadRepository.delete(upload.uploadId(), username))
-
.then(Mono.from(uploadUsageRepository.decreaseSpace(username,
QuotaSizeUsage.size(upload.sizeAsLong())))))
- .then();
- }
+ return Flux.from(uploadRepository.listUploads(username))
+ .sort(Comparator.comparing(UploadMetaData::uploadDate))
+ .flatMap(deleteUpload(username, minimumSpaceToClean,
cleanedSpace), LOW_CONCURRENCY)
+ .map(UploadMetaData::sizeAsLong)
+ .reduce(Long::sum)
+ .filter(totalSpaceUsed -> totalSpaceUsed != currentUploadUsage)
+ .flatMap(totalSpaceUsed ->
Mono.from(uploadUsageRepository.resetSpace(username,
QuotaSizeUsage.size(totalSpaceUsed))))
Review Comment:
The current code should already "call resetSpace only one time after the all
deleteUpload". cf: a lot of `.then` (after `.delete`) and a final `.reduce`.
##########
server/data/data-jmap/src/main/java/org/apache/james/jmap/api/upload/UploadServiceDefaultImpl.java:
##########
@@ -90,36 +88,26 @@ private Mono<Void> cleanupUpload(Username username, long
currentUploadUsage) {
long minimumSpaceToClean = currentUploadUsage -
jmapUploadQuotaConfiguration.getUploadQuotaLimitInBytes() / 2;
AtomicLong cleanedSpace = new AtomicLong(0L);
- if (shouldPerformReadRepair()) {
- return Flux.from(uploadRepository.listUploads(username))
- .sort(Comparator.comparing(UploadMetaData::uploadDate))
- .flatMap(upload -> {
- if (cleanedSpace.get() < minimumSpaceToClean) {
- return
Mono.from(uploadRepository.delete(upload.uploadId(), username))
-
.then(Mono.from(uploadUsageRepository.decreaseSpace(username,
QuotaSizeUsage.size(upload.sizeAsLong()))))
- .then(Mono.fromCallable(() ->
cleanedSpace.addAndGet(upload.sizeAsLong())))
- .then(Mono.empty());
- } else {
- return Mono.fromCallable(() -> upload);
- }
- }, LOW_CONCURRENCY)
- .map(UploadMetaData::sizeAsLong)
- .reduce(Long::sum)
- .filter(totalSpaceUsed -> totalSpaceUsed != currentUploadUsage)
- .flatMap(totalSpaceUsed ->
Mono.from(uploadUsageRepository.resetSpace(username,
- QuotaSizeUsage.size(totalSpaceUsed))))
- .then();
- } else {
- return Flux.from(uploadRepository.listUploads(username))
- .sort(Comparator.comparing(UploadMetaData::uploadDate))
- .takeWhile(upload ->
cleanedSpace.addAndGet(upload.sizeAsLong()) <= minimumSpaceToClean)
- .flatMap(upload ->
Mono.from(uploadRepository.delete(upload.uploadId(), username))
-
.then(Mono.from(uploadUsageRepository.decreaseSpace(username,
QuotaSizeUsage.size(upload.sizeAsLong())))))
- .then();
- }
+ return Flux.from(uploadRepository.listUploads(username))
+ .sort(Comparator.comparing(UploadMetaData::uploadDate))
+ .flatMap(deleteUpload(username, minimumSpaceToClean,
cleanedSpace), LOW_CONCURRENCY)
Review Comment:
Yes, that is why I use `flatMap` with `LOW_CONCURRENCY` to reduce the risk.
I think most of the case users will have multiple small upload files, I
expect leveraging a bit of concurrency here would help clean up uploads faster
(even it is async already), and may result in the actual cleaned space
exceeding only a bit of the `minimumSpaceToClean`.
But yes there is a "rare" case that users use some big files, which could
lead to your case.
I am OK to use `concatMap`, but let's see other opinions on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
