nfsantos commented on code in PR #1716:
URL: https://github.com/apache/jackrabbit-oak/pull/1716#discussion_r1761059022
##########
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMongoDownloadTask.java:
##########
@@ -796,7 +800,7 @@ void download(FindIterable<NodeDocument> mongoIterable)
throws InterruptedExcept
this.lastIdDownloaded = id;
this.documentsDownloadedTotal++;
downloadStatics.incrementDocumentsDownloadedTotal();
- if (this.documentsDownloadedTotal % 20_000 == 0) {
+ if (this.documentsDownloadedTotal % 50_000 == 0) {
Review Comment:
We are inlining similar constants in other places. Maybe it is not a good
practise, but it's ok for the time being, I don't want to expose this as a
constant.
Additionally, the whole approach to logging progress should be reconsidered.
I no longer think that logging based on the number of items processed is the
best approach, as download speeds vary a lot from run to run, so we risk either
not logging often enough or logging way too frequently. A better approach is to
log periodically, so at a certain point in the future I would like to revise
the whole approach to logging progress in the indexing job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
