Dan-nl has uploaded a new change for review.
https://gerrit.wikimedia.org/r/100617
Change subject: job-queue-delay
......................................................................
job-queue-delay
we ran into an issue when the number of metadata records was ~3000. the job
queue
throttle that is built into the extension was exceeded. there were > 1000 media
file jobs in the queue and while the metadata job attempted to run 5 times the
media file jobs were still > 1000 and thus it failed to add its media file jobs
to the queue.
this change attempts to fix this issue by adding a delay via the
jobReleaseTimestamp
param to the metadata job if there are > 1000 media file jobs in the queue. the
current
delay is set to Config::$metadata_job_delay = '5 minutes'.
Change-Id: Iad99aca2251b09930c14017cabcc222ebdcd1e42
---
M includes/Config.php
M includes/Jobs/UploadMetadataJob.php
2 files changed, 18 insertions(+), 0 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/GWToolset
refs/changes/17/100617/1
diff --git a/includes/Config.php b/includes/Config.php
index 2702747..50160a5 100644
--- a/includes/Config.php
+++ b/includes/Config.php
@@ -115,6 +115,13 @@
public static $metadata_file_category = 'GWToolset_Metadata_Sets';
/**
+ * @var {string}
+ * used in conjunction with the job param jobReleaseTimestamp when the
job queue
+ * delayedJobsEnabled() is true, e.g., JobQueueRedis
+ */
+ public static $metadata_job_delay = '5 minutes';
+
+ /**
* @var {int}
* the maximum number of times the UploadMetadataJob will attempt to
add the same
* UploadMediafileJob’s to the job queue. this max is used when the
diff --git a/includes/Jobs/UploadMetadataJob.php
b/includes/Jobs/UploadMetadataJob.php
index 8d487e9..b4fcf65 100644
--- a/includes/Jobs/UploadMetadataJob.php
+++ b/includes/Jobs/UploadMetadataJob.php
@@ -90,6 +90,17 @@
)
);
+ $delayed_enabled =
+ JobQueueGroup::singleton()
+ ->get( 'gwtoolsetUploadMetadataJob' )
+ ->delayedJobsEnabled();
+
+ if ( $delayed_enabled ) {
+ $job->params['jobReleaseTimestamp'] = strtotime(
+ '+' . Utils::sanitizeString(
Config::$metadata_job_delay )
+ );
+ }
+
$result = JobQueueGroup::singleton()->push( $job );
}
--
To view, visit https://gerrit.wikimedia.org/r/100617
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iad99aca2251b09930c14017cabcc222ebdcd1e42
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/GWToolset
Gerrit-Branch: master
Gerrit-Owner: Dan-nl <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
