Brion VIBBER has uploaded a new change for review.
https://gerrit.wikimedia.org/r/286381
Change subject: Shut down database connections during transcode shell-outs
......................................................................
Shut down database connections during transcode shell-outs
Transcoding via ffmpeg/ffmpeg2theora can be quite slow, especially
on very large video files it may well take many minutes or even a
couple hours.
Shut down the database connections in the transcode jobs to avoid
leaving long-running connections or transactions which can cause
trouble.
This is done by calling LBFactory::shutdownInstance(), which closes
out all connections to all databases. When we need to load another
database connection for further work after the transcode is done,
we just call wfGetDB() and it dives down to the LBFactory behind
the scenes to reconnect.
Bug: T97641
Change-Id: I935ca8ec465542a12016924dff6f73eb91df8f8d
---
M WebVideoTranscode/WebVideoTranscodeJob.php
1 file changed, 12 insertions(+), 0 deletions(-)
git pull
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/TimedMediaHandler
refs/changes/81/286381/1
diff --git a/WebVideoTranscode/WebVideoTranscodeJob.php
b/WebVideoTranscode/WebVideoTranscodeJob.php
index 78ae728..dcd8e11 100644
--- a/WebVideoTranscode/WebVideoTranscodeJob.php
+++ b/WebVideoTranscode/WebVideoTranscodeJob.php
@@ -180,6 +180,10 @@
// Avoid contention and "server has gone away" errors as
// the transcode will take a very long time in some cases
wfGetLBFactory()->commitAll( __METHOD__ );
+ // We can't just leave the connection open either or it will
+ // eat up resources and block new connections, so make sure
+ // everything is dead and gone.
+ LBFactory::destroyInstance();
// Check the codec see which encode method to call;
if ( isset( $options[ 'novideo' ] ) ) {
@@ -208,6 +212,9 @@
// Remove any log files,
// all useful info should be in status and or we are done with
2 passs encoding
$this->removeFfmpegLogFiles();
+
+ // Reconnect to the database...
+ $dbw = wfGetDB( DB_MASTER );
// Do a quick check to confirm the job was not restarted or
removed while we were transcoding
// Confirm that the in memory $jobStartTimeCache matches db
start time
@@ -253,6 +260,7 @@
// Avoid "server has gone away" errors as copying can
be slow
wfGetLBFactory()->commitAll( __METHOD__ );
+ LBFactory::destroyInstance();
// Copy derivative from the FS into storage at
$finalDerivativeFilePath
$result = $file->getRepo()->quickImport(
@@ -260,6 +268,7 @@
WebVideoTranscode::getDerivativeFilePath(
$file, $transcodeKey ), // storage
$storeOptions
);
+
if ( !$result->isOK() ) {
// no need to invalidate all pages with video.
// Because all pages remain valid ( no
$transcodeKey derivative )
@@ -272,6 +281,8 @@
intval( filesize(
$this->getTargetEncodePath() ) / $file->getLength() ) * 8
);
// wfRestoreWarnings();
+ // Reconnect to the database...
+ $dbw = wfGetDB( DB_MASTER );
// Update the transcode table with success time:
$dbw->update(
'transcode',
@@ -288,6 +299,7 @@
);
// Commit to reduce contention
$dbw->commit( __METHOD__, 'flush' );
+ LBFactory::destroyInstance();
WebVideoTranscode::invalidatePagesWithFile(
$this->title );
}
} else {
--
To view, visit https://gerrit.wikimedia.org/r/286381
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I935ca8ec465542a12016924dff6f73eb91df8f8d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/TimedMediaHandler
Gerrit-Branch: master
Gerrit-Owner: Brion VIBBER <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
