I've added a phab task for my proposed deprecation of the server-side upload queue -- if folks with shell access are interested in actively taking on what items in the queue are still accessible, please feel free to comment there and we can rescue and un-deprecate it. :)
https://phabricator.wikimedia.org/T391469 I'll also add links there to the specific uploader bugs I'm going to try to fix in the next few weeks, and client-side uploads of large files should become more reliable... It appears to be an issue with internal DB connection timeouts, as it's correctly sending an API error response. -- brooke On Tue, Apr 8, 2025 at 10:06 PM Brooke Vibber <[email protected]> wrote: > On Tue, Apr 8, 2025 at 5:37 PM p858snake <[email protected]> wrote: > >> One of the big issues with V2C is that it doesn't attempt to retry which >> causes a lot of the tasks to be filed: >> https://phabricator.wikimedia.org/T380982 >> > > *nod* Note that we can probably also improve the API uploader's handling > of stash publishing to greatly reduce the number of errors -- I'm going to > see what I can do this week while poking at misc patches, and then schedule > some more fixes for my maintenance/tech debt time. It looks like the most > common problem is a database timeout during assembly and publishing of the > final file, which can likely be resolved by keeping the transaction clean > and allowing the load balancer to shut down connections cleanly during > assembly. (We had similar problems on the transcoder system, which runs in > MediaWiki's job queue so connects to databases the same way.) > > There is also a task about Coreifying/Extensionifing V2C so its more >> closely intregrated: https://phabricator.wikimedia.org/T353659 >> > > I do hope we can push forward on some work here, either adapting > conversions or directly allowing upload. "Big" feature work will require > some dev time and will have to wait its turn with other projects; "small" > fixes could happen quickly if we can work out the administrivia. > > Personally I would recommend not hiding or destroying source files as > video2commons does, as this reduces visual quality and, when done at the > wrong bitrate, can drastically damage the video. If we're not legally > required to avoid distributing an original file, we probably should allow > the uploads directly on general principle. I have to warn that a conversion > that doesn't publish the original MP4 file as the actual, readable upload > file would require development time; simply enabling MP4 uploads is > something we could do at any time (with Legal's permission) by flipping a > config switch. > > Either way will require double-checking with legal, and getting Commons to > fully revert the 2014 MP4 RfC which resolved that we would not support MP4 > uploads, downloads, or conversion. (A 2019 RfC with much less participation > apparently said it was ok to convert but continues to forbid downloads. > Before implementing anything, conversion, upload, or download, we need to > pass it through Legal to ensure that *decoding* and *distributing* H.264 > files for free doesn't cause us any problems, and that MPEG-4 Visual and > AAC and AC3 audio codecs are clean as well. Note that some MP4 files > contain HEVC codec which is different from H.264 and may have different > legal issues still; we may still have to be able to distinguish them on > upload and only handle allowed ones.) > > -- brooke >
_______________________________________________ Wikimedia-l mailing list -- [email protected], guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/[email protected]/message/RJWUBD4MSUOGJCCVZTLJ7YNTWZTQKYVU/ To unsubscribe send an email to [email protected]
