[merged thread]
Hi Bawolff and Daniel,
This explanation makes sense, as I'm running the same set of unit tests on 3
different environments, each environment runs in parallel with one another. And
I've just realized 20191116051316 is just a timestamp with accuracy up to
seconds XD
So for now I think I will just append a random suffix to the file name for this
unit-testing purpose. Thanks for your help!
Regards,
Xinyan
-Original Message [1/2] -
From: Wikitech-l On Behalf Of bawolff
Sent: Tuesday, November 19, 2019 9:37 AM
To: Wikimedia developers
Subject: Re: [Wikitech-l] The difference between fileexists-no-change and
backend-fail-alreadyexists for action=upload
So fileexists-no-change happens when your uploading a new version of a file,
MediaWiki calculates the sha1 sum, and notices that the img_sha1 field in the
db for the current version of the file is the same as what was just calculated
for the new file. Which probably (although not necessarily now that sha1 is
broken) means the file you're uploading is the same as the one already there.
backend-fail-alreadyexists basically means that MediaWiki was about to write a
file somewhere, but there already was a file at that place, and the
overwrite/overwriteSame flags were not set, so there should not have already
been a file in that place (The file already there, may or may not be the same
as the new file). Normally I would assume that something more frontend-y would
give an error before this error would be encountered.
Perhaps it might be encountered during a race condition. This is mostly
speculation as I'm not super familiar with the FileBackend code.
--
bawolff
-Original Message [2/2] -
From: Daniel Kinzler
Sent: Monday, November 18, 2019 8:56 PM
To: Chen Xinyan
Subject: Re: [Wikitech-l] The difference between fileexists-no-change and
backend-fail-alreadyexists for action=upload
Hi Xinyan!
I may be wrong about this, but from my understanding backend-fail-alreadyexists
is an unexpected internal error that indicates a conflict in the storage
backend. My best guess is a problem with the granularity of the timestamp used
to construct the filename - I could imagine that, if you manage to upload two
versions of a file within the same second, the backend fails because the
versions are distinguished only by timestamp (20191116051316 is 2019-11-16
05:13:16). An alternative explanation might be a race conidtion triggered when
two processes are trying to upload a file at the same time - this should be
prevented by applying the appropriate locking, but who knows.
This is all just an "informed guess", my understanding on how the file backend
works is superficial.
HTH
Daniel
On Sat, Nov 16, 2019 at 6:51 AM Chen Xinyan wrote:
> Hi there,
>
> Hope I've found the correct list for asking this question.
>
> I was setting up a CI test case for my MediaWiki Client Library to run
> a bot that will upload a file to https://en.wikipedia.beta.wmflabs.org
> in chunked stash mode. For most of the time, when I perform the final
> upload with filekey parameter, I will receive a fileexists-no-change
> error from the MW API server. This is expected, as there is already a
> same file with the same title on the site. However, today I received
> backend-fail-alreadyexists. The error message looks like
>
> backend-fail-alreadyexists: The file
> "mwstore://local-swift-eqiad/local-public/archive/9/95/20191116051316!Test_image.jpg"
> already exists.
>
> Does this error usually occurs when user tries to upload the same file?
> What's the difference between fileexists-no-change and
> backend-fail-alreadyexists?
>
> Thanks,
> Xinyan
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
