[merged thread]

Hi Bawolff and Daniel,

This explanation makes sense, as I'm running the same set of unit tests on 3 
different environments, each environment runs in parallel with one another. And 
I've just realized 20191116051316 is just a timestamp with accuracy up to 
seconds XD

So for now I think I will just append a random suffix to the file name for this 
unit-testing purpose. Thanks for your help!

Regards,
Xinyan

-----Original Message [1/2] -----
From: Wikitech-l <wikitech-l-boun...@lists.wikimedia.org> On Behalf Of bawolff
Sent: Tuesday, November 19, 2019 9:37 AM
To: Wikimedia developers <wikitech-l@lists.wikimedia.org>
Subject: Re: [Wikitech-l] The difference between fileexists-no-change and 
backend-fail-alreadyexists for action=upload

So fileexists-no-change happens when your uploading a new version of a file, 
MediaWiki calculates the sha1 sum, and notices that the img_sha1 field in the 
db for the current version of the file is the same as what was just calculated 
for the new file. Which probably (although not necessarily now that sha1 is 
broken) means the file you're uploading is the same as the one already there.

backend-fail-alreadyexists basically means that MediaWiki was about to write a 
file somewhere, but there already was a file at that place, and the 
overwrite/overwriteSame flags were not set, so there should not have already 
been a file in that place (The file already there, may or may not be the same 
as the new file). Normally I would assume that something more frontend-y would 
give an error before this error would be encountered.
Perhaps it might be encountered during a race condition. This is mostly 
speculation as I'm not super familiar with the FileBackend code.

--
bawolff

-----Original Message [2/2] -----
From: Daniel Kinzler <dkinz...@wikimedia.org> 
Sent: Monday, November 18, 2019 8:56 PM
To: Chen Xinyan <cxueso...@outlook.com>
Subject: Re: [Wikitech-l] The difference between fileexists-no-change and 
backend-fail-alreadyexists for action=upload

Hi Xinyan!

I may be wrong about this, but from my understanding backend-fail-alreadyexists 
is an unexpected internal error that indicates a conflict in the storage 
backend. My best guess is a problem with the granularity of the timestamp used 
to construct the filename - I could imagine that, if you manage to upload two 
versions of a file within the same second, the backend fails because the 
versions are distinguished only by timestamp (20191116051316 is 2019-11-16 
05:13:16). An alternative explanation might be a race conidtion triggered when 
two processes are trying to upload a file at the same time - this should be 
prevented by applying the appropriate locking, but who knows.

This is all just an "informed guess", my understanding on how the file backend 
works is superficial.

HTH
Daniel

On Sat, Nov 16, 2019 at 6:51 AM Chen Xinyan <cxueso...@outlook.com> wrote:

> Hi there,
>
> Hope I've found the correct list for asking this question.
>
> I was setting up a CI test case for my MediaWiki Client Library to run 
> a bot that will upload a file to https://en.wikipedia.beta.wmflabs.org 
> in chunked stash mode. For most of the time, when I perform the final 
> upload with filekey parameter, I will receive a fileexists-no-change 
> error from the MW API server. This is expected, as there is already a 
> same file with the same title on the site. However, today I received 
> backend-fail-alreadyexists. The error message looks like
>
> backend-fail-alreadyexists: The file
> "mwstore://local-swift-eqiad/local-public/archive/9/95/20191116051316!Test_image.jpg"
> already exists.
>
> Does this error usually occurs when user tries to upload the same file?
> What's the difference between fileexists-no-change and 
> backend-fail-alreadyexists?
>
> Thanks,
> Xinyan
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Reply via email to