Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On Wed, Dec 14, 2011 at 11:35:24AM +1100, Tim Starling wrote: On 13/12/11 01:36, Teofilo wrote: Let us stop asking users to individually tag every wrong picture! Let us have some developers create a tool to find wrong pictures and rotate them back to their original orientation! We could make a list of all images with EXIF rotation. I'm not sure how you would separate that into correctly-rotated and incorrectly-rotated images. There's not any simple way to tell whether a picture is sideways. I wonder... if we run/simulate the old routine vs the new routine, and we notice that there is a difference in outcome between the two, we could add a check me template. scratches head sincerely, Kim We Are The Robots Bruning ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On 20/12/11 12:50, Kim Bruning wrote: On Wed, Dec 14, 2011 at 11:35:24AM +1100, Tim Starling wrote: On 13/12/11 01:36, Teofilo wrote: Let us stop asking users to individually tag every wrong picture! Let us have some developers create a tool to find wrong pictures and rotate them back to their original orientation! We could make a list of all images with EXIF rotation. I'm not sure how you would separate that into correctly-rotated and incorrectly-rotated images. There's not any simple way to tell whether a picture is sideways. I wonder... if we run/simulate the old routine vs the new routine, and we notice that there is a difference in outcome between the two, we could add a check me template. scratches head Every image with EXIF rotation will be different between the old and the new version of MediaWiki. A Commons user (Umherirrender) already generated a list of such images, using the Toolserver database, and thousands of incorrect images were tagged for rotation. The rate at which images were tagged was far in excess of the rate at which RotateBot was able to rotate them, so Sam Reed did some work on optimising it. -- Tim Starling ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On Mon, Dec 12, 2011 at 9:52 PM, David Richfield davidrichfi...@gmail.com wrote: What effect would a less aggressive tone have had? Would you have been more likely to convince your audience? less likely to alienate people? It's a fair point. I think part of the problem is that people are feeling that reasonable, calm, friendly inquiries are likely to be ignored and making noise is necessary to get attention. I want to make sure we do our best to respond to reasonable inquiries in a timely manner, and would ask all WMF staff and contractors to help me in that regard. In general, if you feel that an engineering issue merits escalation, never hesitate to email me directly and, unless I'm totally swamped, I'll try to help. There are other folks whose job it is to help with triaging, like Mark Hershberger (mah at wikimedia dot org) and Sumana Harihareswara (sumanah at wikimedia dot org, especially for things like patch review), and of course you can also contact any of the engineering directors for tech issues, raise them on IRC, on Bugzilla, etc. It's true that sometimes people complaining loudly helps us to take an issue more seriously, but ideally that shouldn't be necessary and our processes should work to understand what's causing pain and what isn't. -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On 13/12/11 02:55, David Gerard wrote: On 12 December 2011 15:26, K. Peachey p858sn...@gmail.com wrote: It's been a requested feature for a while, Someone finally got around to writing it (I believe it needed the Improved metadata handling backend first) and implementing it, It wasn't a sudden oh lets write this and enable it in one day thing, a lot of work went into it and subsequent testing. * How many existing uploads, used on the wikis, were previously wrongly rotated and were fixed by the feature? * How many existing uploads, used on the wikis, were previously correctly rotated and were messed up by the feature? i.e., was there strong reason to apply it to past images, not just new ones? Such statistics were never gathered. I was told by the developers involved that existing images with EXIF rotation would be very rare and that most of them would be fixed by this feature, and I didn't challenge that. I think it's too early to focus on recriminations, we risk distracting people from actually fixing the issue. -- Tim Starling ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On Tue, Dec 13, 2011 at 12:36 AM, Teofilo teofilow...@gmail.com wrote: 1 - Bug or feature ? It is a bug. ... snip ... It is somehow intentional, because it seems that the devs have suddenly decided that the exif orientation tag should be taken into account, while in the past users used had to use other ways to define image orientation. It's been a requested feature for a while, Someone finally got around to writing it (I believe it needed the Improved metadata handling backend first) and implementing it, It wasn't a sudden oh lets write this and enable it in one day thing, a lot of work went into it and subsequent testing. But even if it is intentional, we should call it a bug, because it is annoying to a lot of readers and uploaders whose pictures have been OK sometimes for years, and without warning they must suddenly change the orientation of their uploaded pictures. What about the pictures whose uploaders are no longer active ? So I hope everybody agrees that it is a bug. The bug I see is software people used to edit these images didn't fix the files metadata itself, thus in the end creating this situation 2 - The human bug I think the Wikimedia Foundation should present officially its excuses to the readers and active users annoyed by the bug. The excuses could be linked from the rotatebot template, so that the concerned users could read them. Excuses? The reasons why it's broken have been posted in many places, Last I checked the said template wasn't protected so anyone could and pointers to about why its happening. The devs should find out what went wrong in the decision process to implement the 1.18 version, and try to find preventive measures so that big problems of this size do not occur again when a version upgrade is done. Is it really OK not to consult the Commons community before changing a picture-related feature ? Nothing much went wrong in the planning of this feature, The metadata backend was improved, the rotation feature was written, the feature was tested (and i'm aware of this because I did test it) and the feature did work as intended. And why should commons be notified when a MediaWiki core feature is written, why not ja.wikipedia or en.wikinews? just because commons is a end user of the software doesn't make it all that special, While yes the choice to deploy it to the cluster could have been handled differently it worked from all the testing that was performed (and the issues that were found from the testing were fixed before it was pushed out). Had more end users actually bothered to test the pre release(s) when they were staged on test. and test2.wikipedia, issues like this might had stood out more prominently so that its feature could have been considered after being tested on a wider scale. 3 - The technical bug : deadline ...snip... Let us stop asking users to individually tag every wrong picture! Let us have some developers create a tool to find wrong pictures and rotate them back to their original orientation! I believe that can be done quiet easily with a DB query, Then it's just a matter of fixing the metadata attached in the file compared to actually re-rotating them again. We need a deadline. We need to be able to say, In X month's time, all pictures will be back to normal. A time line like that can't be given since there aren't plans to turn the feature off from my understanding, So this will conciebly be fixed when RotateBot fixes up the meta data on the files, Someone else does it, or a extension/feature is written so humans have a interface on-wiki to manually rotate the files to how they should be. -Peachey, Signing off on what is now a new day. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
The unrepentant attitude expressed above by K. Peachey increases the need for clear excuses from the Wikimedia Foundation, expressing clearly that something has gone wrong in the decision process, and that the people who think the relationship between users-community and developers the way K. Peachey is thinking, are mistaken. I don't want to address every single untruth included in K. Peachey's message. Let's say that when pictures are concerned, the input of the Commons community is useful, as is useful the input of the Georgian wikipedia when a Georgian-language-related feature is concerned. Let's say again that when users have been allowed for years - FOR YEARS - to upload pictures without concern for the exif orientation tag, revoking this allowance without prior warning is a breach of trust. And anyway, this is no reason to suddenly annoy readers, who are third parties in this developer-uploader misunderstanding and absence of dialogue. A Deadline is possible of course. All it needs is the political will from the Wikimedia Foundation management to impose a deadline to the devs. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On 12 December 2011 15:26, K. Peachey p858sn...@gmail.com wrote: It's been a requested feature for a while, Someone finally got around to writing it (I believe it needed the Improved metadata handling backend first) and implementing it, It wasn't a sudden oh lets write this and enable it in one day thing, a lot of work went into it and subsequent testing. * How many existing uploads, used on the wikis, were previously wrongly rotated and were fixed by the feature? * How many existing uploads, used on the wikis, were previously correctly rotated and were messed up by the feature? i.e., was there strong reason to apply it to past images, not just new ones? - d. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
I am unable to find precise answers to your questions. But the scope of the phenomenon can be somehow understood with the following data which hint that today, the demand for rotation service has increased about 56-fold compared to June 2011. But I am unable to say how long the present high demand will last. And we must think about the unused pictures or pictures used on small projects which may require rotation but which people may be not be going to find so soon. Let alone the cases when readers find that something is wrong but are too shy to say it. As of 24-30 June (7 days) Rotatebot was requested to rotate about 250 files in 7 days (1) As of now, Rotatebot is handling about 250 files in 3 hours (2) (which means (24/3)*7*250 = 56*250 in 7 days) (1) http://commons.wikimedia.org/w/index.php?title=Special:Contributionsoffset=2011070100limit=500contribs=usertarget=Rotatebot (2) http://commons.wikimedia.org/w/index.php?title=Special:Contributionslimit=500contribs=usertarget=Rotatebot Le 12 décembre 2011 16:55, David Gerard dger...@gmail.com a écrit : * How many existing uploads, used on the wikis, were previously wrongly rotated and were fixed by the feature? * How many existing uploads, used on the wikis, were previously correctly rotated and were messed up by the feature? i.e., was there strong reason to apply it to past images, not just new ones? ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On 12 December 2011 15:26, K. Peachey p858sn...@gmail.com wrote: Nothing much went wrong in the planning of this feature, Really?! How is not having realised that this new feature would break 1000's of images and preventing it not something going wrong in the planning? (And yes, I mean break - they displayed correctly before and they don't now, the fact that the EXIF data was corrupt isn't anywhere near as important as how they actually display on the sites that use them.) It was an innocent mistake and these things happen, but you need to accept that it was a mistake and consider what you can do in future to avoid such mistakes happening again. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On Mon, Dec 12, 2011 at 7:55 AM, David Gerard dger...@gmail.com wrote: * How many existing uploads, used on the wikis, were previously wrongly rotated and were fixed by the feature? * How many existing uploads, used on the wikis, were previously correctly rotated and were messed up by the feature? As far as I understand the issue, and others can jump and correct me if I'm getting it wrong: Technically, nothing was messed up by the feature. Rather, the software previously did not take EXIF rotation into account, and some images had incorrect EXIF rotation information to begin with. Those images are now shown in an incorrect rotation to the user, because the incorrect EXIF rotation info is being evaluated. It's important to understand this, because it means that those images have been causing problems for re-users all along. If you open those images with modern image editing/viewing software, they will either be automatically rotated, or you'll be prompted by the software whether to apply the rotation noted in the EXIF tag. The situation has been significantly exacerbated by a recent need to purge old thumbnails to free up diskspace. So, while the cleanup that's happening now is very frustrating (and I definitely agree we could have anticipated and communicated this better), it's a cleanup that's long overdue. (Either by stripping EXIF info from files altogether, or by ensuring that the rotation of the image matches the one in the metadata.) Is there more that we can do at the present time to help? Thanks, Erik -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On 12 December 2011 18:18, Erik Moeller e...@wikimedia.org wrote: On Mon, Dec 12, 2011 at 7:55 AM, David Gerard dger...@gmail.com wrote: * How many existing uploads, used on the wikis, were previously wrongly rotated and were fixed by the feature? * How many existing uploads, used on the wikis, were previously correctly rotated and were messed up by the feature? As far as I understand the issue, and others can jump and correct me if I'm getting it wrong: Technically, nothing was messed up by the feature. Rather, the software previously did not take EXIF rotation into account, and some images had incorrect EXIF rotation information to begin with. Those images are now shown in an incorrect rotation to the user, because the incorrect EXIF rotation info is being evaluated. That's a big technicality. Surely the most important thing is how the images display to users? There were right before and now they aren't. That may not be technically messed up, but it is messed up in reality. It's important to understand this, because it means that those images have been causing problems for re-users all along. If you open those images with modern image editing/viewing software, they will either be automatically rotated, or you'll be prompted by the software whether to apply the rotation noted in the EXIF tag. Indeed, it's good to get these images fixed, but surely it would have been better to fix them rather than just break the workaround that was stopping people noticing they were broken? The situation has been significantly exacerbated by a recent need to purge old thumbnails to free up diskspace. How big a contributing factor has that been? As I understand it, only thumbnails of unused images were purged. People (including me) have been stumbling over incorrect images in articles - have they just been unlucky and the thumbnail happened to expire at the wrong time? So, while the cleanup that's happening now is very frustrating (and I definitely agree we could have anticipated and communicated this better), it's a cleanup that's long overdue. (Either by stripping EXIF info from files altogether, or by ensuring that the rotation of the image matches the one in the metadata.) Is there more that we can do at the present time to help? I think, at the moment, the most useful thing would be to automate finding the broken images (basically, it's all images uploaded before the feature was introduced that have a non-zero EXIF rotation). ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On 12 December 2011 18:18, Erik Moeller e...@wikimedia.org wrote: Technically, nothing was messed up by the feature. Rather, the software previously did not take EXIF rotation into account, and some images had incorrect EXIF rotation information to begin with. Those images are now shown in an incorrect rotation to the user, because the incorrect EXIF rotation info is being evaluated. That's ridiculous misuse of words. What was messed up was the presentation of images that were already displayed correctly. It is entirely unclear to me why you appear to be evading rather than answering a fairly simple and straightforward question: How many images used in the wikis had the pages they were on messed up by this? - d. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
From: David Gerard dger...@gmail.com On 12 December 2011 18:18, Erik Moeller e...@wikimedia.org wrote: Technically, nothing was messed up by the feature. Rather, the software previously did not take EXIF rotation into account, and some images had incorrect EXIF rotation information to begin with. Those images are now shown in an incorrect rotation to the user, because the incorrect EXIF rotation info is being evaluated. That's ridiculous misuse of words. What was messed up was the presentation of images that were already displayed correctly. It is entirely unclear to me why you appear to be evading rather than answering a fairly simple and straightforward question: How many images used in the wikis had the pages they were on messed up by this? Actually, I think Erik's use of words here is spot on. The previous images were messed up in such a way that they appeared right by fluke, but their metadata wasn't correct. Now, they can be easily identified and properly fixed by the community. This is a good and useful improvement - well done WMF + tech team for implementing it. :-) From: Andrew Gray andrew.g...@dunelm.org.uk I agree applying it to old images was a bit of an odd thing to do (if they were visibly wrong, someone usually went to the effort of re-uploading them), but that doesn't mean applying it to later ones was somehow a stupid thing to do. With this type of modification, it's natural that it would apply to all images rather than just images uploaded after it was switched on. It would be horribly unnatural and deliberately-buggy if it tried to take the date of upload into account when applying the modification... Thanks, Mike P.S. am replying to the digest - apologies if this ends up in the wrong thread... ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On Mon, Dec 12, 2011 at 12:12 PM, David Gerard dger...@gmail.com wrote: What was messed up was the presentation of images that were already displayed correctly. Well, technically, they were displayed incorrectly. ;-) The image told the software Please rotate me, and the software didn't. But the image would tell any other software the same thing, causing pain for re-users. So it was definitely an issue that needed to get resolved, one way or another. I don't know off-hand how many images are affected (the estimate on Commons is about 50,000, but I don't know what that's based on). The thing is, we've always gotten drive-by uploads by users who didn't bother to fix any rotation issues with their images after upload, and so we can't just go back and strip EXIF info from all old files, because some old files were fixed by the change. It looks to me like the only sensible response is human review followed by rotation of images that need to be fixed -- which is precisely what's happening, with a bot performing rotations as needed. I've asked Rob Lanphier to look at this as well and determine if an additional response is needed; if you think there's more we can/should do to help, please let him know. The best place for further discussion of this issue is: http://commons.wikimedia.org/wiki/Commons_talk:Rotation -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
On Mon, Dec 12, 2011 at 7:48 PM, Erik Moeller e...@wikimedia.org wrote: The best place for further discussion of this issue is: http://commons.wikimedia.org/wiki/Commons_talk:Rotation And, lots more discussions here as well: http://commons.wikimedia.org/wiki/Commons:Bots/Work_requests#Maintenance_category_for_files_with_EXIF_rotation_other_than_0_degrees If I interpret that discussion correctly, the number of globally used files that were affected is estimated to be about 20,000, with an additional 35,000 files that weren't globally used, based on analysis of the image metadata dumps. -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] The Mediawiki 1.18 image rotation bug on Commons and on all Wikimedia projects
A thought to those posting in this thread (especially some of the earlier posts): What effect would a less aggressive tone have had? Would you have been more likely to convince your audience? less likely to alienate people? This list often has too high a heat:light ratio. You can help fix this. -- David Richfield ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
