I've had some time to work on this, I've written up a draft... draft DEP today, and I'd like to request feedback. I've written it quite hastily over the last few hours with the expectation that it will need to be substantially rewritten. I've added it as a local PR on my fork of DEPS to keep the discussion in one place. I'm happy to take broad feedback here, but try to keep specific feedback on the PR itself to keep the noise off this thread. PR: https://github.com/knyghty/deps/pull/1
I'm also requesting someone to shepherd this, and ideally also a co-author, particularly someone on the technical board, a core dev, or someone else with experience of Django's governance, as I think this is the first process DEP written by a non-core dev. Cheers! Tom On Thu, 25 Jun 2020 at 19:31, Tom Carrick <t...@carrick.eu> wrote: > I've managed to have a quick look, it looks really promising. Having a > total number of issues to compare against would be good to avoid > regressions or introducing new issues. I do have one concern. To be viable, > it needs to live in the django repo somewhere. And we'd need a full admin > site to test against, with every admin feature (stacked / tabular inlines, > filtering, search, date filters, list editable, autocomplete, ...) to be > fully representative and catch everything. This seems like a really big > amount of stuff to add into the repo, that will need to be maintained, and > possibly an easy thing to forget about. Also the maintenance is tricky, we > should ideally not update this site too often or the total number of issues > won't be as easily comparable. I wonder if there's another way to handle > it, but I can't think of one. > > I also imagine this would be pretty intensive to run, it would probably be > best to not run it by default and have a buildbot command to run this > manually for PRs where there are changes to the admin. > > On Thu, 25 Jun 2020 at 02:15, Thibaud Colas <thibaudco...@gmail.com> > wrote: > >> Hi Tom, >> >> Great to hear – no rush if you’re busy with other things. Here’s a quick >> proof of concept, with 20 different pages/scenarios, tested with Axe, HTML >> Code Sniffer, and Lighthouse: >> https://github.com/thibaudcolas/django_admin_tests. >> >> I’m not a big fan of Lighthouse personally, and I already had the rest of >> the tools set up, hence why I went with this. I spent a bit of time making >> a report out of the test results, which you can see an example of at >> https://thibaudcolas.github.io/django_admin_tests/. This is generated in >> Travis, currently set up to run those tests & regenerate the report from >> Django’s `master` branch once per week. >> >> I added instructions on the README to run this locally if anyone wants to >> try it out, and a few words about what to make of the report / issues. I >> didn’t look into what the issues were though. >> >> Let me know what you think, and I should be able to spend more time on >> this next week if it helps. >> >> On Wednesday, 24 June 2020 at 09:11:52 UTC+1 t...@carrick.eu wrote: >> >>> A quick update first. I'm pretty busy with a combination of day job and >>> personal projects, but I should have time to start writing the draft DEP in >>> the next week or two. >>> >>> Thibaud, I think the absolute most important thing is a way to measure >>> progress, even if - as others have mentioned - that measure is imperfect. I >>> think getting a proof of concept of Lighthouse running against a few admin >>> pages would be extremely helpful. It's also probably one of the more >>> difficult things. If that seems like too much, I think catching what the CI >>> would miss is the next most important thing, so I think your #1 suggestion >>> would work well. >>> >>> On Tue, 23 Jun 2020 at 22:34, Thibaud Colas <thibau...@gmail.com> wrote: >>> >>>> Hey Tom, >>>> >>>> I wanted to check if there is anything we/I could do to help in the >>>> meantime? Whether that’s by starting to map or audit the Django admin, or >>>> setting up a sample CI pipeline with accessibility tests, or something else >>>> altogether. It’s a bit daunting to get started with this but I think I >>>> could realistically do one of: >>>> >>>> 1. Create a draft map of the Django admin UI for later manual auditing >>>> purposes. >>>> 2. Audi a specific part of Django and creating a ticket with concrete >>>> things to fix. >>>> 3. Set up static analysis for the CSS or HTML to look for common >>>> accessibility gotchas. >>>> 4. Sett up automated in-browser accessibility checks on some parts of >>>> Django to show what that might look like in a CI pipeline. >>>> >>>> … and document the steps along the way, and report back here or as a >>>> ticket. >>>> >>>> Cheers, >>>> >>>> Thibaud >>>> >>>> On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote: >>>>> >>>>> Some further thoughts... >>>>> >>>>> Mariusz, I don't think I agree that WCAG is massive. It can look a >>>>> little large but I think that's mostly because each guideline is split >>>>> into >>>>> smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 >>>>> 50 >>>>> success criteria for AA, many of which can be checked automatically, and >>>>> many of those that can't don't apply to us as we don't use audio / video. >>>>> There's also a nice quick reference >>>>> <https://www.w3.org/WAI/WCAG21/quickref/> that can be filtered to >>>>> remove AAA and anything else unwanted. Once you drill down it can again >>>>> get >>>>> quite wordy, but it covers a lot. I think it's worth remembering here is >>>>> that the point is not to strictly adhere to a standard for the sake of it, >>>>> just to improve the overall accessibility, and WCAG is a useful and >>>>> measurable tool to get us some of the way there. >>>>> >>>>> Thibaud, I more or less agree with everything there. I'm not sure >>>>> developers should have to do full accessibility checks for their changes >>>>> in >>>>> the admin. Using a screen reader for example can be very frustrating and >>>>> confusing if you're not used to it, and it can take an inordinate amount >>>>> of >>>>> time, and I don't really wish that burden on (for example) first time >>>>> committers who already have a lot of stuff to do, I think this will only >>>>> discourage contributions. I think it should be the responsibility of the >>>>> accessibility team to provide feedback, suggestions, etc. on relevant PRs >>>>> or fix things when they're noticed after manual audits. >>>>> >>>>> ATAG looks handy, but at first glance we couldn't even get to A, it >>>>> requires auto-saving. It also generally seems like something that would >>>>> cover a rich text editor plugged into the admin more than the admin >>>>> itself, >>>>> though I think it's laudable as a future target. >>>>> >>>>> The more I think about it, I think the DEP should be a process DEP >>>>> rather than a feature DEP. As others have mentioned, we don't need a DEP >>>>> to >>>>> fix accessibility issues - just fix them. But forming a team seems useful, >>>>> and I think a DEP is required for that. I think the DEP should be limited >>>>> to: >>>>> >>>>> 1. The accessibility team, how membership is decided, who it's >>>>> accountable to, it's role/responsibilities, etc. >>>>> 2. Perhaps the initial standard(s) to target, although this could also >>>>> be decided by the a11y team post-assembly and agreed with the technical >>>>> board and/or core devs. >>>>> >>>>> As I said, I'm happy to write up the DEP, but since it'll be a process >>>>> DEP, I think I'd need the support of a core dev, someone on the technical >>>>> board, or just someone who has more knowledge of Django's "bureaucracy", >>>>> for lack of a better word. >>>>> >>>>> Tom >>>>> >>>>> On Tue, 26 May 2020 at 01:20, Thibaud Colas <thibau...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi Tom, >>>>>> >>>>>> It’s exciting to see this getting started! To me a DEP would be >>>>>> highly beneficial, because there is a lot to this that goes beyond >>>>>> finding >>>>>> and fixing individual issues – it’s more about detailing the a process >>>>>> for >>>>>> parts of Django to stay accessible over time. Here are things I’d suggest >>>>>> to cover, in addition to your list: >>>>>> >>>>>> - Going further than the targeted standard and tests in CI – outline >>>>>> clear steps developers should take when testing their work: using >>>>>> developer >>>>>> tools, screen readers, etc. It’s not practical to just state the standard >>>>>> as WCAG is quite big, and open to interpretation. And CI testing will >>>>>> never >>>>>> be enough to reach any degree of compliance. >>>>>> - I’m not familiar with Django’s development process but generally >>>>>> I would recommend to use a combination of linting, browser extensions, >>>>>> manual testing instructions – and CI tests. >>>>>> - If the accessibility team was to cover more than the Django admin, >>>>>> I think it would be worthwhile to also include accessibility of sites >>>>>> built >>>>>> with Django. Django is largely unopinionated about markup, but I remember >>>>>> its forms markup not being particularly good for screen readers, and the >>>>>> HTML code snippets in the docs would also be worth reviewing. >>>>>> - Same goes for admin docs – not too familiar with them myself so I’m >>>>>> not sure whether they are considered to be part of the admin, or >>>>>> separate. >>>>>> - I wouldn’t find it surprising if an audit of the Django admin >>>>>> turned up a lot of issues. IMHO part of the DEP should cover how to >>>>>> manage >>>>>> this – whether individual issues warrant individual tickets or not, and >>>>>> how >>>>>> to prioritize the issues if relevant. Which kinds of issues are likely to >>>>>> need design input. If the DEP is done first, also talk about what kind of >>>>>> auditing would be valuable, and how to make it happen. >>>>>> >>>>>> To help with the initial audit and prioritisation of issues, I would >>>>>> suggest to first start by creating a map of all of the parts of the >>>>>> Django >>>>>> admin that are to be audited – list all of the individual pages, and all >>>>>> of >>>>>> the states the pages can be in (success, error, loading – empty content / >>>>>> some content / paginating, etc). I also find it worthwhile to add a >>>>>> succinct definition of how each of those pages is likely to be used – for >>>>>> example things that are only configured once for a site’s lifetime are >>>>>> presumably not as worth improving as a screen a user would see on a daily >>>>>> basis. Here is an example in practice: >>>>>> https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit >>>>>> <https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit#gid=0>. >>>>>> This can also be used to track the auditing effort, and form the basis of >>>>>> an automated test suite for CI (or simply repeat testing). >>>>>> >>>>>> Finally, consider ATAG2.0 <https://www.w3.org/TR/ATAG20/> (Authoring >>>>>> Tool Accessibility Guidelines) compliance additionally to WCAG. At a high >>>>>> level, ATAG is made of two parts: >>>>>> >>>>>> - Part A: Make the authoring tool user interface accessible. That >>>>>> sounds like what we’re discussing here. >>>>>> - Part B: Support the production of accessible content. That’s a >>>>>> whole other topic but feels relevant too. >>>>>> >>>>>> ATAG is nowhere near as well known / established / easy to test for, >>>>>> but it feels very relevant to the Django admin in particular, and I’m >>>>>> sure >>>>>> there will at least be some useful bits to consider in its success >>>>>> criteria >>>>>> >>>>>> Hope this helps! >>>>>> >>>>>> Thibaud >>>>>> >>>>>> >>>>>> On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Thanks for the feedback. I had thought about a DEP when I was >>>>>>> writing up the original post actually, I just wasn't sure what it should >>>>>>> contain. Here are my thoughts, based on the feedback so far: >>>>>>> >>>>>>> - Defining a standard to target. >>>>>>> - Forming an a11y team that covers the django admin and all sites in >>>>>>> the django github organization, and defining its role, membership, etc. >>>>>>> - Deciding on a CI process >>>>>>> - An outline of current issues and how to solve them. >>>>>>> >>>>>>> If anyone can think of anything else, please let me know. If/when >>>>>>> there's a consensus I'll start writing a draft. >>>>>>> >>>>>>> Mariusz, I mentioned this in the original post, but >>>>>>> https://www.w3.org/WAI/policies/ has a good overview of laws and EU >>>>>>> directives in this area. From my browsing through, it seems that they >>>>>>> all >>>>>>> either, have their own national standard, or are using WCAG 2.0/2.1 AA. >>>>>>> Since we probably don't want to define our own standard, I think AA is >>>>>>> the >>>>>>> way to go. This seems to also be the recommendation I hear from people >>>>>>> in >>>>>>> the accessibility field working on regular websites. AAA seems to be >>>>>>> more >>>>>>> suited for very specialist sites that explicitly target disabled people. >>>>>>> >>>>>>> Most of AAA is completely feasible for us, but there are some >>>>>>> reasons it would be difficult: >>>>>>> >>>>>>> - All language needs to be at a lower secondary education level, or >>>>>>> have an alternative that is. This doesn't seem feasible, for e.g. the >>>>>>> documentation. >>>>>>> - Users that are logged out need to be able to resume their session >>>>>>> where they left off after logging in, with all form fields filled, etc. >>>>>>> >>>>>>> There are others that are difficult, but I don't think any apply to >>>>>>> us currently. >>>>>>> >>>>>>> Cheers, >>>>>>> Tom >>>>>>> >>>>>>> >>>>>>> On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <felisia...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Tobias, >>>>>>>> >>>>>>>> I'm not a WCAG expert, and it's not clear to me what steps we >>>>>>>> would like to take. In the ticket we have only steps describes as >>>>>>>> *"very >>>>>>>> basics which should include"*, so I can imagine that's not all we >>>>>>>> need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as >>>>>>>> I'm >>>>>>>> concerned WCAG is quite massive. Moreover if we want to make changes >>>>>>>> in CI >>>>>>>> we should discussed available tools, etc. We can change colors but what >>>>>>>> next? We will be not able to guarantee that the contrast of all >>>>>>>> elements >>>>>>>> remains appropriate, we cannot do this manually. >>>>>>>> >>>>>>>> Best, >>>>>>>> Mariusz >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to django-d...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to django-d...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Django developers (Contributions to Django itself)" group. >>>> >>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to django-develop...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com >>>> <https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-developers+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/125fc1db-6d9f-404c-86aa-2f2eeb75a192n%40googlegroups.com >> <https://groups.google.com/d/msgid/django-developers/125fc1db-6d9f-404c-86aa-2f2eeb75a192n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMbUD6w4Dj8tGDbBR2o4Xv2-L6m7GT15%3DZ3vsz54Yw0AMg%40mail.gmail.com.
