RE: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 [IAccessible2 support in JAWS]
Hi Michael,
yes, it is correct that IAccessible2 and ATK were both devised with the Open
document format model in mind, with the implementation in Firefox following
closely behind, introducing some web specific modifications into the
specifications for both platforms.
So, yes, as Michael W. suggested, for the near term, and probably some years to
come, it is a good strategy to fix bugs in the IAccessible2 and ATK layers to
get them up to speed. And as I said, even Impress can be made to work, at least
with NVDA, by making sure to collaborate with the NVDA project so they support
it when it is ready.
It is, however, a good strategy to also keep an eye on UIA, at least on the
side, and also watch what Mozilla do with Firefox in the longer term with
regards to an UIA implementation. Just to not get hit out of the blue with
suddenly IAccessible2 support twindling within assistive technologies because
everyone else has moved to UIA.
Marco
-Original Message-
From: Michael Stahl
Sent: Thursday, June 9, 2022 11:51 AM
To: Michael Weghorn ; Marco Zehe ;
accessibility@global.libreoffice.org
Cc: libreoffice...@lists.freedesktop.org; libreoff...@lists.freedesktop.org
Subject: Re: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26
[IAccessible2 support in JAWS]
On 09.06.22 11:21, Michael Weghorn wrote:
> I suppose that supporting new UIA concepts/features in addition would
> probably require more fundamental changes than "just" adding a new
> wrapper/bridge for UIA around existing LO a11y interfaces (either a
> custom one, or using Gtk/Qt).
> But I'm not an expert and have only little experience with a11y so far
> and not looked into UIA any closer, so all of the above would need
> deeper knowledge/further investigation for a more reliable statement.
>
> Unless there are resources to work on UIA specifically, I tend to
> think it would make sense to focus on improving the existing
> IAccessible2-based implementation ("winaccessibility") (and fixing
> issues that are not platform-specific) for now if that's (still)
> sufficiently supported by AT in practice, and reconsider what to do
> about UIA at some later point in time (at which there might also be
> news on the state of gtk4 and qt5/qt6 a11y).
i tend to agree, one aspect of this is that AFAIK the AT-SPI and
IAccessible2 APIs that concern document content were designed specifically with
OOo's document models in mind - there might be a bigger impedance mismatch to
other A11y APIs, needing more code in the "translation layer".
but i don't know any details about UIA.
--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy
Re: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 [IAccessible2 support in JAWS]
Hi Marco, all,
On 09/06/2022 09.39, Marco Zehe wrote:
AFAIK, QT exposes accessibility information to UIA on Windows. They switched
over from an MSAA implementation to UIA some time in the QT5 time frame:
https://www.qt.io/blog/2018/02/20/qt-5-11-brings-new-accessibility-backend-windows.
Indeed, that was what I had come across as well. (I had linked the
corresponding Qt commit in the wiki article about adding UIA support.)
So, using QT widgets in the VCL certainly would help with that, UIA
implementation bugs in the Windows QT layer not withstanding of course. So,
there are advantages of using QT on Windows where appropriate, but that also
entails the danger of inheriting QT UIA bugs.
Very true. While looking a bit into a11y of the Qt-based LO VCL plugins
on Linux, I also came across some issues that need fixing for the AT-SPI
integration in the Qt library rather than in LO.
I don't know anything about GTK4, so cannot make any qualified statement.
At a quick glance, everything underneath Gtk 4's "gtk/a11y" directory in
git ( https://gitlab.gnome.org/GNOME/gtk/-/tree/main/gtk/a11y ) still
looks AT-SPI only, though that might change at some point.
The original advantages of IAccessible2 and GTK3 was that they were made to be
very closely related to one another in terms of concepts. But with GTK4 and
other frameworks becoming more prominent, this has only been in a maintained
state for years, not really further developed, except for adding necessary
missing pieces for new HTML widgets or markup. UIA, on the other hand, has
turned out to be much more flexible especially with the latest enhancements
published by Microsoft. Custom property sets etc., which allows for various
annotations in MS Word, Excel etc. NVDA has a pull request for implementation
of some of these newest UIA technologies into their support for MS Office here:
https://github.com/nvaccess/nvda/pull/13387
So, in the long run, it is probably safest to indeed invest in switching to an UIA implementation.
Note, however, that you may still need to do some work yourselves for the document specific stuff for Writer, Calc, and Impress specifically, since probably not everything is available in the QT libraries that you need.
As I understand it from what I have seen so far, the concept for current
LO a11y is also largely based on the same concepts as
IAccessible2/AT-SPI, and Qt a11y API is also largely based on the same.
IIUC, what current platform-specific a11y integrations in LO (gtk3,
qt5/qt6, winaccessibility) do is mostly provide a wrapper around/bridge
between LO a11y API and the platform-specific API (ATK for gtk3 which is
bridged to AT-SPI by ATK, Qt for qt5/qt6 which is then bridged to AT-SPI
by the Qt library for the Linux case, MSAA/IAccessible2 for
winaccessibility).
Most of the LO UI and document-specific a11y is implemented in a
platform-independent way (VCL toolkit for the UI) and the mapping to the
platform-specific implementations happens in a thin layer.
In my understanding, that would remain unchanged in case of using either
Gtk or Qt on Windows as well. As far as a11y is concerned, this would
essentially mean dropping the MSAA/IAccessible2 bridge contained in the
"winaccessibility" directory and reusing the code that is also used for
Linux (and leave the mapping to UIA to Gtk/Qt).
Should that turn out to be a reasonable approach (which I don't know!),
my expectation would be that this would essentially give us the same set
of features with UIA that we currently have with IAccessible2.
I suppose that supporting new UIA concepts/features in addition would
probably require more fundamental changes than "just" adding a new
wrapper/bridge for UIA around existing LO a11y interfaces (either a
custom one, or using Gtk/Qt).
But I'm not an expert and have only little experience with a11y so far
and not looked into UIA any closer, so all of the above would need
deeper knowledge/further investigation for a more reliable statement.
Unless there are resources to work on UIA specifically, I tend to think
it would make sense to focus on improving the existing
IAccessible2-based implementation ("winaccessibility") (and fixing
issues that are not platform-specific) for now if that's (still)
sufficiently supported by AT in practice, and reconsider what to do
about UIA at some later point in time (at which there might also be news
on the state of gtk4 and qt5/qt6 a11y).
Happy to hear what others think about this.
Michael
--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy
RE: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 [IAccessible2 support in JAWS]
Hi Michael, AFAIK, QT exposes accessibility information to UIA on Windows. They switched over from an MSAA implementation to UIA some time in the QT5 time frame: https://www.qt.io/blog/2018/02/20/qt-5-11-brings-new-accessibility-backend-windows. So, using QT widgets in the VCL certainly would help with that, UIA implementation bugs in the Windows QT layer not withstanding of course. So, there are advantages of using QT on Windows where appropriate, but that also entails the danger of inheriting QT UIA bugs. I don't know anything about GTK4, so cannot make any qualified statement. The original advantages of IAccessible2 and GTK3 was that they were made to be very closely related to one another in terms of concepts. But with GTK4 and other frameworks becoming more prominent, this has only been in a maintained state for years, not really further developed, except for adding necessary missing pieces for new HTML widgets or markup. UIA, on the other hand, has turned out to be much more flexible especially with the latest enhancements published by Microsoft. Custom property sets etc., which allows for various annotations in MS Word, Excel etc. NVDA has a pull request for implementation of some of these newest UIA technologies into their support for MS Office here: https://github.com/nvaccess/nvda/pull/13387 So, in the long run, it is probably safest to indeed invest in switching to an UIA implementation. Note, however, that you may still need to do some work yourselves for the document specific stuff for Writer, Calc, and Impress specifically, since probably not everything is available in the QT libraries that you need. Marco -Original Message- From: Michael Weghorn Sent: Thursday, June 9, 2022 9:01 AM To: Marco Zehe ; accessibility@global.libreoffice.org Cc: libreoffice...@lists.freedesktop.org; libreoff...@lists.freedesktop.org Subject: Re: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 [IAccessible2 support in JAWS] Hi Marco, On 08/06/2022 13.09, Marco Zehe wrote: > JAWS does support IAccessible2, but only if it needs to, like in Firefox, and > some parts of Chromium-based browsers. However, with the UI Automation > implementation for the latter becoming stronger, there might be a time when > JAWS moves to UIA for web content support in Chromium browsers. But you never > know what plans Vispero actually has for which version of JAWS, ZoomText, and > Fusion. All I notice is that, with Windows 11 having an even stronger UIA > implementation than 10, more things are being done through that interface > rather than traditional MSAA or IAccessible2 channels where possible. And as > has been said elsewhere, UIA properties and events are even very accessible > from within the JAWS scripting language, which makes this even more > compelling because the end user experience can be customized further. As > IAccessible2 is an extension of MSAA, and MSAA is largely deprecated by > Microsoft, it will probably never get the same treatment. thanks a lot for that valuable input. I have added a new sub-section for adding UIA support to the wiki page ( https://wiki.documentfoundation.org/Development/Under-loved_areas#UIA_support_on_Windows ), mostly based on the above and more information from your other email about announcing slide content in Impress presentation mode ( https://listarchives.libreoffice.org/global/accessibility/msg01007.html ). A probably rather crazy thought I once had was whether it would be a good idea to try to get rid of our custom a11y bridge on Windows in the long run after all, e.g. by switching to Gtk or Qt there as well (at least as one option). But that would certainly have to make sense not only from the a11y perspective but the UI as a whole and would probably be rather contentious. (I have close to zero knowledge about Windows-specific bits in LO besides winaccessibility, so don't know whether there would be any value in even spending any time in looking into this any further at some point in time. And as of now, neither the gtk4 nor the qt5/qt6 VCL plugins in LO have proper a11y anyway, and the Gtk 4 library presumably doesn't have any a11y implementation for Windows yet either, even though the new a11y architecture [1] in Gtk 4 should allow for one to be added at some stage.) Michael [1] https://blog.gtk.org/2020/10/21/accessibility-in-gtk-4/ -- To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette List archive: https://listarchives.libreoffice.org/global/accessibility/ Privacy Policy: https://www.documentfoundation.org/privacy
Re: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26
Hi Marco, thanks for the additional input. I have added that to the Bugzilla ticket: https://bugs.documentfoundation.org/show_bug.cgi?id=149488#c1 Michael On 08/06/2022 13.03, Marco Zehe wrote: Correct, in PowerPoint, when a presentation is started, both NVDA and JAWS, and presumably at some point, Narrator, too, will grab the slide contents and present it slide by slide like a web page. JAWS even annotates links, tables, lists, etc., in a very rich way. NVDA, to my knowledge, doesn't do that yet. In MS Office, screen readers have largely switched to using UI Automation (UIA) for access to all things documents and UI. This is primarily because of Narrator, which doesn't support anything other than UIA (MSAA and IA2 are only supported by way of an IA2 to UIA bridge, which is slow and unreliable). As a consequence, Microsoft never got on the IAccessible2 bandwaggon, but has pushed the UIA implementation in the Chromium project so they can stop using the IAccessible2ToUIA Bridge for Narrator's access to web content. There were even plans and experiments to switch Firefox over to UIA when I was still working at Mozilla. But since I am no longer involved there, I don't know if this is still on the table for the time after they finish the "Cache The World" project. So, in the long term, and as resources permit, the more future-proof way forward for LibreOffice on Windows might be to switch over to an UIA implementation as well. But even without that, there would need to be a concerted effort between the Impress and screen reader teams, like NVDA, to make NVDA realize that it is in a slide show in presentation mode, and gain all the access to the slide contents like it were a web page or similar. That cannot be achieved by one party alone I think. And getting Vispero on board for JAWS support is an even bigger fish to fry. Marco -Original Message- From: Jason White Sent: Tuesday, June 7, 2022 9:34 PM To: accessibility@global.libreoffice.org Subject: Re: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 On 7/6/22 06:07, Michael Weghorn wrote: I tried again with just a single screen instead of two, and then NVDA announces "Slide 1", then reads out the slide content, and when moving further: "Slide 2" and its content, etc. Is that what you think Impress should do as well? (It didn't in a quick test with gtk3 on Linux.) Yes. If I recall correctly, under MS-Windows/PowerPoint, NVDA and JAWS both support arrow key navigation in the slide contents when the slides are being presented (i.e., after F5 is used to start a presentation). Ideally, one should be able to do the same in LibreOffice/Impress, and under Linux also. Space/Backspace navigate among slides in Windows/PowerPoint too. Obviously, Impress needs a similar keyboard mechanism for slide navigation. None of this should depend on the number of attached displays. I don't think anyone wants their accessibility to fail just in virtue of the number of displays that happen to be connected. -- To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette List archive: https://listarchives.libreoffice.org/global/accessibility/ Privacy Policy: https://www.documentfoundation.org/privacy -- To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette List archive: https://listarchives.libreoffice.org/global/accessibility/ Privacy Policy: https://www.documentfoundation.org/privacy
Re: [libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 [IAccessible2 support in JAWS]
Hi Marco, On 08/06/2022 13.09, Marco Zehe wrote: JAWS does support IAccessible2, but only if it needs to, like in Firefox, and some parts of Chromium-based browsers. However, with the UI Automation implementation for the latter becoming stronger, there might be a time when JAWS moves to UIA for web content support in Chromium browsers. But you never know what plans Vispero actually has for which version of JAWS, ZoomText, and Fusion. All I notice is that, with Windows 11 having an even stronger UIA implementation than 10, more things are being done through that interface rather than traditional MSAA or IAccessible2 channels where possible. And as has been said elsewhere, UIA properties and events are even very accessible from within the JAWS scripting language, which makes this even more compelling because the end user experience can be customized further. As IAccessible2 is an extension of MSAA, and MSAA is largely deprecated by Microsoft, it will probably never get the same treatment. thanks a lot for that valuable input. I have added a new sub-section for adding UIA support to the wiki page ( https://wiki.documentfoundation.org/Development/Under-loved_areas#UIA_support_on_Windows ), mostly based on the above and more information from your other email about announcing slide content in Impress presentation mode ( https://listarchives.libreoffice.org/global/accessibility/msg01007.html ). A probably rather crazy thought I once had was whether it would be a good idea to try to get rid of our custom a11y bridge on Windows in the long run after all, e.g. by switching to Gtk or Qt there as well (at least as one option). But that would certainly have to make sense not only from the a11y perspective but the UI as a whole and would probably be rather contentious. (I have close to zero knowledge about Windows-specific bits in LO besides winaccessibility, so don't know whether there would be any value in even spending any time in looking into this any further at some point in time. And as of now, neither the gtk4 nor the qt5/qt6 VCL plugins in LO have proper a11y anyway, and the Gtk 4 library presumably doesn't have any a11y implementation for Windows yet either, even though the new a11y architecture [1] in Gtk 4 should allow for one to be added at some stage.) Michael [1] https://blog.gtk.org/2020/10/21/accessibility-in-gtk-4/ -- To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette List archive: https://listarchives.libreoffice.org/global/accessibility/ Privacy Policy: https://www.documentfoundation.org/privacy
