Hi Michael,
On 2024-06-11 11:40, Michael Meeks wrote:
Some great questions and points here; forgive my intruding into the
space again =)
No need to apologize - thanks a lot for your valuable input! :-)
* How do other interfaces (like AT-SPI Table, TableCell and Selection)
expose information? Does e.g. the table report it only has 50 rows and
30 columns if that's what's visible on screen? Does cell Q227 report a
row and column index of 0 if it's the first one in the visible area?
I think exposing the whole thing through the Table interface may
make some sense; it is clearly a crazy set of cells - and think it's
reasonable to blame ATs if they use this interface for doing something
silly.
In the meantime, Joanmarie has added some further very useful
information in tdf#156657 comment [1], also giving a reference to a grid
example from the ARIA WG [2].
There, the Table interface also only exposes the same amount of cells as
are exposed via the a11y tree.
(An even more advanced example also managing selection that that example
references [3] currently doesn't exist yet, unfortunately. ("This
example has not yet been developed."))
Right; so - I mentioned "near to the screen" - by near; I mean we
will probably want a number of things that are navigationally close: eg.
"next heading" or somesuch - to lurk around as real & tracked peers. The
content of the Navigator headings should prolly always be present in a
writer document's object hierarcy IMHO. That should let ATs very quickly
enumerate headings, jump focus to them with a simple API etc.
That sounds interesting, but in a way also like a rather strange tree to
me if it contains elements of some type for the whole doc, but other
parts of the document in between are missing.
From looking into the NVDA and Orca doc, other objects of interest are
e.g. tables, lists, list items, or links. I suppose that including all
of these (and more) in the tree could potentially again result in a
large tree and the re-pagination update problems you've mentioned, for
particular documents.
* How do screen readers implement features like "read the whole row"?
This comes down to the navigation API I mentioned: having a good
API to allow continuous screen-reading of large data-sets - with caching
pre-loading & fetching along eg. a selection is really useful. Current
writer behavior is far from optimal since you need to do something odd
to get a simple navigation such as "next page" IIRC.
Do they just read the part of the row that's currently visible on
screen and leave out the rest? Or do they somehow implement some extra
logic
to retrieve the remaining content?
Extra navigation / enumeration logic I think.
I see, the navigation API apparently is definitely an essential key
aspect then.
If those need changes to the platform a11y APIs, we theoretically have
more chances for these for some (the FLOSS ones: Linux/AT-SPI,
Windows/IAccessible2, maybe Linux/Newton) than others
(macOS/NSAccessibility, Windows/UIA - the latter not used by LO at this
point in time).
though of course it is then ideal to have some nice navigation API
support wrapped around that
What kind of API does that refer to? Existing or new API on the
platform a11y level that LO (or the toolkits it uses) would then
implement, or something else? Do you have anything particular in mind?
I was actually somewhat optimistic about the UIA API Navigation API
conceptually:
https://learn.microsoft.com/en-us/windows/win32/api/uiautomationcore/nf-uiautomationcore-irawelementproviderfragment-navigate
AT-SPI's flows-from and flows-to relations (and ARIA's aria-flowto) seem
somewhat similar to the UIA Navigation API you mention.
If they allow consistent access to off-screen content (related:
tdf#96492), they could potentially be used to retrieve the previous/next
heading,...
IIUC, that would require retrieving one object at a time via the AT-SPI
bus, though, as compared to using e.g. AT-SPI Collection (which can only
take into account objects that are already part of the a11y tree).
Although - I'd really suggest that a11y doesn't work against the
application, and if navigating - it should allow the AT to scroll the
actual visible/view-port to match what is being interrogated.
Interesting thought, and maybe that could be part of the solution, if it
becomes clearer what that can look like in practice.
E.g. it would seem odd to me if an AT starts scrolling through the
document if a "go to next heading/list item" navigation command is
triggered, and then e.g. goes back if it doesn't find anything, because
it can't otherwise access the previously off-screen content to search
for the item.
to look into at some point. My idea so far is to also expose pages on
the a11y level, which should avoid the problem of a single object (the
document) having an enormous amount of children due to that.
If there any general concerns about that, please raise them. :-)
I guess this moves the problem to re-pagination; where we can get
300+ pages re-built for the sake of moving a single paragraph; then
again - I guess if we are notifying changes in position on large sets of
accessible peers we have a similar problem.
Good point! That could indeed be problematic performance-wise.
The feedback I've received from a11y experts so far is that off-screen
doc content should *generally* be exposed on the a11y level, and
limiting Calc to not do that with its huge amount of table cells is
meant to be an exception to the rule in that regard (see e.g. the
discussion in [2] and tdf#156657).
I really think that's a mistake that will ultimately hurt ATs
performance and that we should focus on the end-user use-cases we want
to succeed with - rather than having an abstract absolutist
pre-conception that we can expose everything in an efficient way =)
Sure - if there's a better way to properly make the AT use cases a
reality, then let's go that route instead. :-)
Of course; I'm just one viewpoint. My strong feeling is that
focusing on things that make it easier to code fast, simple ATs that
meet the common use-cases people want is the vital thing; and I really
think that trying to re-build arbitrary document models and synchronize
them on the other end of a bus - particularly if we require lots of
synchronous round-trips to interrogate the content - is not going to fly.
On 11/06/2024 09:49, Michael Weghorn wrote:
Otherwise, as long as the underlying platform a11y protocols are
pull-based and given the input I've received up to this point, I tend
to think that ATs actively querying the tree are primarily
responsible for limiting that to a reasonable amount of information,
but I'm thankful for any guidance here...
Its a nice hope =) I'd want to create APIs that capture the common
things that ATs want to do, make them easy, and really hard to screw up.
But now I shut up ;-) we're working on the web side of this;
caching bits in the browser and adding another protocol latency there -
and I'm sure we want to be handling a reasonably bounded set of data
there =)
Is there an easy way to test COOL a11y web and impacts of potential changes?
(I just opened a sample Writer doc on nextcloud.documentfoundation.org
and couldn't find the doc content via Accerciser in a quick test, but am
also not very familiar with web content/browser a11y.)
As an additional note, one more potential source to get some interesting
insights could be to check how NVDA's browse mode is currently
implemented for MS Word, for example.
Thanks again for your thoughts!
Regards,
Michael
[1] https://bugs.documentfoundation.org/show_bug.cgi?id=156657#c11
[2] https://www.w3.org/WAI/ARIA/apg/patterns/grid/examples/data-grids/
[3]
https://www.w3.org/WAI/ARIA/apg/patterns/grid/examples/advanced-data-grid/
--
To unsubscribe e-mail to: accessibility+unsubscr...@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/accessibility/
Privacy Policy: https://www.documentfoundation.org/privacy