https://bugs.documentfoundation.org/show_bug.cgi?id=35652
--- Comment #32 from Antoine Brûlé <[email protected]> --- I have started working on making browse mode work properly with Writer. The initial goal is to properly expose the tree to Orca in minimal time to be used in a Linux + Orca + GTK3 environment. Once this is done, efforts will go towards making it work for all other major environments, so they will be kept in mind as work progresses. This work is guided by @cwendling on a weekly basis. # Current progress ## Test files Two freely available files as main subjects, to test behavior and performance: - OpenDocument-v1.3-os-part3-schema.odt specification (791 pages), mostly dense text, with tables, footers, links - WG262-WriterGuide.odt (526 pages), less dense, but rich in images, tables, footers, varied paragraph styles If anyone has suggestions of other big realistic files (in .odt or .docx format), please tell me :) ## Time and memory measurements Simply exposing the whole tree works, but needed optimizing to avoid having to wait multiple seconds or timing out Orca when navigating around big documents. After adding some caching to the a11y tree retrieval, and avoiding running the same code multiple times, initial measurements are encouraging on the OpenDocument spec document for a full tree walk: Before fixes (Visible tree only) - Exposed nodes: 16 - Memory before walk: 842.6 MiB - Memory after walk: 842.8 MiB - CPU time to load doc: 22.91s - Wall time to load doc: 32.73s - Cached tree walks: 2ms After fixes (Full tree) - Exposed nodes: 22827 - Memory before walk: 841.2 MiB - Memory after walk: 871.0 MiB (+29.8 MiB) - CPU time to load doc: 22.97s - Wall time to load doc: 33.18s - Cached tree walks: 20-25ms This is with static content. When the content structure changes, e.g. when nodes are added/removed, big parts of the cache may get invalidated (when shifting pages for example), so it may take some time to rebuild on the next lookup at the moment. The extra memory usage is reasonable (+3-4%) but could probably be reduced more. With Orca, the real bottleneck is getting the whole tree even when it is not necessary (when jumping to next heading or paragraph for instance), and then filtering it. I was able to jump to the next heading/paragraph using Atspi.Collection.get_matches_from() limited to 1 node instead of Atspi.Collection.get_matches(). Behavior is the same (in basic scenarios at least), but as only one node is retrieved, this is much quicker: getting the next paragraph goes from 0.9-1.2s to at most 20ms. Moving the caret is what takes the most time. # Remaining issues Relative navigation with Orca (jumping to next/previous paragraph) is broken when repeated elements are exposed: header/footer content present on multiple pages, repeated header on a table split over two pages. In these cases some elements exposed in the a11y tree refer to a single document element, with the caret present in multiple a11y nodes at once. This causes confusion for the AT which leads to a loop. I can provide specific example files for every scenario. They will be used for unit tests later on. Repeated elements can be handled on AT side with special guards (properly identifying where the caret should go with ambiguous destinations), but properly exposing them would be a better option. I am still unsure how headers/footers/footnotes should be exposed. Currently they are exposed as "interleaved" between body paragraphs, and can even be present between 2 parts of a paragraph/table split over 2 pages. This does not seem optimal, this feels more appropriate to expose on a page level (see macOS behavior below). @joanmarie, do you have an opinion on this? ## MS Word behavior I have done some manual testing with MS Word, to have a better idea of the "expected" behavior: ### Behavior on Windows + NVDA Tested on W11 + NVDA 2026.1.1.55980 + MS Word 365 version 2609 - The full tree is exposed, though it takes a while to come up with NVDA+F7 on big documents - unsure where the bottleneck is. - Headers/footers are not reachable directly, though they may be exposed somehow. They are reachable by keyboard using the interface menus. - Pages as a whole do not seem reachable - Tables are unified across pages, as if there was no page break and no repeated header. - Paragraphs are unified across pages - Jumping to next heading works. - Footnotes are skipped when jumping to next paragraph, but reachable via link to footnote. I haven't tried with JAWS. ### Behavior on macOS + VoiceOver Tested on macOS Sonoma 14.9.8 + MS Word 16.113.3 Behavior may be different on more recent versions. - Full tree is exposed, quick to navigate. - Possible to choose whether to navigate on pages/headers/footers/floating elements as a whole, or only the body structure. - Tables are unified across pages. - I did not find a way to jump to next paragraph (with built-in shortcuts), only line by line. - Jumping to next heading does not work (no heading found, though they exist in the full heading list "Rotor"). - Footnotes are skipped, reachable via link. -- You are receiving this mail because: You are the assignee for the bug.
