Hi Ann and list, The following may sound a bit technical but it is crucial that we discuss this: In order to talk about object navigation, three (actually, four) important concepts must be discussed first: screen elements in graphical user interfaces, object hierarchy and how screen readers obtain object information through accessibility API's (and an overview of them; for resident geeks, I need to talk about event loop and handling events): Graphical user interface (GUI), contrary to popular imagination set forth by technical documentation for the blind and visually impaired people, is not linear. When people think of graphical user interfaces and how they are laid out, people may say it is a linear representation like that of a typical document. In reality, GUI elements are hierarchical in nature, and each screen element in GUI's contains other elements or are contained by elements. For example, in Windows Explorer, each icon may represent a drive, a folder, a file or other things. These icons are contained within a window that presents these icons, and each icon may contain additional information. Another example is on the web where a list or a table may contain items such as text, links, headings and so on, and certain web elements are contained inside another web element such as a frame. What I've just described is a parent-child relationship: screen elements (objects) containing other objects. Naturally a follow-up question arises: how do screen readers navigate this complex hierarchy of objects? This is done by utilizing accessibility API's. An accessibility API is a layer in user interface portion of operating systems that allows assistive technologies to gather necessary information about an object, such as its name, value, role (what kind of element it is), location and other identifying information. Microsoft Windows ships with two API's: Microsoft Active Accessibility (sometimes called IAccessible) and UI Automation (UIA). Both API's allow screen readers to obtain the following information about an object: * Name (sometimes called label). * Value (progress bar percentage, text in an eit field and so on). * Role (checkbox, radio button, document and so on). * State (checked, focused, read-only and so on). * Hierarchy (parent (object which contains this object), children (one or more objects that are contained within this screen element), next and previous (sibling objects, objects that are contained by the same parent). In effect, when you use touch cursor or equivalent functionality in other screen readers, you are basically telling the screen reader to consult an accessibility API to locate screen elements that you are interested in in relation to the object you are working with. There are tutorials out there that explains how to use object navigation in various screen readers (my own tutorial on another screen reader covers this in more detail; please ask me offlist if interested or would like to have the link to the tutorial series). Bonus: Applications that utilize GUI's are event-driven. In other words, these programs (screen readers included) are sensitive to events around the user interface, such as clicks, keyboard input, changed state of an object and so on. Somewhere inside a GUI application (including jfw.exe) is an event loop that allows applications to receive various events and process them (called event handling or callbacks). JAWS and other screen readers can be told to monitor specific events such as state change, something appears on screen and so on. To illustrate this, let me show you what really goes on when JAWS is attempting (or should attempt) to announce toast notifications. A toast notification is a rectangular window that appears when an app needs to notify a user about something. This includes auto play, news from apps, reminders and others. Toasts are UIA objects, thus screen readers are expected to listen to UIA events for this object and act accordingly (as some of us found out, current JAWS 17 version does not recognize certain UIA events and does not track certain objects). An ideal interaction between JAWS, UIA and toasts is as follows: 1. Freedom scientific developers (or other script writers) takes note of technical information about a toast, such as its window class name and other crucial information (Windows 8.x and 10 uses two differing information about toasts, hence this may explain why JFW 17 does not read toasts in Windows 10 correctly). 2. A toast appears. Windows raises a show event in hopes that a screen reader will announce them (Narrator and certain screen readers do take note of this). 3. Because toasts are UIA objects, screen readers will (or are asked to) consult a table of UIA objects, locate toast handler code and will perform needed action such as announcing toast notifications (a screen reader will not be complete without support for accessibility API's; any screen reader claiming to support Windows 7 and later MUST implement UIA support, especially now that Windows 10 is here to stay and scripts for modern (universal) apps calls for use of UIA). In closing, I believe that it is essential that screen reader users learn a thing or two about internal operation of a screen reader. Not only it helps in appreciating the work involved to make our lives more productive, but also allows us to explain how a screen reader works in hopes of letting app developers and others think about accessibility. I'd like to leave you with a quote I heard a while back that makes more sense now: whatever is done in secret, it'll be shouted from rooftops. Cheers, Joseph
-----Original Message----- From: Ann Byrne [mailto:[email protected]] Sent: Thursday, February 11, 2016 5:11 AM To: [email protected] Subject: What is object navigation? >What is object navigation? somehow I can't sort it out. Thanks, >Despite its name, touch cursor isn't limited to tablets - it is >actually a fancy name for object navigation. Touch cursor comes in >handy if you need to work with modern apps (including universal apps) >where object navigation/touch cursor usage is essential. > >Cheers, > >Joseph -=-=-=-=-=-=-=-=-=-=-=- -- JFW related links: JFW homepage: http://www.freedomscientific.com/ JFW List instructions: To post a message to the list, send it to [email protected] To unsubscribe from this mailing list, send a message to [email protected] Archives located at: http://www.groups.io/g/jfw/threads Alternative archives located at: http://n2.nabble.com/JAWS-for-Windows-f2145279.html If you have any concerns about the list, posts received from the list, or the way the list is being run, do not post them to the list. Rather contact the list owner at [email protected]. -=-=- Groups.io Links: You receive all messages sent to this group. View This Message (#38525): https://groups.io/g/jfw/message/38525 Mute This Thread: https://groups.io/mt/605477?uid=21656 Change Your Subscription: https://groups.io/g/jfw/editsub?uid=21656 Unsubscribe: https://groups.io/g/jfw/leave/46502/1292106160/xyzzy Group Home: https://groups.io/g/jfw Contact Group Owner: [email protected] Terms of Service: https://groups.io/static/tos -=-=-=-=-=-=-=-=-=-=-=-
