Luci for Chai Tea
so when designing an accessibility api for an OS, it turns out it looks a heck of a lot like an automation/scripting api, like Mac’s OSI (open scripting interface) which forms the foundation for applescript.
so if you’re designing a new OS workspace environment or GUI toolkit and going “uhg” at the idea of implementing accessibility? think instead of implementing scripting and automation, it’s the exact same thing and way more fun to think about.
Luci for Chai Tea
the basic concept is an application implements a “server” that emits events when parts of the UI change, and recieves events that are requests for information about the UI, or requests to activate things on the screen. The UI itself is expected to have detailed descriptions for each UI element and its location within a heirarchy so screen readers can sufficiently describe “where you are”
Luci for Chai Tea
i just looked up Microsoft’s accessibility API.
its name?
Microsoft UI automation.
because they apparently consider it to be the exact same thing as a scripting interface
Luci for Chai Tea
a client can then subscribe to listen to events, and send events, brokered by the OS. the client can be a refreshable braile display, screen reader, switch controller, ui testing framework, automation scheme, terminal console app… anything that can send and recieve events
Luci for Chai Tea
what’s important here is the exact shape of the events, because accessibility tech that already exists is software that hooks into microsoft’s, apple’s or (linux’s? ) version of this. so if we wanted to make a cross platform one, we’d need to make 3 proxy “servers” that do the translation, subscribing to our generic scripting server and translating our events into WinEvents, etc.
Luci for Chai Tea
in this way it’s an interesting abstract input/output interface that turns your gui app into something like an interactive text adventure.
you are standing in a room with 5 buttons and 6 menus. would you like to go north or south?
Luci for Chai Tea
here’s an interesting and fun aspect: requests for information about the UI could be in the form of something like a database query:
SELECT * heading_elements from Window1
not literally of course, but thinking of a gui window as a kind of database table is fun
Luci for Chai Tea
we could of course make our own clients that hook directly into our “generic” server, but i feel like a rocket load of user research has already gone into the tech that is out there, and it would be silly to assume that stuff is irrelevant/easy to replicate. i say steal what we know works
０７６萌ＳＮＳ is a social network, courtesy of ０７６. It runs on GNU social, version 2.0.2-beta0, available under the GNU Affero General Public License.
All ０７６萌ＳＮＳ content and data are available under the Creative Commons Attribution 3.0 license.