Thanks James! This is an interesting link. I mentioned some of the pitfalls and numerous unsolved problems
in this area in the call we had the other week. As far as I can see, most of the github project consists of
a bolting together of two other projects, MathJax itself, and speak.js, a port of a C++ speech synthesizer
using the wacky JS compilation system "emscripten". As a brief aside, I should mention that such a thing is
far from genuinely wacky any more since as of quite recently, the performance characteristics of JavaScript
code on certainly regular kinds of numerical algorithm when using Google's V8 engine has pulled ahead of the
equivalent code in Java. This is a quite incredible result that I think noone would have predicted even 3
years ago.
The main "guts" of the conversion system appears to lie within this file, and a
few others clustered near it:
https://github.com/gcapiel/mathjax/blob/master/unpackedlanguagebuffer/extensions/jax2MathSpeak.js
I think the system is a good start, but doesn't do anything to attack the kinds of issues that can make
formulas very hard to understand when spoken, which particularly relate to helping the hearer to match
together matching pairs of delimiters or nesting. Here is an example of a formula which I think will be very
hard to tackle -
http://en.wikipedia.org/wiki/Lagrange%27s_trigonometric_identities#Other_sums_of_trigonometric_functions
This is a relatively basic formula which could well appear at the high school level. The difficulty is with
clearly expressing the multiple levels of nesting in the sin functions in the expression on the right hand
side. That said, we should actually put the system to the test! But these are the kinds of things we will
need to deal with in order not to end up with a system which is just unintelligible, no matter how carefully
people listen and how often they repeat the text. I'm also concerned that using MathML as the input format,
some important semantic information may have been lost already - I have a feeling, although little evidence
for it, that processing TeX expressions rather than MathML would lead to a better possibility of preserving
semantically important information such as common nesting level.
I found a link to the system, AsTeR, which I referred to during the talk - it mentions explicitly these
difficult issues I'm describing and presents some solutions to them - e.g. "vary audio state along a
dimension that is orthogonal to (independent of ) the dimension used to convey sub-expressions" - this could
of course only be vindicated using a real user study but at least it is addressing the issue:
http://www.cs.cornell.edu/home/raman/aster/demo.html
Not unexpectedly, this system was put together a while ago by our revered
friend T.V. Raman.
Cheers,
Antranig
On 06/09/2012 13:08, James W. Yoon wrote:
Lisa from ISKME sent along a link to a pretty neat notation(MathML)-to-speech
library. What do folks think
of this? (Antranig, I'm especially interested in your thoughts.)
James
Begin forwarded message:
*Resent-From: *<[email protected] <mailto:[email protected]>>
*From: *Lisa McLaughlin <[email protected] <mailto:[email protected]>>
*Subject: **Aural MathJax Tool Demo from Benetech*
*Date: *4 September, 2012 5:19:31 PM EDT
*To: *"James W. Yoon" <[email protected] <mailto:[email protected]>>
James,
I thought you might be interested in checking this out, benetech sent this to us
Demo of the work in progress in having MathJax generate aural output of MathML:
http://gcapiel.github.com/mathjax/
Let me know if you think it would make sense to consider adding this to open
author.
Thanks and best, Lisa
--
Lisa McLaughlin
Director of Open Knowledge Networks
OER Commons Manager
Institute for the Study of Knowledge
Management in Education (ISKME)
(734) 730-3747
www.oercommons.org <http://www.oercommons.org/>
www.iskme.org <http://www.iskme.org/>
www.twitter.com/lmclaug <http://www.twitter.com/lmclaug>
_______________________________________________________
fluid-work mailing list - [email protected]
To unsubscribe, change settings or access archives,
see http://lists.idrc.ocad.ca/mailman/listinfo/fluid-work
