Hello, Matthew. The problem is sure feasible, you can test a great implementation here: http://webdemo.visionobjects.com Unfortunately I haven't found any opensource formula recognition tools, so maybe there is no code to adapt. The are lots of articles on the topic even from the 80s. For example, there is a recent survey article<http://link.springer.com/article/10.1007/s10032-011-0174-4>about math expressions retrieval. The three main steps are as follows: symbol shape classification, formula layout estimation and syntax analysis. Most of the approaches try to parse printed formulas, in our case the task is rather different - the formula is already detected and binarized, but the shapes may significantly differ. Robust shape estimation looks like the most challenging part, though it's the one best known to me. For the syntax analysis it could be useful to adapt some of sympy parsing functionality. Also if the project proves to be too difficult for one summer, even single character recognition with smart interface can provide good usability.
среда, 12 марта 2014 г., 23:03:39 UTC+4 пользователь Matthew написал: > > Handwritten input sounds neat. I use and find great value in > http://detexify.kirelabs.org/classify.html > > Handwritten mathematical expression recognition is general and useful > enough that I'd like to see it done outside of SymPy and then imported, > this way other projects can also use it. > > Fedor, can you provide some background on this topic? What other attempts > have there been? Are there other systems? What resources have you found? > Is this even a feasible problem? What are the hard parts? > > -Matthew > > > On Wed, Mar 12, 2014 at 11:46 AM, Fedor Morozov > <[email protected]<javascript:> > > wrote: > >> Hello, David. >> Unfortunately I have no experience with JavaScript yet, but since the >> main challenge of the project is the backend part (formula recognition), I >> can get enough JavaScript knowledge before the coding phase starts. >> I also think that it would be great to develop Sympy Gamma mobile app for >> the project and I have some skills for this (Java and Android programming). >> Now that the idea is confirmed to be useful I'll take a look into the >> issues list, thank you. >> >> среда, 12 марта 2014 г., 20:38:33 UTC+4 пользователь David Li написал: >> >>> Hello, >>> >>> This sounds like a great idea for Gamma. How familiar are you with web >>> development? In particular, do you have any experience with JavaScript? >>> >>> I would also encourage you to look at the application >>> template<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fsympy%2Fsympy%2Fwiki%2FGSoC-2014-Application-Template&sa=D&sntz=1&usg=AFQjCNHo2vVXxypTvp0G5lRmErpTdAsBRQ> >>> and >>> also perhaps contribute to SymPy if you haven't already (as it is a >>> requirement). >>> >>> David >>> >>> On Tuesday, March 11, 2014 5:48:21 PM UTC-7, Fedor Morozov wrote: >>>> >>>> Hello. >>>> My name is Fedor Morozov, I'm a third year CS student at Moscow State >>>> University specializing in computer vision. >>>> >>>> Since touchscreen devices are highly popular nowadays, there are lots >>>> of cases when it's easier to handwrite a formula then to enter it using a >>>> keyboard. >>>> State-of-the arts computer vision approaches achieve good results in >>>> formula recognition, so it is possible to implement such a feature for >>>> SymPy Gamma. >>>> Along with NLP parsing this would provide a very powerful user input >>>> capabilities. >>>> There are already some similar 3rd-party applications for Wolfram >>>> Alpha, you can see an example here <http://webdemo.visionobjects.com>. >>>> A more challenging task would be formula recognition from photos, but >>>> that's not necessary right now, I guess. >>>> >>>> So the proposed project is to implement formula recognition >>>> capabilities for SymPy Gamma, it would also be great to make some >>>> interface >>>> for touchscreen input (web/mobile app?). >>>> I've recently read a few articles on the topic, if you believe that the >>>> project if useful, I'll come up with a full-fledged proposal and >>>> algorithms >>>> review. >>>> The easiest (conceptually) approach would be to select some of the >>>> recent articles and code up the algorithm. >>>> Than some improvements can be made, for example it would be cool to use >>>> SymPy resources for formula correctness checking. >>>> >>>> As stated above, I specialize in computer vision, so I'm familiar with >>>> basic algorithms in the area. >>>> Now I'm researching feature descriptors, so I have some experience with >>>> shape recognition (for example, shape context), a year ago I worked on a >>>> car plate recognition task. >>>> Also last year I participated in GSoC with computational photography >>>> project for opencv library. >>>> >>>> Thanks for your attention, hope to hear some comments soon. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] <javascript:> >> . >> Visit this group at http://groups.google.com/group/sympy. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/sympy/307c7fbd-50fd-4358-a767-5d6460bb5193%40googlegroups.com<https://groups.google.com/d/msgid/sympy/307c7fbd-50fd-4358-a767-5d6460bb5193%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/371a8501-c8ac-477d-9dd8-e1e4f1a957b9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
