El 2017-04-02 18:38, Mikel L. Forcada escribió: > HI there, > > > El 02/04/17 a les 15:30, Francis Tyers ha escrit: >> El 2017-04-02 12:27, Mikel L. Forcada escribió: >>> Dear Masha, >>> >>> I still have to be convinced that this is an Apertium project as I do >>> not see any connection to the core in Apertium, namely, >>> shallow-transfer rule-based machine translation (other mentors may >>> correct me as I may have missed something). >>> >>> What would be the impact of having this tool in improving the >>> language >>> resources currently used in Apertium? How would it improve Apertium >>> to >>> be used as a machine translation system or a better language >>> processor? >> Here are my thoughts: >> >> 1) The standardised annotated corpora available through the UD project >> could be immensely useful to training the disambiguation >> component >> of Apertium. > You bet. But full dependency trees are not needed for disambiguation > purposes. The statistical (HMM, sliding-window) PoS taggers do not need > such richly-annotated corpora.
They are not, and nor does the perceptron tagger, but I don't know of any plain tagged corpora (e.g. that don't have dependency annotation) that cover so many languages in a standardised way. Most tagged corpora (that just have part-of-speech tags and no other annotation) are either not uniformly annotated with tags and/or are not free/open-source. >> 2) Our current annotatrix tool is functional, but in practice we are >> unlikely to get many people doing Apertium-specific annotation. >> Thus >> if we want to be able to train (e.g. the tagger) we should get a >> "cheaper ride" >> by using what is already there and free. > If I understood correctly, getting people to do a more general-purpose > annotation (such as dependency syntactic annotation) would be easier > and > we could get what we need in Apertium as a side product. Yes, exactly. > If this is the case, this should be clearer in the proposal. Agree! >> 3) In the Turkic languages we are already doing a lot of annotation, >> which has >> been used for different purposes and there are efforts for other >> languages too: >> >> $ find /home/fran/source/apertium/ -name '*.conllu' | wc -l >> 23 > How much of this is actually being used for MT for these Turkic > languages? As far as I know, most of these languages rely heavily on > CG. > Is there an existing reliable tool or method to extract CG rules from > dependency annotation? No existing tool, but Joonas was working on one. To be able to learn CG rules that aren't just n-grams, the dependency annotation (with its clause structure) will be useful. And while it can't be used for extracting CG rules, it can still be used for training the unigram tagger or the perceptron tagger. (The other taggers are harder to train) >> 4) We have previously done projects (like the HFST project) which have >> immensely helped Apertium, although they weren't specifically >> working >> on Apertium core code. > But HFST processors were direct replacements for lttoolbox-based > morphological processors. The analogy is weak. True, the analogy is weak... :) Fran ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Apertium-stuff mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/apertium-stuff
