W dniu 02.02.2016 o 18:08, Andriy Rysin pisze: > Hey Marcin > > this is great addition, though I have one remark. Besides valency > information some other type of information could be useful too (if we > starting to head this direction). E.g. I have rules in Ukrainian that > suggests superlative form for adjective when "самий" (very) + base > form is used. Currently I have the relation between base form and > comparative/superlative forms encoded in the dictionary but in general > this is higher-level information that should be stored outside of the > tag dictionary.
I would argue that in some languages (at least in Polish and English) this is not a semantic-level information, this is a grammatical information, or morphosyntactic information. > > I am wondering if we could develop more generic approach for such > additional (semantic) information, e.g. split each type of this info > into category and allow generic references in the token/exception, > something like this: > > <token > semantic_info="<semantic_category_name>:<WHATEVER_STRING_FROM_THAT_CATEGORY>"/> > > or even as a subelement (I assume semantic information can get pretty > long/complicated so child element may be better choice and will allow > to add new attributes easily on it later) > > <token> > <semantic category="<semantic_category_name1>" > value="<WHATEVER_STRING_FROM_THAT_CATEGORY>"/> > <semantic category="<semantic_category_name2>" > value="<WHATEVER_STRING_FROM_THAT_CATEGORY>"/> > </token> > > so in valency case you described (1st case) it could be: > > <token postag="verb"> > <semantic category="valence" value="WHATEVER_STRING"/> > </token> Valency is definitely not a semantic category: https://en.wikipedia.org/wiki/Valency_(linguistics) But your approach seems quite elegant. I would argue that valency is one kind of information that should be treated as key-value <token postag="verb"> <valency category="noun phrase" value="accusative"/> </token> This would match a verb that takes an accusative noun phrase (of course, the values would be defined per valency lexicon in a language). There are free valency lexicons for many languages beside Polish. > > Thus if we add other semantic information into LT we can use this info > in the logic without changing the LT core. The core XML parsing will have to be changed anyway. Best, Marcin > > Thanks > Andriy > > 2016-01-28 7:30 GMT-05:00 Marcin Miłkowski <list-addr...@wp.pl>: >> Hi all, >> >> To allow for better disambiguation and have better rules, I need to >> include a valency dictionary with LT. These are dictionaries that >> specify which grammatical cases or prepositions go with which verbs etc. >> There are such resources for many languages that we support. And using >> these resources, we could enrich POS tag disambiguation a lot (I'm using >> a horribly long regular expression right now instead of a dictionary, >> for example), and write up a lot of important rules. >> >> The obvious choice for representing the dictionary (which is available >> for Polish on a fairly liberal license) is to use a finite-state lexicon >> that we normally use for taggers. The dictionary will be applied after >> tagging because valency dictionary will require POS tag + lexeme >> information. In Polish, the entries look like this: >> >> absurdalny: pewny: : : : {prepnp(dla,gen)} >> absurdalny: pewny: : : : {prepnp(w,loc)} >> absurdalny: pewny: : pred: : {prepnp(dla,gen)}+{cp(gdy)} >> absurdalny: pewny: : pred: : {prepnp(dla,gen)}+{cp(int)} >> absurdalny: potoczny: : pred: : {prepnp(dla,gen)}+{cp(jak)} >> absurdalny: pewny: : pred: : {prepnp(dla,gen)}+{cp(jeśli)} >> absurdalny: pewny: : pred: : {prepnp(dla,gen)}+{cp(kiedy)} >> absurdalny: pewny: : pred: : {prepnp(dla,gen)}+{cp(że)} >> absurdalny: pewny: : pred: : {prepnp(dla,gen)}+{cp(żeby)} >> >> But for French (see http://bach.arts.kuleuven.be/dicovalence/) they are >> paragraph-based: >> >> VAL$ abaisser: P0 P1 >> VTYPE$ predicator simple >> VERB$ ABAISSER/abaisser >> NUM$ 10 >> EG$ il faudra abaisser la persienne >> TR_DU$ laten zakken, neerhalen, neerlaten, doen dalen >> TR_EN$ let down, lower >> FRAME$ subj:pron|n:[hum], obj:pron|n:[nhum,?abs] >> P0$ (que), qui, je, nous, elle, il, ils, on, (ça), (ceci), celui-ci, >> ceux-ci >> P1$ que, la, le, les, en Q, ça, ceci, celui-ci, ceux-ci >> RP$ passif être, se passif >> AUX$ avoir >> >> >> VAL$ abaisser: P0 (P1) >> VTYPE$ predicator simple >> VERB$ ABAISSER/abaisser >> NUM$ 20 >> EG$ il a raconté cette anecdote pour m'abaisser >> TR_DU$ vernederen, kleineren >> TR_EN$ humiliate >> FRAME$ subj:pron|n:[hum], ?obj:pron|n:[hum] >> P0$ (que), qui, je, nous, elle, il, ils, on, (ça), celui-ci, ceux-ci, >> (ça(de_inf)) >> P1$ 0, qui, te, vous, la, le, les, se réc., en Q, celui-ci, ceux-ci, >> l'un l'autre >> RP$ passif être, se faire passif >> AUX$ avoir >> >> >> I would also add a new list of valency attributes to every >> AnalyzedToken, simply as a string value (parsing the string would be >> overkill because there might be different ways of encoding valency >> information for different languages), with appropriate getters and setters. >> >> To use this, we will need some additional attributes in XML elements: >> token and exception. The following notation seems to be fairly fine: >> >> <token postag="verb" valence="WHATEVER_STRING"/> >> >> I think matching valence should, by default, use regular expressions. >> I'm not sure if negation is needed. >> >> Alternatively, we could go for attribute-value pairs but the problem >> would be how to make this language-independent and use this in our XML >> notation. The easiest way would be then probably define types and their >> ids just the way I did this for unification. For French, this could look >> like: >> >> <token>abaisser<valency id="AUX">avoir</valency><valency >> id="VTYPE">predicator simple</valency></token> >> >> Of course, the textual values of <valency> could be made into regexes >> and negated. >> >> For this, of course we would need to write up a key-value parser for all >> particular valency dictionaries. But this would definitely speed up >> matching and would make writing rules much easier. >> >> Best >> >> ------------------------------------------------------------------------------ >> Site24x7 APM Insight: Get Deep Visibility into Application Performance >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month >> Monitor end-to-end web transactions and take corrective actions now >> Troubleshoot faster and improve end-user experience. Signup Now! >> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 >> _______________________________________________ >> Languagetool-devel mailing list >> Languagetool-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/languagetool-devel > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 > _______________________________________________ > Languagetool-devel mailing list > Languagetool-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/languagetool-devel > ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ Languagetool-devel mailing list Languagetool-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/languagetool-devel
