--- Behdad Esfahbod <[EMAIL PROTECTED]> wrote:
> Can you please educate us on how these are supposed
> to work? I
> can't get anything out of them. I choose UTF-8, and
> type a verb
> in the stemmer, I get back the verb verbatim.
Sorry about the late reply. The perl script is run
from the command-line, taking input from STDIN and
outputing to STDOUT under Unix/Linux/Cygwin.
I just finished version 0.7, which natively supports
input and output to & from UTF-8, CP-1256 (aka.
Windows-1256), and ISIRI 3342. The default input &
output is romanized text. This version also fixes
some bugs (eg. --root). A tentative name for the
stemmer is "Perstem". If any of you think of a better
name, please let me know. Also, I'll try to get the
web page version updated sometime soon, which
hopefully will fix the problem that Behdad mentioned.
Sample usage might include (after removing the single
quotation marks around commands):
Input a UTF-8 webpage and output to CP-1256,
preserving only the roots of words, and remove HTML
'perstem -i utf8 -o cp1256 --root --noroman <
my_utf8.html > my_cp1256.txt'
Input romanized sentence from the command-line, output
to UTF-8, show the morphological links, remove short
vowels, and tokenize punctuation:
'echo "man ketAb-hAie tu rA nemi-binam." | perstem -o
utf8 --links --unvowel --tokenize > my_utf8.txt'
For a full list of commands, try the -h or --help
The stemmer and syntax parser were well recieved last
week at the First International Conference on Aspects
of Iranian Linguistics in Leipzig, Germany.
Thanks for all your feedback so far,
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
PersianComputing mailing list