Hi,
you'd need as well to convert these document to pure text in order to
process them; you can try to spawn OOo for conversion in a batch mode
but the easier option is to use unzip in a script, and take content.xml
only from the files. Then process the files using awk (define the field
separator just like you would define the word boundary) and filter out
all words that match <[a-z]+>. This should kill all xml from the files.
Regards
Marcin
ge pisze:
Hello,
I did word collection several times using different sources( web sources)
I use linux, but these tools are also available
for windows as gnu tools.
I used awk, like:
for (i = 1; i <= $NF; i++)
print $i;
This prints each word in a single line.
Then I sorted the file using sort <infile > outfile
and then used further awk scripts to get rid of word endings,
this is probably much easier for Danish, than for Hungarian.
Good luck! Eleonora
[lingu-dev] Help needed - bulk extraction of words
Hi all,
The Danish project has been so fortunate to receive a bunch of articles
from a news magazine. These are odt files and we would like to extract
the words from these documents. We have programs for this purpose, but
we usually get donations one document at the time. This time we have
several thousand documents and I believe it would take about a year to
load these documents one by one.
Do any of you have a program that can extract words from several documents ?
The words will be loaded into our workflow for linguistic processing and
at the end be a part of the Danish spelling directory.
Thanks in advance.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
