Thank you for your kind words, Ted. It does seem a long time ago - and I know lots of others (as you mention) have made lots of changes to the original code so I doubt I'd recognize the code if I looked at it today. I'm glad it has been useful.
Thanks
Bano
On 11/22/05, ted pedersen <[EMAIL PROTECTED]> wrote:
I have, at long last, finally cleaned up SenseTools so that the newest
release of WekaClassify is properly integrated into the package. If you
are using SenseTools 0.3, please consider upgrading to version 0.5 asap.
http://www.d.umn.edu/~tpederse/sensetools.html
(As a reminder, SenseTools is a package that converts text into arff
feature files suitable for use with Weka.)
As you might know, WekaClassify is a program in SenseTools that allows
a user to sense tag text using a stored model (learned by Weka). There
are a number of interesting compatibility issues raised by Weka that have
affected this program over the years. It is important to understand that
the version of Weka you are using determines which version of SenseTools
you must use.
SenseTools 0.5 is compatible with Weka 3-4 (the exact version of Weka I
am now using is 3-4-6). If you are for some reason using the older version
of Weka (3-2, which I would generally discourage) then you need to use
SenseTools 0.3.
There are two issues with WekaClassify - the first is that when moving
from Weka 3-2 to 3-4, the structure of the classes in Weka changed! So
weka.classifiers.j48.J48 became weka.classifiers.trees.J48, which caused
WekaClassifier version 0.3 to break when used with Weka 3-4. So, Bano
kindly provided WekaClassifier version 0.4, which is what is included in
SenseTools 0.5, and will work with Weka 3-4.
In addition, it appears that there are some issues with saving Weka
models. I used to think that you could save a Weka model and distribute it
to other users and they could run it, assuming that they were using the
same version of Weka as you. However, Bano explained to me that the
serializing (a sort of encoding I guess) that Java does of model/objects
makes them have unique identifiers, and that cause them to more or less
only work for a single user on a particular machine! So, rather than
distributing models that were used during Testing as we did in 0.3, I have
modified the test scripts in 0.5 so that they create their own models.
As a final comment, and perhaps this is one I have made before, but I
think SenseTools is really a terrific package that has had a much bigger
impact on our work here than we usually appreciate. It is not as
"glamerous" perhaps as NSP, WordNet-Similarity, or SenseClusters (with
their own web pages, sourceforge projects, mailing lists, etc. :), but
consider the following:
1) SenseClusters uses (and in fact now includes) preprocess.pl and
nsp2regex.pl, both SenseTools programs, and uses the logic of xml2arff.pl.
That benefited Ted, Amruta, and Anagha.
2) The DuluthShell systems that participated in Senseval-2 in 2001 used
SenseTools. That benefited Ted.
3) The DuluthShell and Syntalex systems that participated in Senseval-3
used SenseTools. That benefited Ted and Saif.
4) The WSD-Shell uses SenseTools. That benefited Mahesh. So to summarize
points 2, 3, and 4, all of our previous work in supervised Word Sense
Disambiguation has been on the shoulders of SenseTools.
5) I receive occasional reports from students or other researchers who are
using SenseTools. It does have a user base, and indeed that is how we keep
finding out about these Java/Weka issues. :)
6) Insert your own SenseTools success story here, I am sure to have missed
a few. :)
I have not really explored SenseTools much in recent years, it installs
and I run it, but over the last day or two I have been fiddling around
with it getting 0.5 ready, and I gained a newfound appreciation for how
well it was done, and how well it has held up. There is no doubt in my
mind that the decision to rely on SenseTools for all of the work mentioned
above was very wise indeed, and I think it opens up lots of nice
possiblities for future extensions, etc.
And who do we have to thank for SenseTools? Well, that would be Bano! It
seems like ages ago, and in some sense perhaps it was, but even years
after leaving UMD, SenseTools remains an important part of our
infrastructure here, and that is not likely to change. Well done indeed!
I must also mention that Amruta made a number of important changes to
preprocess and xml2arff a few years after the original release, and
worked very hard at integrating SenseTools in SenseClusters, and that has
really paid off nicely too. And of course Saif integrated and extended
aspects of SenseTools into his Syntalex system, and now Mahesh is doing
the same with his WSD-Shell. So several generations of students have
worked with and extended SenseTools here at UMD, and that will no doubt
continue.
In any case, please check out SenseTools! It's good fun, and if you ever
think about doing supervised or unsupervised learning (and who doesn't
think of that???) then SenseTools has something for you!
Enjoy,
Ted
--
Ted Pedersen
http://www.d.umn.edu/~tpederse
------------------------ Yahoo! Groups Sponsor --------------------~-->
Give at-risk students the materials they need to succeed at DonorsChoose.org!
http://us.click.yahoo.com/iEagnA/LpQLAA/HwKMAA/x3XolB/TM
--------------------------------------------------------------------~->
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/nlpatumd/
<*> To unsubscribe from this group, send an email to:
[EMAIL PROTECTED]
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
YAHOO! GROUPS LINKS
- Visit your group "nlpatumd" on the web.
- To unsubscribe from this group, send an email to:
[EMAIL PROTECTED]
- Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
