On May 6, 2008, at 8:04 AM, phonechen wrote:
sorry , I make a mistake,
what I means is that ,shall we put the doc to be classified to HDFS
and
leave the Model files on the HDFS and
make the whole classify process run on the HDFS,
so what to change is :
=====================
Configuration conf = new JobConf();
FileSystem raw = new RawLocalFileSystem();
raw.setConf(conf);
FileSystem fs = new LocalFileSystem(raw);
==================
to
========================
Configuration conf = new JobConf();
FileSystem fs = new DistributedFileSystem();
=======================
so we can classify a batch of inputs using mapreduce instead of run
multiple
process of ClassifierDriver ,
correct me if there are something wrong.
Ps:Can we make the classify process parallel?
Sure. The Driver class is just an easy way to access the Classifier.
You should be able to call the Classifier as needed as well, and I
suppose we could add Mapper and Reducer for the classification task as
well. I'll look into it the next chance I get, otherwise, feel free
to update my patch w/ your suggestions.
Cheers,
Grant
