I think what you are discussing is about data transformation or data preparation. It is not necessary part of the Mahout project, or maybe the applicants can implement their own InputFormat.
Best, Shunkai -----邮件原件----- 发件人: Paul Elschot [mailto:[EMAIL PROTECTED] 发送时间: 2008年3月21日 19:50 收件人: [email protected] 主题: Re: application of GSoC For text applications, I think we can leave the basic feature (ie. term) extraction safely to Lucene, see the org.apache.lucene.analysis package. Likewise, term vectors in Lucene are pretty close to general feature vectors. Combining features into other features is another thing. For text this boils down to making queries combining basic terms. There are quite a few opportunities for parallelism there, so I'd like this to be part of mahout. I wouldn't know whether a similar breakdown into feature extraction and feature combination also applies to image recognition. Is there (more or less) general purpose software available for basic feature extraction from images? Regards, Paul Elschot Op Friday 21 March 2008 10:08:18 schreef Hao Zheng: > I understand. Actually, I mean the other thing. Maybe "feature > selection" is not precise, let me restate my question. > > Generally, no matter for image recognition or text classification, we > have to ture the original material into a featrue vector. This step > is called "feature extration" or sth like that. My question is will > this step be part of the mahout project? If yes, we have to care > about the transformation step; if not, all we need to process are the > numbers, which will make thing easier. > > On Fri, Mar 21, 2008 at 9:02 AM, Ted Dunning <[EMAIL PROTECTED]> wrote: > > I think a better description is that this project is about ML > > algorithms that need large scale. > > > > If you have very inexpensive feature selection that can run > > sequentially, then it probably doesn't matter to use hadoop/mahout > > for that. Some forms of feature extraction is very expensive, > > however, and could definitely benefit from parallelism. For > > instance, you could imagine that the feature extraction step > > involves a large scale non-deterministic clustering. It might even > > be that the the feature extraction requires parallel processing, > > but the actual learning algorithm does not. > > > > On 3/20/08 5:57 PM, "Hao Zheng" <[EMAIL PROTECTED]> wrote: > > > Another question, this project is all about the ML algorithm > > > itself? all we will deal with is feature vectors/matrix > > > constructed already? that is, the project will not include > > > feature selection part of ML, e.g. extracting feature vector > > > from a document collection?
