Hi All,
I am trying Mahout Naïve Bayes Algorithm for Classification. I have created a
custom sequence file with \t separated with Key as Label and Value as Text
strings.
Now to convert it to vector , I used seq2sparse utility and found out that
vectors are not generating correctly.
I debug each step and below are my findings :
1. My train data contain 83 records in seq file format
2. The output of wordcount, tokenized-document and dictionary.file-0
steps are getting generated correctly
3. Then comes the step of partial-vector-0 generator. In this step the MR
job outputs only 2 records
4. Since this step is incorrect ,so the output of
tf-vectors,frequency.file-0 ,df-count and tfidf-vector output are incorrect.
Final vector file tfidf-vectors contain only 2 vectorized document and that too
not correct.
Output of tfidf-vectors:
Key: /Irrelevant/: Value: {50:1.0,83:1.0}
Key: /Relevant/: Value: {62:1.0,128:1.0,329:1.0,289:1.0}
The command used for seq2sparse is :
bin/mahout seq2sparse -i /data-seq1 -o /data-vectors
Please help me to figure out how can I fix this. To my understanding, the
vectorized document should also have 83 records.
Thanks
Stuti Awasthi
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended
for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information
could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in
transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on
the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the
author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction,
dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written
consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please
delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and
other defects.
----------------------------------------------------------------------------------------------------------------------------------------------------
