[
https://issues.apache.org/jira/browse/CHUKWA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714069#action_12714069
]
Jerome Boulon commented on CHUKWA-265:
--------------------------------------
It's working fine for me. Tested using chukwa-0.1.2-rc4
Time between the 2 commands was less that 10 seconds.
So can you do a list before adding the same file again? If the file is still
there then, you could use the agent.log to validate if the file has already
been processed.
You should see something like:
2009-05-28 17:17:22,518 INFO Thread-39 FileAdaptor - Adding
/homes/jboulon/test.txt to the queue
2009-05-28 17:17:22,518 INFO Thread-39 FileAdaptor - /homes/jboulon/test.txt
added to the queue
2009-05-28 17:17:22,518 INFO Thread-39 ChukwaAgent - hardStop on adaptorId: 11,
JobConf 0 /homes/jboulon/test.txt 10
If not then make sure that you inMemoryQueue is not full.
-- Trace --
Connected to localhost (127.0.0.1).
Escape character is '^]'.
add org.apache.hadoop.chukwa.datacollection.adaptor.FileAdaptor JobConf 0
/homes/jboulon/test.txt 0
OK add completed; new ID is 12
list
1)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 381984
2)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Ps 0 /grid/0/chukwa/var/log/Ps.log 50995095
3)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 378774
4)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 0
/grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 599964
5)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Df 0 /grid/0/chukwa/var/log/Df.log 559980
6)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Top 0 /grid/0/chukwa/var/log/Top.log 17933888
7)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Netstat 0 /grid/0/chukwa/var/log/Netstat.log 112626
8)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Iostat 0 /grid/0/chukwa/var/log/Iostat.log 1509872
9)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Sar 0 /grid/0/chukwa/var/log/Sar.log 2817132
10)
org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
HDFSUsage 0 /grid/0/chukwa/var/log/HDFSUsage.log 275070
add org.apache.hadoop.chukwa.datacollection.adaptor.FileAdaptor JobConf 0
/homes/jboulon/test.txt 0
OK add completed; new ID is 13
> FileAdaptor does not remove cleanly from chukwa agent
> -----------------------------------------------------
>
> Key: CHUKWA-265
> URL: https://issues.apache.org/jira/browse/CHUKWA-265
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection
> Affects Versions: 0.1.2, 0.2.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Priority: Blocker
>
> When FileAdaptor shutdown, it should remove itself from adaptorPositions and
> adaptorByNumber. This doesn't appear to be happening correctly, and adding
> the same file for streaming fails.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
