Hi,
I am new to Flume, was trying out some basic's on my Windows 7 dev machine and
I ran into problems with the spooling directory source, the issue was in the
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile()
method... we were getting an instance of a DurablePositionTracker and not
calling close on it before then creating another instance... thus not calling
close on the underlying DataFileWriter. This left the file locked in Windows
and hence any delete attempts were failing...causing the Agent to spin around
and around trying to delete this file and creating endless numbers of the
temporary tracker files.
I have the code change here and can submit a JIRA item, I don't see anything
similar in JIRA, but before I created something I wanted to discuss this in the
community (realizing that Windows is not supported)
My code change is very simple, I added the close() call in the else block :
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile()
....
if (!tracker.getTarget().equals(nextPath)) {
tracker.close();
deleteMetaFile();
}
else{
//Explicitly close the tracker instance received so that
//there is no sharing violation on Windows.
tracker.close();
}
tracker = DurablePositionTracker.getInstance(metaFile, nextPath);
....
Thanks
Phil
Phil Scala
Developer / Architect
Global Relay
[email protected]<mailto:[email protected]>
866.484.6630
New York | Chicago | Vancouver | London (+44.0800.032.9829) |
Singapore (+65.3158.1301)
Global Relay Archive supports email, instant messaging, BlackBerry, Bloomberg,
Thomson Reuters, Pivot, YellowJacket, LinkedIn, Twitter, Facebook and more.
Ask about Global Relay Message<http://www.globalrelay.com/services/message> -
The Future of Collaboration in the Financial Services World
All email sent to or from this address will be retained by Global Relay's email
archiving system. This message is intended only for the use of the individual
or entity to which it is addressed, and may contain information that is
privileged, confidential, and exempt from disclosure under applicable law.
Global Relay will not be liable for any compliance or technical information
provided herein. All trademarks are the property of their respective owners.
