actually there was one more change:
[~/hadoop-src/chukwa/trunk] svn diff
Index:
src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java
===================================================================
---
src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java
(revision
831608)
+++
src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java
(working
copy)
@@ -225,7 +225,9 @@
@Override
public CommitStatus add(List<Chunk> chunks) throws WriterException {
- CommitStatus rv = next.add(chunks); //pass data through
+ CommitStatus rv = ChukwaWriter.COMMIT_OK;
+ if (next != null)
+ rv = next.add(chunks); //pass data through
synchronized(tees) {
Iterator<Tee> loop = tees.iterator();
while(loop.hasNext()) {
@@ -240,7 +242,8 @@
@Override
public void close() throws WriterException {
- next.close();
+ if (next != null)
+ next.close();
running = false;
listenThread.shutdown();
}
[~/hadoop-src/chukwa/trunk]
On Sat, Oct 31, 2009 at 5:11 PM, Thushara Wijeratna <[email protected]> wrote:
> I wanted to use SocketTeeWriter without going through the steps of
> writing to HDFS.
> It seems that PipelineStageWriter is designed to have any number of
> PipeLineable writers, optionally followed by a SeqFileWriter.
> So I changed my collector config :
>
> <property>
> <name>chukwaCollector.writerClass</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
> </property>
>
> <property>
> <name>chukwaCollector.pipeline</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter</value>
> </property>
>
> After doing one minor change to SocketTeeWriter, I could get this to
> work. The advantage is that now I do not need to set up HDFS.
>
> Please let me know if this is something we should patch, I will submit
> the patch.
>
> SocketTeeWriter changes:
>
> [~/hadoop-src/chukwa/trunk] svn diff
> Index:
> src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java
> ===================================================================
> ---
> src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.jav(revision
> 831608)
> +++
> src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.jav(working
> copy)
> @@ -225,7 +225,9 @@
>
> @Override
> public CommitStatus add(List<Chunk> chunks) throws WriterException {
> - CommitStatus rv = next.add(chunks); //pass data through
> + CommitStatus rv = ChukwaWriter.COMMIT_OK;
> + if (next != null)
> + rv = next.add(chunks); //pass data through
> synchronized(tees) {
> Iterator<Tee> loop = tees.iterator();
> while(loop.hasNext()) {
> [~/hadoop-src/chukwa/trunk]
>
