I had originally intended the NullWriter for this purpose. But your approach is cleaner. If you open a JIRA, I'm happy to commit it.
--Ari On Sat, Oct 31, 2009 at 4:31 PM, Thushara Wijeratna <[email protected]> wrote: > actually there was one more change: > > [~/hadoop-src/chukwa/trunk] svn diff > Index: > src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java > =================================================================== > --- > src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java > (revision > 831608) > +++ > src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java > (working > copy) > @@ -225,7 +225,9 @@ > > @Override > public CommitStatus add(List<Chunk> chunks) throws WriterException { > - CommitStatus rv = next.add(chunks); //pass data through > + CommitStatus rv = ChukwaWriter.COMMIT_OK; > + if (next != null) > + rv = next.add(chunks); //pass data through > synchronized(tees) { > Iterator<Tee> loop = tees.iterator(); > while(loop.hasNext()) { > @@ -240,7 +242,8 @@ > > @Override > public void close() throws WriterException { > - next.close(); > + if (next != null) > + next.close(); > running = false; > listenThread.shutdown(); > } > [~/hadoop-src/chukwa/trunk] > > > On Sat, Oct 31, 2009 at 5:11 PM, Thushara Wijeratna <[email protected]> wrote: >> I wanted to use SocketTeeWriter without going through the steps of >> writing to HDFS. >> It seems that PipelineStageWriter is designed to have any number of >> PipeLineable writers, optionally followed by a SeqFileWriter. >> So I changed my collector config : >> >> <property> >> <name>chukwaCollector.writerClass</name> >> >> <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value> >> </property> >> >> <property> >> <name>chukwaCollector.pipeline</name> >> >> <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter</value> >> </property> >> >> After doing one minor change to SocketTeeWriter, I could get this to >> work. The advantage is that now I do not need to set up HDFS. >> >> Please let me know if this is something we should patch, I will submit >> the patch. >> >> SocketTeeWriter changes: >> >> [~/hadoop-src/chukwa/trunk] svn diff >> Index: >> src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.java >> =================================================================== >> --- >> src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.jav(revision >> 831608) >> +++ >> src/java/org/apache/hadoop/chukwa/datacollection/writer/SocketTeeWriter.jav(working >> copy) >> @@ -225,7 +225,9 @@ >> >> @Override >> public CommitStatus add(List<Chunk> chunks) throws WriterException { >> - CommitStatus rv = next.add(chunks); //pass data through >> + CommitStatus rv = ChukwaWriter.COMMIT_OK; >> + if (next != null) >> + rv = next.add(chunks); //pass data through >> synchronized(tees) { >> Iterator<Tee> loop = tees.iterator(); >> while(loop.hasNext()) { >> [~/hadoop-src/chukwa/trunk] >> > -- Ari Rabkin [email protected] UC Berkeley Computer Science Department
