Awesome. Thanks for the contribution. Two comments: - Can you open a JIRA for the issue? That's our preferred way of managing patches. (https://issues.apache.org/jira/browse/CHUKWA) - Can you attach a copy of the stack trace you got from the NPE? That'll help us figure out what your conf problem was, and update the documentation (or code) accordingly.
--Ari On Tue, Mar 16, 2010 at 4:30 PM, Kirk True <[email protected]> wrote: > Hi all, > > I'm getting a NullPointerException showing up in the postprocess.log file > because I don't have things configured quite right. Attached is a patch > against the trunk which allows me to get past this. I'm not suggesting this > is the correct fix :) Obviously I should configure things properly, but in > the absence of that, we should give the user something instructive to help > him fix the problem. > > Thanks, > Kirk > > Index: > src/java/org/apache/hadoop/chukwa/extraction/demux/PostProcessorManager.java > =================================================================== > --- > src/java/org/apache/hadoop/chukwa/extraction/demux/PostProcessorManager.java > (revision 924034) > +++ > src/java/org/apache/hadoop/chukwa/extraction/demux/PostProcessorManager.java > (working copy) > @@ -170,22 +170,28 @@ > public boolean processDemuxPigOutput(String directory) throws IOException > { > long start = System.currentTimeMillis(); > try { > - String[] classes = conf.get(POST_DEMUX_DATA_LOADER).split(","); > - for(String dataLoaderName : classes) { > - Class<? extends DataLoaderFactory> dl = (Class<? extends > DataLoaderFactory>) Class.forName(dataLoaderName); > - java.lang.reflect.Constructor<? extends DataLoaderFactory> c = > - dl.getConstructor(); > - DataLoaderFactory dataloader = c.newInstance(); > + String postDemuxDataLoaderClasses = conf.get(POST_DEMUX_DATA_LOADER); > + > + if (postDemuxDataLoaderClasses != null) { > + String[] classes = postDemuxDataLoaderClasses.split(","); > + for(String dataLoaderName : classes) { > + Class<? extends DataLoaderFactory> dl = (Class<? extends > DataLoaderFactory>) Class.forName(dataLoaderName); > + java.lang.reflect.Constructor<? extends DataLoaderFactory> c = > + dl.getConstructor(); > + DataLoaderFactory dataloader = c.newInstance(); > > //DataLoaderFactory dataLoader = (DataLoaderFactory) Class. > // forName(dataLoaderName).getConstructor().newInstance(); > - log.info(dataLoaderName+" processing: "+directory); > - StringBuilder dirSearch = new StringBuilder(); > - dirSearch.append(directory); > - dirSearch.append("/*/*/*.evt"); > - Path demuxDir = new Path(dirSearch.toString()); > - FileStatus[] events = fs.globStatus(demuxDir); > - dataloader.load(conf, fs, events); > + log.info(dataLoaderName+" processing: "+directory); > + StringBuilder dirSearch = new StringBuilder(); > + dirSearch.append(directory); > + dirSearch.append("/*/*/*.evt"); > + Path demuxDir = new Path(dirSearch.toString()); > + FileStatus[] events = fs.globStatus(demuxDir); > + dataloader.load(conf, fs, events); > + } > + } else { > + log.warn(POST_DEMUX_DATA_LOADER + " value was null. Please check > configuration"); > } > } catch(Exception e) { > log.error(ExceptionUtil.getStackTrace(e)); > > -- Ari Rabkin [email protected] UC Berkeley Computer Science Department
