Bryan, That was a brilliant suggestion. I got it to work. Many thanks.
Huagen > 在 2016年6月3日，下午12:51，Bryan Rosander <[email protected]> 写道： > > Hi Huagen, > > It looks to me like the ExecuteStreamCommand is designed to run a single > command. A potential work around could be to make a shell script that > executes the pipeline you’d like and then use the ExecuteStreamCommand to > invoke that. > > I’d use your shell command (e.g. /bin/bash) for the command and the script > name for the first argument. > (http://stackoverflow.com/questions/18240944/cannot-launch-shell-script-with-arguments-using-java-processbuilder#answer-18241604) > > <http://stackoverflow.com/questions/18240944/cannot-launch-shell-script-with-arguments-using-java-processbuilder#answer-18241604)> > > If any of this info is bad, hopefully someone else will chime in. I’m still > new around here J > > Thanks, > Bryan Rosander > > From: Huagen peng <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Friday, June 3, 2016 at 12:38 PM > To: "[email protected]" <[email protected]> > Subject: Piping commands in ExecuteStreamCommand/ExecuteProcess > > Hi, > > I was trying to get the ExecuteStreamCommand processor to execute one command > and then pipe the result right to another Linux command in the same > processor. This is how I try to configure the processor: > > Property Value > Command Arguments -r;${log_dir};|;bzip2;-;>;/tmp/test.bz2 > Command Path zcat > > The error I got is this: > execution.command > zcat > execution.command.args > -r;/data/similarweb/staging/170_16-05-22_01/log;|;bzip2;-;>;/tmp/test.bz2 > execution.error > zcat: can't stat: | (|.gz): No such file or directory zcat: can't stat: bzip2 > (bzip2.gz): No such file or directory gzcat: unknown compression format > gzcat: can't stat: > (>.gz): No such file or directory gzcat: can't stat: > /tmp/test.bz2 (/tmp/test.bz2.gz): No such file or directory > previous > Empty string set > > > I understand that I can execute the bzip2 command in a second > ExecuteStreamCommand processor or simply use the CompressContent processor, > and I event got them to work. The issue I was trying to avoid is to have a > lot of data in the data flow clogging up the memory. What I wanted to > achieve in my naive approach is to leverage the series of Linux commands do > that job for me without bringing the huge amount of data into the multiple > steps of flow. > > Can piping commands in the processor be done in NiFi? > > Thanks, > > Huagen > > >
