Hi flume (cc'd bigtop also)....
I've been playing with the bigtop tests, and this snippet is particularly
difficult to
understand for a flume novice :
private void compressionCommonTest(String id, String decompress, String
glob) {
String node_config =
"node:text(\"events.txt\")|collectorSink(\"${hdfs_sink_dir}\",\"data\");";
sh.exec("export FLUME_CONF_DIR=./${id}",
"flume node_nowatch -s -1 -n node -c '${node_config}'");
assertEquals("Flume failed to accept events",
0, sh.ret);
sh.exec("hadoop fs -cat ${hdfs_sink_dir}/${glob} | ${decompress} | wc
-l");
assertEquals("Wrong # of lines in output found at ${hdfs_sink_dir}",
"10000", sh.out[0]);
}
Can someone explain the logic of it to me? In particular...
1) What is the difference between flume and flume ng - and does "flume_ng"
support these options (i.e. node_nowatch) ?
2) What is the "node_config |" syntax accomplishing?
The entire Flume bigtop smoke test can be found here:
https://github.com/apache/bigtop/tree/master/bigtop-tests/test-artifacts/flume/src/main/groovy/org/apache/bigtop/itest/flumesmoke
--
Jay Vyas
http://jayunit100.blogspot.com
