[ https://issues.apache.org/jira/browse/METRON-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Zeolla updated METRON-2017: ------------------------------- Description: In METRON-1990, the `process_data_file.sh` script was modified to use `xargs` instead of `find -exec` in order to exit nonzero when `bro` encountered failures when parsing the provided pcap files. In some cases, this is causing a parsing error because the `xargs` command is providing the output of the find command to `bro` twice (as shown below). This is the effective command being run after removing the find and xargs: ``` [root@7fb8a51d00ba exercise-traffic_pcap]# bro -r /root/data/example-traffic/exercise-traffic.pcap /usr/local/bro/share/bro/site/local.bro -C /root/data/example-traffic/exercise-traffic.pcap error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unknown identifier K, at or near "K" ``` The fix is to simplify the command and allow the pcap to be provided solely at the end of the bro call. was: In METRON-1990, the `process_data_file.sh` script was modified to use xargs instead of find with -exec in order to properly exit nonzero when the scripts encountered failures. In some cases, this is causing a parsing erro because the xargs command is providing the output of the find command twice. The result is that xargs is sometimes being passed the pcap file in two places, which results in the below error. This is the effective command being run after removing the find and xargs: ``` [root@7fb8a51d00ba exercise-traffic_pcap]# bro -r /root/data/example-traffic/exercise-traffic.pcap /usr/local/bro/share/bro/site/local.bro -C /root/data/example-traffic/exercise-traffic.pcap error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - � error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized character - error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unknown identifier K, at or near "K" ``` The fix is to simplify the command and allow the pcap to be provided solely at the end of the bro call. > The Bro plugin docker data processing script incorrectly runs bro > ----------------------------------------------------------------- > > Key: METRON-2017 > URL: https://issues.apache.org/jira/browse/METRON-2017 > Project: Metron > Issue Type: Bug > Reporter: Jon Zeolla > Assignee: Jon Zeolla > Priority: Minor > > In METRON-1990, the `process_data_file.sh` script was modified to use `xargs` > instead of `find -exec` in order to exit nonzero when `bro` encountered > failures when parsing the provided pcap files. In some cases, this is causing > a parsing error because the `xargs` command is providing the output of the > find command to `bro` twice (as shown below). This is the effective command > being run after removing the find and xargs: > ``` > [root@7fb8a51d00ba exercise-traffic_pcap]# bro -r > /root/data/example-traffic/exercise-traffic.pcap > /usr/local/bro/share/bro/site/local.bro -C > /root/data/example-traffic/exercise-traffic.pcap > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - � > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: > unrecognized character - > error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unknown > identifier K, at or near "K" > ``` > The fix is to simplify the command and allow the pcap to be provided solely > at the end of the bro call. -- This message was sent by Atlassian JIRA (v7.6.3#76005)