[
https://issues.apache.org/jira/browse/METRON-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Zeolla updated METRON-2017:
-------------------------------
Description:
In METRON-1990, the `process_data_file.sh` script was modified to use `xargs`
instead of `find -exec` in order to exit nonzero when `bro` encountered
failures when parsing the provided pcap files. In some cases, this is causing a
parsing error because the `xargs` command is providing the output of the find
command to `bro` twice (as shown below). This is the effective command being
run after removing the find and xargs:
```
[root@7fb8a51d00ba exercise-traffic_pcap]# bro -r
/root/data/example-traffic/exercise-traffic.pcap
/usr/local/bro/share/bro/site/local.bro -C
/root/data/example-traffic/exercise-traffic.pcap
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unknown
identifier K, at or near "K"
```
The fix is to simplify the command and allow the pcap to be provided solely at
the end of the bro call.
was:
In METRON-1990, the `process_data_file.sh` script was modified to use xargs
instead of find with -exec in order to properly exit nonzero when the scripts
encountered failures. In some cases, this is causing a parsing erro because
the xargs command is providing the output of the find command twice.
The result is that xargs is sometimes being passed the pcap file in two places,
which results in the below error. This is the effective command being run
after removing the find and xargs:
```
[root@7fb8a51d00ba exercise-traffic_pcap]# bro -r
/root/data/example-traffic/exercise-traffic.pcap
/usr/local/bro/share/bro/site/local.bro -C
/root/data/example-traffic/exercise-traffic.pcap
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character - �
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unrecognized
character -
error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unknown
identifier K, at or near "K"
```
The fix is to simplify the command and allow the pcap to be provided solely at
the end of the bro call.
> The Bro plugin docker data processing script incorrectly runs bro
> -----------------------------------------------------------------
>
> Key: METRON-2017
> URL: https://issues.apache.org/jira/browse/METRON-2017
> Project: Metron
> Issue Type: Bug
> Reporter: Jon Zeolla
> Assignee: Jon Zeolla
> Priority: Minor
>
> In METRON-1990, the `process_data_file.sh` script was modified to use `xargs`
> instead of `find -exec` in order to exit nonzero when `bro` encountered
> failures when parsing the provided pcap files. In some cases, this is causing
> a parsing error because the `xargs` command is providing the output of the
> find command to `bro` twice (as shown below). This is the effective command
> being run after removing the find and xargs:
> ```
> [root@7fb8a51d00ba exercise-traffic_pcap]# bro -r
> /root/data/example-traffic/exercise-traffic.pcap
> /usr/local/bro/share/bro/site/local.bro -C
> /root/data/example-traffic/exercise-traffic.pcap
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character - �
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1:
> unrecognized character -
> error in /root/data/example-traffic/exercise-traffic.pcap, line 1: unknown
> identifier K, at or near "K"
> ```
> The fix is to simplify the command and allow the pcap to be provided solely
> at the end of the bro call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
