[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15632769#comment-15632769
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/276


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Assignee: Kyle Richardson
>Priority: Minor
> Fix For: 0.2.2BETA
>
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[Otto Fowler (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631257#comment-15631257
 ] 

Otto Fowler commented on METRON-363:


I'm sorry, I didn't mean for it to change assign to me.

> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Assignee: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15626045#comment-15626045
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Ok, need some helping figuring out why the CI build keeps failing...

I get several of these at the end of the log:
```
Running org.apache.metron.parsers.integration.JSONMapIntegrationTest
2016-11-01 15:54:52 FATAL KafkaServer:116 - [Kafka Server 0], Fatal error 
during KafkaServer startup. Prepare to shutdown
kafka.common.KafkaException: Socket server failed to bind to 
localhost:6667: Address already in use.
```

and prior to that I see:
```
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.64 sec 
<<< FAILURE! - in org.apache.metron.parsers.integration.YafIntegrationTest
test(org.apache.metron.parsers.integration.YafIntegrationTest)  Time 
elapsed: 8.637 sec  <<< ERROR!
java.lang.NoClassDefFoundError: org/slf4j/event/LoggingEvent
```

This occurred for both of the CI builds since I rebased to the latest 
master. Any ideas?



> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623990#comment-15623990
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Rebased against master to incorporate the global junit version change. 
Should be good to go now pending Travis.

Thanks again to everyone for all of the suggestions, feedback, and testing.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623711#comment-15623711
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user james-sirota commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
+1. Great job.   Any more revisions you want to make to this?  Or are we 
good to commit? 


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621164#comment-15621164
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user james-sirota commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Still testing...bare with me


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15619005#comment-15619005
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r85651135
  
--- Diff: metron-platform/metron-parsers/src/main/resources/patterns/asa ---
@@ -107,7 +108,7 @@ COMBINEDAPACHELOG %{COMMONAPACHELOG} %{QS:referrer} 
%{QS:agent}
 LOGLEVEL 
([A|a]lert|ALERT|[T|t]race|TRACE|[D|d]ebug|DEBUG|[N|n]otice|NOTICE|[I|i]nfo|INFO|[W|w]arn?(?:ing)?|WARN?(?:ING)?|[E|e]rr?(?:or)?|ERR?(?:OR)?|[C|c]rit?(?:ical)?|CRIT?(?:ICAL)?|[F|f]atal|FATAL|[S|s]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)
 
 #== Cisco ASA ==
-CISCO_TAGGED_SYSLOG ^<%{POSINT:syslog_pri}>%{CISCOTIMESTAMP:timestamp}( 
%{SYSLOGHOST:sysloghost})? ?:? %%{CISCOTAG:ciscotag}:
--- End diff --

The ASA patterns build off of several of the more generic patterns 
referenced earlier in the file; however, I should be able to reduce it down to 
just the ones being used.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15612856#comment-15612856
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user james-sirota commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Testing this in production this week on production hardware.  Will have 
feedback in the next few days 


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606114#comment-15606114
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Any other feedback or suggestions for me?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588679#comment-15588679
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Whew, got the CI build to finally pass. All integration and unit tests are 
passing. I've also re-testing in the single node vm environment I described 
above.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[Dima Kovalyov (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588577#comment-15588577
 ] 

Dima Kovalyov commented on METRON-363:
--

Hello,

First of all, thank you for a huge effort on developing this parser!

We were about to develop our own Cisco ASA parser, but stumbled across this 
case. Can you please advise if this parser is ready for testing and deployment?



> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588558#comment-15588558
 ] 

ASF GitHub Bot commented on METRON-363:
---

GitHub user kylerichardson reopened a pull request:

https://github.com/apache/incubator-metron/pull/276

METRON-363 Fix Cisco ASA Parser

I've rewritten the ASA parser which can be extended, as needed, to new ASA 
message types by editing the bundled asa patterns file and the static map used 
for grok patterns in the code. I've also tried to make it easier to deploy the 
asa topology by including zookeeper config files and creating the kafka topic 
during metron install. Sample data is also included for integration testing.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kylerichardson/incubator-metron METRON-363

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/276.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #276


commit 5be7c60448f73fcc72c81451a67ef1e40fd29793
Author: kylerichardson 
Date:   2016-08-16T01:12:42Z

Initial rewrite of Cisco ASA parser

Summary of changes:
- Complete rewrite of ASA parser including new test suite
- ZK configurations for ease of topology deployment (parser and enrichment)
- Add field constant for original_string in metron-common
- Minor changes to ASA patterns file for
  (1) Syslog severity/facility capture
  (2) Interface capture on CISCOFW106006_106007_106010
- Updates to various POMs to allow easier validation of logging during unit 
testing
  (1) Exclusions for slf4j-log4j12 on various dependencies for 
metron-parsers and metron-integration-test
  (2) Explicit dependency on slf4j-api for metron-parsers
  (3) Test dependency on slf4j-simple for metron-parsers

commit c87e6edaf0e308be9f417e07016508f87067ae0c
Author: kylerichardson 
Date:   2016-09-20T02:33:09Z

METRON-363 Reworked parser to handle nulls and field validation

Includes the following:
- Static map for ASA message patterns (vs pattern discovery)
- Minor changes to ASA patterns file
- Broke out common syslog parsing elements
- Broke out reusable field validations

commit a8c4903dd0bcac18e15c98aca7264dce1c455bee
Author: kylerichardson 
Date:   2016-09-27T00:30:16Z

METRON-363 Add integration test and sample data

Includes the following:
- Extend BasicParser
- Handle both types of syslog timestamps (with and without year)
- Include integration test and supporting sample data

commit 011d389bdf43f1790384dbcd13ec7da148c53ef2
Author: kylerichardson 
Date:   2016-09-27T00:40:51Z

METRON-363 Add license and kafka topic

commit 04a936d75cf782254105993b2804912b4659257a
Author: kylerichardson 
Date:   2016-09-28T00:29:21Z

METRON-363 Adjust log level

commit abd7fb92fe4c38530e10141d0aba6bd07a335ae8
Author: kylerichardson 
Date:   2016-10-08T01:11:22Z

METRON-363 Enhance logging, remove unused code

commit a885ecc762a8d5296d7c7ebfe7600c910ce3478b
Author: kylerichardson 
Date:   2016-10-11T17:40:25Z

METRON-363 Refactored and enhanced based on feedback

Changes include:
(1) New/additional unit tests
(2) Reworked Syslog Timestamp (no year) logic
(3) Enhanced error checking and logging (introduced new ParseException)

commit fb6ed83eab8704607dc75c37982b0f98b819047d
Author: kylerichardson 
Date:   2016-10-12T13:54:54Z

METRON-363 Default to UTC in zookeeper config

commit d7d327a3b03584fd3d03d4f6468d54c15786bda7
Author: kylerichardson 
Date:   2016-10-13T02:10:14Z

METRON-363 Update tests

commit 4e3cba6682eaf3130325d4c27bf32240ad7a0a92
Author: kylerichardson 
Date:   2016-10-18T00:33:34Z

METRON-363 Refactor to add Clock dependency for testing

commit db8686615533470e8a3273ee268f2eb0efb4999c
Author: kylerichardson 
Date:   2016-10-18T01:15:29Z

METRON-363 Add tests for back dating RFC3164 timestamps




> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588542#comment-15588542
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson closed the pull request at:

https://github.com/apache/incubator-metron/pull/276


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581043#comment-15581043
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user mattf-horton commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83572123
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
 ---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --

Bummer.  Sorry, I have no advice.  Any Mockito experts out there?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15580412#comment-15580412
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83558126
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
 ---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --

Yes, in theory that is.

I tried doing just that with PowerMock and Mockito; unfortunately, it seems 
there is a bug in the underlying `javassist` library that doesn't play well 
with the new `java.time` classes.

The issue is reportedly fixed in Javassist 3.20.0-GA; however, it doesn't 
appear that PowerMock has updated to this version.

Reference: [JASSIST-246](https://issues.jboss.org/browse/JASSIST-246) and 
https://github.com/jayway/powermock/issues/557


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15577117#comment-15577117
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user mattf-horton commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83523155
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
 ---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --

Wouldn't it suffice to Mock the ZonedDateTime.now() call?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576987#comment-15576987
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83520638
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
 ---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --

Agreed. There currently isn't test coverage for that logic.

I was trying to avoid having to add a dependency on a Clock object but it 
may be the only way to throughly test this code.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576964#comment-15576964
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83520042
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
+int year = now.getYear();
+if (now.getDayOfYear() == 1 && 
!now.getMonth().toString().substring(0,3).equals(logTimestamp.substring(0,3).toUpperCase()))
+year--;
--- End diff --

I like the idea of checking how far the date in the current year would be 
in the future and basing the back date decision on that. Let me work on that.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572338#comment-15572338
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user mattf-horton commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
I added a comment above, to SyslogUtils.java line 36, which the system did 
not email to the list, probably because I immediately edited it to fix a format 
error.  @kylerichardson please consider it.  Thanks.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569016#comment-15569016
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user mmiklavc commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83029350
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/asa/BasicAsaParserTest.java
 ---
@@ -0,0 +1,149 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import org.json.simple.JSONObject;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.time.*;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.Assert.*;
+
+public class BasicAsaParserTest {
+
+private static BasicAsaParser asaParser;
+
+@BeforeClass
+public static void setUpOnce() throws Exception {
+Map parserConfig = new HashMap<>();
+asaParser = new BasicAsaParser();
+asaParser.configure(parserConfig);
+asaParser.init();
+}
+
+@Test
+public void testConfigureDefault() {
+Map parserConfig = new HashMap<>();
+BasicAsaParser testParser = new BasicAsaParser();
+testParser.configure(parserConfig);
+testParser.init();
+assertTrue(testParser.deviceTimeZone.equals(ZoneOffset.UTC));
+}
+
+@Test
+public void testConfigureTimeZoneOffset() {
+Map parserConfig = new HashMap<>();
+parserConfig.put("deviceTimeZone", "UTC-05:00");
+BasicAsaParser testParser = new BasicAsaParser();
+testParser.configure(parserConfig);
+testParser.init();
+ZonedDateTime deviceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
testParser.deviceTimeZone);
+ZonedDateTime referenceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
ZoneOffset.ofHours(-5));
+assertTrue(deviceTime.isEqual(referenceTime));
+}
+
+@Test
+public void testConfigureTimeZoneText() {
+Map parserConfig = new HashMap<>();
+parserConfig.put("deviceTimeZone", "America/New_York");
+BasicAsaParser testParser = new BasicAsaParser();
+testParser.configure(parserConfig);
+testParser.init();
+ZonedDateTime deviceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
testParser.deviceTimeZone);
+ZonedDateTime referenceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
ZoneOffset.ofHours(-5));
+assertTrue(deviceTime.isEqual(referenceTime));
+}
+
+@Test
+public void testCISCOFW106023() {
+String rawMessage = "<164>Aug 05 2016 01:01:34: %ASA-4-106023: 
Deny tcp src Inside:10.30.9.121/54580 dst Outside:192.168.135.51/42028 by 
access-group \"Inside_access_in\" [0x962df600, 0x0]";
+JSONObject asaJson = asaParser.parse(rawMessage.getBytes()).get(0);
+assertEquals(asaJson.get("original_string"), rawMessage);
+assertTrue(asaJson.get("ip_src_addr").equals("10.30.9.121"));
+assertTrue(asaJson.get("ip_dst_addr").equals("192.168.135.51"));
+assertTrue(asaJson.get("ip_src_port").equals(new Integer(54580)));
+assertTrue(asaJson.get("ip_dst_port").equals(new Integer(42028)));
+assertTrue((long) asaJson.get("timestamp") == 1470358894000L);
+}
+
+@Test
+public void testCISCOFW106006() {
+String rawMessage = "<162>Aug 05 2016 01:02:25: %ASA-2-106006: 
Deny inbound UDP from 10.25.177.164/63279 to 10.2.52.71/161 on interface 
Inside";
+JSONObject asaJson = asaParser.parse(rawMessage.getBytes()).get(0);
+assertEquals(asaJson.get("original_string"), 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569020#comment-15569020
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Thanks for bearing with me. I really appreciate the feedback and direction. 
I should be able to get these changes in later tonight after I finish up my 
"day job" :).


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568771#comment-15568771
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83000386
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,125 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+
+import java.time.ZoneId;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+import java.time.temporal.TemporalAccessor;
+import java.util.regex.Pattern;
+
+import static java.time.temporal.ChronoField.*;
+
+public class SyslogUtils {
+
+public static long parseTimestampToEpochMillis(String logTimestamp, 
ZoneId timeZone) throws ParseException {
+// RFC3164 (standard syslog timestamp; no year)
+// MMM ppd HH:mm:ss
+// Oct  9 2015 13:42:11
+if 
(Pattern.matches("[A-Z][a-z]{2}(?:(?:\\s{2}\\d)|(?:\\s\\d{2}))\\s\\d{2}:\\d{2}:\\d{2}",
 logTimestamp)) {
+DateTimeFormatter inputFormat = 
DateTimeFormatter.ofPattern("MMM ppd HH:mm:ss").withZone(timeZone);
+
+TemporalAccessor inputDate = inputFormat.parse(logTimestamp);
+int inputMonth = inputDate.get(MONTH_OF_YEAR);
+int inputDay = inputDate.get(DAY_OF_MONTH);
+int inputHour = inputDate.get(HOUR_OF_DAY);
+int inputMinute = inputDate.get(MINUTE_OF_HOUR);
+int inputSecond = inputDate.get(SECOND_OF_MINUTE);
+
+ZonedDateTime currentDate = ZonedDateTime.now(timeZone);
+int normalizedYear = currentDate.getYear();
+
+/**
+ * Since no year is provided, one must be derived.
+ *   During the month of January (first 31 days of the year), 
assume logs coming in from
+ *   November (11) and December (12) are from the previous 
year.
+ */
+if (currentDate.getDayOfYear() <= 31 && inputMonth >= 11)
+normalizedYear--;
+ZonedDateTime normalizedTimestamp = 
ZonedDateTime.of(normalizedYear, inputMonth, inputDay, inputHour, inputMinute, 
inputSecond, 0, timeZone);
+return normalizedTimestamp.toInstant().toEpochMilli();
+}
+
+// CISCO timestamp (standard syslog + year)
+// MMM dd  HH:mm:ss
+// Oct 09 2015 13:42:11
+else if 
(Pattern.matches("[A-Z][a-z]{2}\\s\\d{2}\\s\\d{4}\\s\\d{2}:\\d{2}:\\d{2}", 
logTimestamp))
+return convertToEpochMillis(logTimestamp, 
DateTimeFormatter.ofPattern("MMM dd  HH:mm:ss").withZone(timeZone));
+
+// RFC5424 (ISO timestamp)
+// 2015-10-09T13:42:11.52Z or 2015-10-09T13:42:11.52-04:00
+else if 
(Pattern.matches("\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?(?:Z|[+-]\\d{2}:\\d{2})",
 logTimestamp))
+return convertToEpochMillis(logTimestamp, 
DateTimeFormatter.ISO_OFFSET_DATE_TIME);
+
+else
+throw new ParseException(String.format("Unsupported date 
format: '%s'", logTimestamp));
--- End diff --

Just curious, any reason we're using a checked exception here?  In other 
places we're just using run time exceptions.  The ParseException that you 
created is used only for this, I believe. 

Not a big deal either way.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568769#comment-15568769
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83001278
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
 ---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
+
+@Test
+public void testRfc3164Timestamp() {
+String originalTimestamp = "Oct  9 13:42:11";
+assertEquals(getParsedEpochMillis(originalTimestamp), 
1476020531000L);
+}
+
+@Test
+public void testCiscoTimestamp() {
+String originalTimestamp = "Oct 09 2015 13:42:11";
+assertEquals(getParsedEpochMillis(originalTimestamp), 
1444398131000L);
+}
+
+@Test
+public void testRfc5424TimestampUTC() {
+String originalTimestamp = "2015-10-09T13:42:11.52Z";
+assertEquals(getParsedEpochMillis(originalTimestamp), 
1444398131520L);
+}
+
+@Test
+public void testRfc5424TimestampWithOffset() {
+String originalTimestamp = "2015-10-09T08:42:11.52-05:00";
+assertEquals(getParsedEpochMillis(originalTimestamp), 
1444398131520L);
+}
+
+private long getParsedEpochMillis(String originalTimestamp) {
+try {
--- End diff --

There is no need to try/catch here.  Just have the method `throw 
ParseException`.  Any of your test methods can also `throw ParseException`.  
This simplifies the logic and JUnit will fail the test if a ParseException is 
thrown.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568775#comment-15568775
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83000705
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/asa/BasicAsaParserTest.java
 ---
@@ -0,0 +1,149 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import org.json.simple.JSONObject;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.time.*;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.Assert.*;
+
+public class BasicAsaParserTest {
+
+private static BasicAsaParser asaParser;
+
+@BeforeClass
+public static void setUpOnce() throws Exception {
+Map parserConfig = new HashMap<>();
+asaParser = new BasicAsaParser();
+asaParser.configure(parserConfig);
+asaParser.init();
+}
+
+@Test
+public void testConfigureDefault() {
+Map parserConfig = new HashMap<>();
+BasicAsaParser testParser = new BasicAsaParser();
+testParser.configure(parserConfig);
+testParser.init();
+assertTrue(testParser.deviceTimeZone.equals(ZoneOffset.UTC));
+}
+
+@Test
+public void testConfigureTimeZoneOffset() {
+Map parserConfig = new HashMap<>();
+parserConfig.put("deviceTimeZone", "UTC-05:00");
+BasicAsaParser testParser = new BasicAsaParser();
+testParser.configure(parserConfig);
+testParser.init();
+ZonedDateTime deviceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
testParser.deviceTimeZone);
+ZonedDateTime referenceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
ZoneOffset.ofHours(-5));
+assertTrue(deviceTime.isEqual(referenceTime));
+}
+
+@Test
+public void testConfigureTimeZoneText() {
+Map parserConfig = new HashMap<>();
+parserConfig.put("deviceTimeZone", "America/New_York");
+BasicAsaParser testParser = new BasicAsaParser();
+testParser.configure(parserConfig);
+testParser.init();
+ZonedDateTime deviceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
testParser.deviceTimeZone);
+ZonedDateTime referenceTime = 
ZonedDateTime.ofInstant(Instant.ofEpochSecond(1475323200), 
ZoneOffset.ofHours(-5));
+assertTrue(deviceTime.isEqual(referenceTime));
+}
+
+@Test
+public void testCISCOFW106023() {
+String rawMessage = "<164>Aug 05 2016 01:01:34: %ASA-4-106023: 
Deny tcp src Inside:10.30.9.121/54580 dst Outside:192.168.135.51/42028 by 
access-group \"Inside_access_in\" [0x962df600, 0x0]";
+JSONObject asaJson = asaParser.parse(rawMessage.getBytes()).get(0);
+assertEquals(asaJson.get("original_string"), rawMessage);
+assertTrue(asaJson.get("ip_src_addr").equals("10.30.9.121"));
+assertTrue(asaJson.get("ip_dst_addr").equals("192.168.135.51"));
+assertTrue(asaJson.get("ip_src_port").equals(new Integer(54580)));
+assertTrue(asaJson.get("ip_dst_port").equals(new Integer(42028)));
+assertTrue((long) asaJson.get("timestamp") == 1470358894000L);
+}
+
+@Test
+public void testCISCOFW106006() {
+String rawMessage = "<162>Aug 05 2016 01:02:25: %ASA-2-106006: 
Deny inbound UDP from 10.25.177.164/63279 to 10.2.52.71/161 on interface 
Inside";
+JSONObject asaJson = asaParser.parse(rawMessage.getBytes()).get(0);
+assertEquals(asaJson.get("original_string"), 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568773#comment-15568773
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83004884
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,209 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.ParseException;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.time.ZoneId;
+import java.time.ZoneOffset;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+protected ZoneId deviceTimeZone;
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", "CISCOFW305012")
+   .put("ASA-7-609001", "CISCOFW609001")
+  

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568774#comment-15568774
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83005558
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,209 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.ParseException;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.time.ZoneId;
+import java.time.ZoneOffset;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+protected ZoneId deviceTimeZone;
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", "CISCOFW305012")
+   .put("ASA-7-609001", "CISCOFW609001")
+  

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568770#comment-15568770
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83001038
  
--- Diff: 
metron-platform/metron-parsers/src/test/java/org/apache/metron/parsers/utils/SyslogUtilsTest.java
 ---
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import org.apache.metron.parsers.ParseException;
+import org.junit.Test;
+
+import java.time.ZoneOffset;
+
+import static org.junit.Assert.*;
+
+public class SyslogUtilsTest {
--- End diff --

A tricky, but necessary, part of your timestamp logic is rolling the year 
backwards in certain cases.  Are there specific tests that hit on that?  Maybe 
I am missing them.  We need to make sure we cover all the edges on those 
scenarios.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568662#comment-15568662
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r83000270
  
--- Diff: 
metron-platform/metron-parsers/src/main/config/zookeeper/parsers/asa.json ---
@@ -0,0 +1,7 @@
+{
+  "parserClassName": "org.apache.metron.parsers.asa.BasicAsaParser",
+  "sensorTopic": "asa",
+  "parserConfig": {
+"deviceTimeZone": "UTC-05:00"
--- End diff --

Yes, absolutely. I'll remove it. I left this in as an example. If no 
`deviceTimeZone` is provided, the code will default to UTC.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568622#comment-15568622
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
@kylerichardson When you say "tested in single node vm", what do you mean 
exactly?  Do you not use the Vagrant deployment mechanism at 
`metron-deployment/vagrant/quick-dev-platform` or 
`metron-deployment/vagrant/full-dev-platform` to create a single node VM for 
testing?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568587#comment-15568587
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Thanks. Looks like re-opening did the trick.

I've done my best to incorporate everyone's feedback into this version. 
Re-tested in single node vm successfully.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567065#comment-15567065
 ] 

ASF GitHub Bot commented on METRON-363:
---

GitHub user kylerichardson reopened a pull request:

https://github.com/apache/incubator-metron/pull/276

METRON-363 Fix Cisco ASA Parser

I've rewritten the ASA parser which can be extended, as needed, to new ASA 
message types by editing the bundled asa patterns file and the static map used 
for grok patterns in the code. I've also tried to make it easier to deploy the 
asa topology by including zookeeper config files and creating the kafka topic 
during metron install. Sample data is also included for integration testing.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kylerichardson/incubator-metron METRON-363

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/276.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #276


commit 5be7c60448f73fcc72c81451a67ef1e40fd29793
Author: kylerichardson 
Date:   2016-08-16T01:12:42Z

Initial rewrite of Cisco ASA parser

Summary of changes:
- Complete rewrite of ASA parser including new test suite
- ZK configurations for ease of topology deployment (parser and enrichment)
- Add field constant for original_string in metron-common
- Minor changes to ASA patterns file for
  (1) Syslog severity/facility capture
  (2) Interface capture on CISCOFW106006_106007_106010
- Updates to various POMs to allow easier validation of logging during unit 
testing
  (1) Exclusions for slf4j-log4j12 on various dependencies for 
metron-parsers and metron-integration-test
  (2) Explicit dependency on slf4j-api for metron-parsers
  (3) Test dependency on slf4j-simple for metron-parsers

commit c87e6edaf0e308be9f417e07016508f87067ae0c
Author: kylerichardson 
Date:   2016-09-20T02:33:09Z

METRON-363 Reworked parser to handle nulls and field validation

Includes the following:
- Static map for ASA message patterns (vs pattern discovery)
- Minor changes to ASA patterns file
- Broke out common syslog parsing elements
- Broke out reusable field validations

commit a8c4903dd0bcac18e15c98aca7264dce1c455bee
Author: kylerichardson 
Date:   2016-09-27T00:30:16Z

METRON-363 Add integration test and sample data

Includes the following:
- Extend BasicParser
- Handle both types of syslog timestamps (with and without year)
- Include integration test and supporting sample data

commit 011d389bdf43f1790384dbcd13ec7da148c53ef2
Author: kylerichardson 
Date:   2016-09-27T00:40:51Z

METRON-363 Add license and kafka topic

commit 04a936d75cf782254105993b2804912b4659257a
Author: kylerichardson 
Date:   2016-09-28T00:29:21Z

METRON-363 Adjust log level

commit abd7fb92fe4c38530e10141d0aba6bd07a335ae8
Author: kylerichardson 
Date:   2016-10-08T01:11:22Z

METRON-363 Enhance logging, remove unused code

commit a885ecc762a8d5296d7c7ebfe7600c910ce3478b
Author: kylerichardson 
Date:   2016-10-11T17:40:25Z

METRON-363 Refactored and enhanced based on feedback

Changes include:
(1) New/additional unit tests
(2) Reworked Syslog Timestamp (no year) logic
(3) Enhanced error checking and logging (introduced new ParseException)




> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567002#comment-15567002
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson closed the pull request at:

https://github.com/apache/incubator-metron/pull/276


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566633#comment-15566633
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
I would close and re-open. As our test suite has expanded and is more 
demanding, at certain times Travis will fail the build when there is not really 
a problem.  We need to figure out how to fix this problem, but right now I'd 
try a reboot.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566368#comment-15566368
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Not entirely sure why the CI build failed.

The error was:
```

testExample1(org.apache.metron.profiler.integration.ProfilerIntegrationTest)  
Time elapsed: 35.546 sec  <<< FAILURE!
java.lang.AssertionError: expected:<1950.0> but was:<390.0>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:553)
at org.junit.Assert.assertEquals(Assert.java:683)
at 
org.apache.metron.profiler.integration.ProfilerIntegrationTest.testExample1(ProfilerIntegrationTest.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at 
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at 
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
```

Slightly earlier in the log:
```
106738 [Curator-Framework-0] ERROR o.a.c.ConnectionState - Connection timed 
out for connection string (127.0.0.1:51857) and timeout (15000) / elapsed 
(18872)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = 
ConnectionLoss
at 
org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:197) 
[metron-common-0.2.1BETA.jar:?]
at 
org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87) 
[metron-common-0.2.1BETA.jar:?]
at 
org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
 [metron-common-0.2.1BETA.jar:?]
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:806)
 [metron-common-0.2.1BETA.jar:?]
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:792)
 [metron-common-0.2.1BETA.jar:?]
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:62)
 [metron-common-0.2.1BETA.jar:?]
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:257)
 [metron-common-0.2.1BETA.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_31]
at 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15558015#comment-15558015
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r82502874
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
+int year = now.getYear();
+if (now.getDayOfYear() == 1 && 
!now.getMonth().toString().substring(0,3).equals(logTimestamp.substring(0,3).toUpperCase()))
+year--;
--- End diff --

Gotcha.  So anything that comes in on the first day of the year, with a 
month that is not January, will be backdated.

If something comes in on the 2nd day of the year, with a month of December, 
it will NOT be backdated.  The period of time that we are willing to backdate, 
is effectively 1 day currently.  

Maybe that time period needs to be configurable.  The user defines the 
period of time, 1 day, 2 days, 1 week, after the beginning of the year in which 
messages can possibly be backdated.

Are there certain conditions under which the logic should blow-up and 
error?  What if we are going to backdate a message where the month is July?  
Should we just do that or should we error?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557992#comment-15557992
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r82502674
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
--- End diff --

As part of a future enhancement, maybe we can allow the user to define a 
map as part of the configuration.  This maps the value of some indicator field 
to a timezone.  For example, based on something like `%ASA-6-302013` the parser 
will choose the appropriate input timezone.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557986#comment-15557986
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r82502633
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556771#comment-15556771
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r82490277
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
+int year = now.getYear();
+if (now.getDayOfYear() == 1 && 
!now.getMonth().toString().substring(0,3).equals(logTimestamp.substring(0,3).toUpperCase()))
+year--;
--- End diff --

Sure. I see how this is not entirely obvious. I'm trying to solve an edge 
case here where a message comes in for parsing without a year in the timestamp 
on January 1st but the message was actually generated on the device on December 
31st. I'll add in some comments for clarity.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556748#comment-15556748
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r82489921
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
--- End diff --

Of course you're right, the timestamp will not always be in UTC. ASA logs 
consumed via syslog (either raw off the wire or through another syslog server) 
will generally follow the syslog standard.

There are a number of possibilities to explore here. If we assume that we 
will be collecting the raw syslog from the ASAs off the wire, the timestamp 
will not include the timezone/offset. This code assumes the device is logging 
in UTC, which, to your point, is probably a bad assumption. I made this 
assumption because it seems to me we would want all of the timestamps indexed 
to be in the same timezone and the easiest way to accomplish that would be to 
normalize all of the telemetry data to UTC.

Question for the team. How are other parsers handling timezone? Are they 
passing through the device timezone?

The way I'm thinking of solving this is by adding a configuration option to 
the parser to specify the device timezone. (This would require that all ASAs 
put through the parser we configured to the same timezone though.) I would then 
convert the timestamp to UTC prior to writing it into the metron normalized 
JSON message.

Any feedback or other ideas on solving this one?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15538856#comment-15538856
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r81454500
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15538855#comment-15538855
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r81454486
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15538812#comment-15538812
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r81453779
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15538782#comment-15538782
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r81453295
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/FieldValidators.java
 ---
@@ -0,0 +1,41 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class FieldValidators {
+
+public static boolean isValidPort(int portNumber) {
+if (portNumber > 1 && portNumber < 65536)
+return true;
+else
+return false;
+}
+
+public static boolean isValidIpAddr(String ipAddress) {
--- End diff --

No my `isValidIpAddr` function is probably more rudimentary than 
`org.apache.commons.validator.routines.InetAddressValidator`. I actually 
switched to using the `InetAddressValidator` in the latest version of the code, 
so this is just an unused function that I will remove.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529690#comment-15529690
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80920348
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529659#comment-15529659
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80917788
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/FieldValidators.java
 ---
@@ -0,0 +1,41 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class FieldValidators {
+
+public static boolean isValidPort(int portNumber) {
+if (portNumber > 1 && portNumber < 65536)
+return true;
+else
+return false;
+}
+
+public static boolean isValidIpAddr(String ipAddress) {
--- End diff --

Yeah, agreed.  I'd just do 
`InetAddressValidator.getInstance().isValidInet6Address(ip) || 
InetAddressValidator.getInstance().isValidInet4Address(ip)` here.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529661#comment-15529661
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80917333
  
--- Diff: metron-deployment/roles/metron_kafka_topics/defaults/main.yml ---
@@ -21,6 +21,7 @@ topics_to_create:
   - { topic: "bro", num_partitions: 1, replication_factor: 1, 
retention_gb: 10 }
   - { topic: "yaf", num_partitions: 1, replication_factor: 1, 
retention_gb: 10 }
   - { topic: "snort",   num_partitions: 1, replication_factor: 1, 
retention_gb: 10 }
+  - { topic: "asa", num_partitions: 1, replication_factor: 1, 
retention_gb: 10 }
--- End diff --

should we create the kafka topic if we aren't starting the sensor as part 
of the default set of sensors?  Shouldn't we handle this like squid, where we 
have the user create the topic if they set up the sensor?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529660#comment-15529660
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user cestella commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80917150
  
--- Diff: 
metron-platform/metron-common/src/main/java/org/apache/metron/common/Constants.java
 ---
@@ -43,6 +43,7 @@
 ,DST_PORT("ip_dst_port")
 ,PROTOCOL("protocol")
 ,TIMESTAMP("timestamp")
+,ORIGINAL("original_string")
--- End diff --

I particularly like this one.  We should refactor the GrokParser to use it 
as a follow-on.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529616#comment-15529616
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80909564
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529621#comment-15529621
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80915380
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
+int year = now.getYear();
+if (now.getDayOfYear() == 1 && 
!now.getMonth().toString().substring(0,3).equals(logTimestamp.substring(0,3).toUpperCase()))
+year--;
--- End diff --

This logic stands out to me as not totally obvious.  Would be good to 
comment why you are doing this.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529619#comment-15529619
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80908563
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/FieldValidators.java
 ---
@@ -0,0 +1,41 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class FieldValidators {
+
+public static boolean isValidPort(int portNumber) {
+if (portNumber > 1 && portNumber < 65536)
+return true;
+else
+return false;
+}
+
+public static boolean isValidIpAddr(String ipAddress) {
--- End diff --

Does your `isValidIpAddr` function do something different than 
`org.apache.commons.validator.routines.InetAddressValidator`?


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529615#comment-15529615
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80906283
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/asa/BasicAsaParser.java
 ---
@@ -0,0 +1,165 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.asa;
+
+import com.google.common.collect.ImmutableMap;
+import oi.thekraken.grok.api.Grok;
+import oi.thekraken.grok.api.Match;
+import oi.thekraken.grok.api.exception.GrokException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+import org.apache.metron.common.Constants;
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.utils.FieldValidators;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.*;
+import java.util.*;
+
+public class BasicAsaParser extends BasicParser {
+
+protected static final Logger LOG = 
LoggerFactory.getLogger(BasicAsaParser.class);
+
+private Grok asaGrok;
+
+private static final InetAddressValidator ipValidator = 
InetAddressValidator.getInstance();
+
+private static final Map patternMap = 
ImmutableMap.builder()
+.put("ASA-2-106001", "CISCOFW106001")
+   .put("ASA-2-106006", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106007", "CISCOFW106006_106007_106010")
+   .put("ASA-2-106010", "CISCOFW106006_106007_106010")
+   .put("ASA-3-106014", "CISCOFW106014")
+   .put("ASA-6-106015", "CISCOFW106015")
+   .put("ASA-1-106021", "CISCOFW106021")
+   .put("ASA-4-106023", "CISCOFW106023")
+   .put("ASA-5-106100", "CISCOFW106100")
+   .put("ASA-6-110002", "CISCOFW110002")
+   .put("ASA-6-302010", "CISCOFW302010")
+   .put("ASA-6-302013", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302014", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302015", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302016", "CISCOFW302013_302014_302015_302016")
+   .put("ASA-6-302020", "CISCOFW302020_302021")
+   .put("ASA-6-302021", "CISCOFW302020_302021")
+   .put("ASA-6-305011", "CISCOFW305011")
+   .put("ASA-3-313001", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313004", "CISCOFW313001_313004_313008")
+   .put("ASA-3-313008", "CISCOFW313001_313004_313008")
+   .put("ASA-4-313005", "CISCOFW313005")
+   .put("ASA-4-402117", "CISCOFW402117")
+   .put("ASA-4-402119", "CISCOFW402119")
+   .put("ASA-4-419001", "CISCOFW419001")
+   .put("ASA-4-419002", "CISCOFW419002")
+   .put("ASA-4-54", "CISCOFW54")
+   .put("ASA-6-602303", "CISCOFW602303_602304")
+   .put("ASA-6-602304", "CISCOFW602303_602304")
+   .put("ASA-7-710001", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710002", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710003", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710005", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-7-710006", 
"CISCOFW710001_710002_710003_710005_710006")
+   .put("ASA-6-713172", "CISCOFW713172")
+   .put("ASA-4-733100", "CISCOFW733100")
+   .put("ASA-6-305012", 

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529618#comment-15529618
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80914488
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
--- End diff --

Do you have unit tests for these methods?  Would be good to add 
specifically for the `SyslogUtils` methods.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529622#comment-15529622
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80905725
  
--- Diff: 
metron-platform/metron-integration-test/src/main/sample/data/asa/raw/asa_raw ---
@@ -0,0 +1,128 @@
+<167>Jan  5 08:52:35 10.22.8.216 %ASA-7-609001: Built local-host 
inside:10.22.8.205
+<166>Jan  5 08:52:35 10.22.8.216 %ASA-6-302021: Teardown ICMP connection 
for faddr 10.22.8.74/0(LOCAL\user.name) gaddr 10.22.8.205/0 laddr 10.22.8.205/0
+<167>Jan  5 08:52:35 10.22.8.216 %ASA-7-609002: Teardown local-host 
inside:10.22.8.205 duration 0:00:00
+<142>Jan  5 08:52:35 10.22.8.201 %ASA-6-302014: Teardown TCP connection 
488167725 for Outside_VPN:147.111.72.16/26436 to DMZ-Inside:10.22.8.53/443 
duration 0:00:00 bytes 9687 TCP FINs
+<166>Jan  5 08:52:35 10.22.8.216 %ASA-6-302014: Teardown TCP connection 
212805593 for outside:10.22.8.223/59614(LOCAL\user.name) to 
inside:10.22.8.78/8102 duration 0:00:07 bytes 3433 TCP FINs (user.name)
+<174>Jan  5 14:52:35 10.22.8.212 %ASA-6-302013: Built inbound TCP 
connection 76245503 for outside:10.22.8.233/54209 (10.22.8.233/54209) to 
inside:198.111.72.238/443 (198.111.72.238/443) (user.name)
+<166>Jan  5 08:52:35 10.22.8.216 %ASA-6-302013: Built inbound TCP 
connection 212806031 for outside:10.22.8.17/58633 
(10.22.8.17/58633)(LOCAL\user.name) to inside:10.22.8.12/389 (10.22.8.12/389) 
(user.name)
+<142>Jan  5 08:52:35 10.22.8.201 %ASA-6-302014: Teardown TCP connection 
488168292 for DMZ-Inside:10.22.8.51/51231 to Inside-Trunk:10.22.8.174/40004 
duration 0:00:00 bytes 2103 TCP FINs
--- End diff --

Has this data been scrubbed? I just want to make sure that none of it is 
proprietary.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529620#comment-15529620
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/276#discussion_r80915159
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/SyslogUtils.java
 ---
@@ -0,0 +1,89 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.parsers.utils;
+
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+
+public class SyslogUtils {
+
+public static long convertToEpochMillis(String logTimestamp, String 
logTimeFormat) {
+ZonedDateTime timestamp = ZonedDateTime.parse(logTimestamp, 
DateTimeFormatter.ofPattern(logTimeFormat).withZone(ZoneOffset.UTC));
+return timestamp.toEpochSecond() * 1000;
+}
+
+public static long parseTimestampToEpochMillis(String logTimestamp) {
+if (logTimestamp.length() < 20) {
+ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
--- End diff --

Is a Syslog timestamp always UTC?  More importantly do ASAs follow the 
Syslog standard, if so?  ;)


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528108#comment-15528108
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
Currently my branch doesn't have build_utils. Going to rebase and see if 
that fixes the CI build.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527106#comment-15527106
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user danieljue commented on the issue:

https://github.com/apache/incubator-metron/pull/278
  
@nickwallen , interesting because the jodatime dependency is already in 
that CSV (it was included in API, but had to add it to the Common module pom)

Seems to  be failing at the same place as METRON-363 Fix Cisco ASA Parser :

`Sep 27, 2016 1:28:55 AM 
com.google.inject.servlet.InternalServletModule$BackwardsCompatibleServletContextProvider
 get
WARNING: You are attempting to use a deprecated API (specifically, 
attempting to @Inject ServletContext inside an eagerly created singleton. While 
we allow this for backwards compatibility, be warned that this MAY have 
unexpected behavior if you have more than one injector (with ServletModule) 
running in the same JVM. Please consult the Guice documentation at 
http://code.google.com/p/google-guice/wiki/Servlets for more information.
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider 
as a provider class
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering 
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices 
as a root resource class
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering 
org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices as a root 
resource class
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as 
a provider class
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 
AM'
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to 
GuiceManagedComponentProvider with the scope "Singleton"
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to 
GuiceManagedComponentProvider with the scope "Singleton"
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding 
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices 
to GuiceManagedComponentProvider with the scope "Singleton"
Sep 27, 2016 1:28:55 AM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding 
org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to 
GuiceManagedComponentProvider with the scope "Singleton"
2016-09-27 01:28:59,179 ERROR [Curator-TreeCache-0] curator.ConnectionState 
(ConnectionState.java:checkTimeouts(200)) - Connection timed out for connection 
string (127.0.0.1:48078) and timeout (15000) / elapsed (31009)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = 
ConnectionLoss
at 
org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:197)
at 
org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87)
at 
org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
at 
org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
at 
org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
at 
org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
at 
org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:200)
at 
org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
at 
org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
at 
org.apache.curator.x.discovery.details.ServiceDiscoveryImpl.queryForNames(ServiceDiscoveryImpl.java:276)
at 
org.apache.metron.maas.discovery.ServiceDiscoverer.updateState(ServiceDiscoverer.java:129)
at 
org.apache.metron.maas.discovery.ServiceDiscoverer.lambda$new$2(ServiceDiscoverer.java:93)
at 
org.apache.metron.maas.discovery.ServiceDiscoverer$$Lambda$34/648409124.childEvent(Unknown
 Source)
  

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527117#comment-15527117
 ] 

ASF GitHub Bot commented on METRON-363:
---

Github user danieljue commented on the issue:

https://github.com/apache/incubator-metron/pull/276
  
FYI the PR for METRON-451 is failing at the same place.


> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (METRON-363) Fix Cisco ASA Parser

[Kyle Richardson (JIRA)]

[ 
https://issues.apache.org/jira/browse/METRON-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417869#comment-15417869
 ] 

Kyle Richardson commented on METRON-363:


I have this mostly complete. I just need to do some additional testing and then 
submit the PR for review.

> Fix Cisco ASA Parser
> 
>
> Key: METRON-363
> URL: https://issues.apache.org/jira/browse/METRON-363
> Project: Metron
>  Issue Type: Improvement
>Reporter: Kyle Richardson
>Priority: Minor
>
> The current ASA parser is broken. This effort is to rework the current parser 
> to support the variety of syslog messages produced by Cisco ASA devices as 
> well as provide the necessary support files/configs for easier deployment of 
> the Storm topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)