date:20170211

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862628#comment-15862628
 ] 

ASF GitHub Bot commented on METRON-157:
---

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/451#discussion_r100682204
  
--- Diff: 
metron-platform/metron-parsers/src/test/resources/org/apache/metron/parsers/cef/adallom.cef
 ---
@@ -0,0 +1 @@
+2016-04-01T09:29:11.356-0400 
CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
 policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
suser=wander...@rock.com start=1459517280810 end=1459517280810 
audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
 services=["APPID_SXC"] users=["lva...@hotmail.com"] 
cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
cs6Label=consoleUrl
--- End diff --

May be a good idea to use ASF owned domains on emails as other domains may 
be under trademark and/or copyright 


> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
>

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862629#comment-15862629
 ] 

ASF GitHub Bot commented on METRON-157:
---

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/451#discussion_r100680678
  
--- Diff: 
metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/cef/CEFParser.java
 ---
@@ -0,0 +1,272 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.parsers.cef;
+
+import java.nio.charset.Charset;
+import java.time.Clock;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.metron.parsers.BasicParser;
+import org.apache.metron.parsers.ParseException;
+import org.apache.metron.parsers.utils.DateUtils;
+import org.apache.metron.parsers.utils.SyslogUtils;
+import org.json.simple.JSONObject;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class CEFParser extends BasicParser {
+   private static final long serialVersionUID = 1L;
+   
+   protected static final Logger LOG = 
LoggerFactory.getLogger(CEFParser.class);
+   private static final String HEADER_CAPTURE_PATTERN = "[^\\|]*";
+   private static final String EXTENSION_CAPTURE_PATTERN = "(?";
+   String syslogHost = "[a-z0-9\\.-_]+";
+
+   StringBuilder sb = new StringBuilder("(?");
+
+   sb.append(syslogTime);
+   sb.append(")?");
+
+   sb.append("(?");
+   sb.append(syslogHost);
+   sb.append(")?");
+
+   sb.append("(?");
+   sb.append(syslogPriority);
+   sb.append(")?");
+
+   sb.append(".*");
+
+   sb.append("CEF:0\\|");
+
+   headerBlock("DeviceVendor", sb);
+   sb.append("\\|");
+   headerBlock("DeviceProduct", sb);
+   sb.append("\\|");
+   headerBlock("DeviceVersion", sb);
+   sb.append("\\|");
+   headerBlock("DeviceEvent", sb);
+   sb.append("\\|");
+   headerBlock("Name", sb);
+   sb.append("\\|");
+   headerBlock("Severity", sb);
+   sb.append("\\|");
+
+   // extension capture:
+   sb.append("(?.*)");
+   String pattern = sb.toString();
+
+   p = Pattern.compile(pattern);
+
+   // key finder for extensions
+   pext = Pattern.compile(EXTENSION_CAPTURE_PATTERN);
+   }
+
+   @SuppressWarnings("unchecked")
+   public List parse(byte[] rawMessage) {
+   List messages = new ArrayList<>();
+
+   String cefString = new String(rawMessage, UTF_8);
+
+   Matcher matcher = p.matcher(cefString);
+
+   while (matcher.find()) {
+   JSONObject obj = new JSONObject();
+   if (matcher.matches()) {
+   LOG.info(String.format("Found %d groups", 
matcher.groupCount()));
+   obj.put("DeviceVendor", 
matcher.group("DeviceVendor"));
+   obj.put("DeviceProduct", 
matcher.group("DeviceProduct"));
+   obj.put("DeviceVersion", 
matcher.group("DeviceVersion"));
+   obj.put("DeviceEvent", 
matcher.group("DeviceEvent"));
+   obj.put("Name", matcher.group("Name"));
+   obj.put("Severity", 
standardizeSeverity(matcher.group("Severity")));
+   }
+
+   String ext = matcher.group("extensions");
+   Matcher m = pext.matcher(ext);
+
+   int

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862630#comment-15862630
 ] 

ASF GitHub Bot commented on METRON-157:
---

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/451#discussion_r100682233
  
--- Diff: 
metron-platform/metron-parsers/src/test/resources/org/apache/metron/parsers/cef/cyberark.json
 ---
@@ -0,0 +1,21 @@
+{
+"\"Other info\"": "101.198.70.93",
+"\"Safe Name\"": "Security Vulnerability Mgmt",
+"\"Ticket Id\"": "Needed to verify config files being pulled ",
+"deviceAction": "Retrieve password",
+"deviceAddress": "120.99.70.3",
--- End diff --

I suspect metron will take care of the IP parsing but it may be a good idea 
to add ipv6 to the test units?


> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
> pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 
> CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password 
> suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 
> shost=10.44.134.78 dhost= duser= externalId= app= reason=

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862591#comment-15862591
 ] 

ASF GitHub Bot commented on METRON-157:
---

Github user trixpan commented on the issue:

https://github.com/apache/incubator-metron/pull/451
  
@simonellistonball seems like the code will hit the same issue we hit in 
NIFI-3466? The CEF spec doesn mention but it seems like ArcSight behavior is to 
be able to MMM even when MMM is written with localised values for "Short Month 
Name"


> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
> pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 
> CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password 
> suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 
> shost=10.44.134.78 dhost= duser= externalId= app= reason= cs1Label=\"Affected 
> User Name\" cs1= cs2Label=\"Safe Name\" cs2=Security Vulnerability Mgmt 
> cs3Label=\"Device Type\" cs3= cs4Label=\"Database\" cs4= cs5Label=\"Other 
> info\" cs5=101.198.70.93 cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" 
> cn2=Needed to verify config files being pulled  msg=Needed to verify config 
> files

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862587#comment-15862587
 ] 

ASF GitHub Bot commented on METRON-157:
---

Github user simonellistonball commented on the issue:

https://github.com/apache/incubator-metron/pull/451
  
@kylerichardson no problem at all, would really appreciate it if you could 
review, and add anything from any work you have on this. 


> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
> pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 
> CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password 
> suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 
> shost=10.44.134.78 dhost= duser= externalId= app= reason= cs1Label=\"Affected 
> User Name\" cs1= cs2Label=\"Safe Name\" cs2=Security Vulnerability Mgmt 
> cs3Label=\"Device Type\" cs3= cs4Label=\"Database\" cs4= cs5Label=\"Other 
> info\" cs5=101.198.70.93 cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" 
> cn2=Needed to verify config files being pulled  msg=Needed to verify config 
> files being pulled","\"Safe Name\"":"Security Vulnerability 
> Mgmt","header":"Mar 21 14:05:02

[jira] [Created] (METRON-714) Migrate documentation into git

2017-02-11 Thread Jon Zeolla (JIRA)

Jon Zeolla created METRON-714:
-

 Summary: Migrate documentation into git
 Key: METRON-714
 URL: https://issues.apache.org/jira/browse/METRON-714
 Project: Metron
  Issue Type: Improvement
Affects Versions: 0.3.1
Reporter: Jon Zeolla


This morning I had an opportunity to watch the video from yesterday's community 
demo, and there was some really good discussion towards the end about 
documentation of examples that I wanted to follow up with.  For future 
reference, here is the recording 
of what I'm referring to - this is all as a follow-up to Matt's great work via 
METRON-660.

I am looking for feedback on an idea for the future of Metron documentation.  
At a high level, I would like to migrate materials from the wiki pages 
throughout the git repo and modify our documentation generation scripts to key 
in on tutorials vs readmes.  Once we have agreement on this I would be happy to 
handle any data migration and manipulation as necessary.

More specifically, I would like to establish a convention for the names of 
example or tutorial md files that we could then use when generating the release 
documentation.  Say we use "examples.md", we could then generate an 
examples/tutorials top level area in the site-docs without having to add it 
into the git repo itself.  In addition, this lets the examples.md files exist 
more closely to the code they are about, which seems to be the preference of 
most people currently working on the project.

A good example of this would be to break Casey's outlier analysis example into 
a new examples.md in the same directory.  I would think more generalized 
examples/tutorials would exist in the root of the git repo.  I'm also game for 
arguments that we take another approach, such as making a new top level folder 
in the repo for all examples/tutorials, but that would be less preferred in my 
opinion.

We could probably move the overview, architecture, tutorials, and governance 
wiki materials without much of an issue.  Pages like the tech talks and 
community information probably fit better in the Metron site area of GitHub, 
and not as a md.  The items that I wouldn't be sure about migrating are things 
like the user research or meeting notes.  Is there still value in having these 
materials published?  Maybe we leave them behind in the Wiki and use it as more 
of an archive store for historical context?

If I don't get any strong disagreement with this idea, I'm going to throw 
together a first attempt.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862497#comment-15862497
 ] 

ASF GitHub Bot commented on METRON-157:
---

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/451
  
@simonellistonball, thanks for picking this one up! I have unassigned the 
JIRA from myself as I've clearly not had the time to work on it recently.


> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
> pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 
> CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password 
> suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 
> shost=10.44.134.78 dhost= duser= externalId= app= reason= cs1Label=\"Affected 
> User Name\" cs1= cs2Label=\"Safe Name\" cs2=Security Vulnerability Mgmt 
> cs3Label=\"Device Type\" cs3= cs4Label=\"Database\" cs4= cs5Label=\"Other 
> info\" cs5=101.198.70.93 cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" 
> cn2=Needed to verify config files being pulled  msg=Needed to verify config 
> files being pulled","\"Safe Name\"":"Security Vulnerability 
> Mgmt","header":"Mar 21

[jira] [Commented] (METRON-157) Create CEF Parser

2017-02-11 Thread Kyle Richardson (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862494#comment-15862494
 ] 

Kyle Richardson commented on METRON-157:


Apologies, I have clearly not had the cycles to work on this lately. A big 
thank you to Simon for picking up the slack.

> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
> pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 
> CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password 
> suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 
> shost=10.44.134.78 dhost= duser= externalId= app= reason= cs1Label=\"Affected 
> User Name\" cs1= cs2Label=\"Safe Name\" cs2=Security Vulnerability Mgmt 
> cs3Label=\"Device Type\" cs3= cs4Label=\"Database\" cs4= cs5Label=\"Other 
> info\" cs5=101.198.70.93 cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" 
> cn2=Needed to verify config files being pulled  msg=Needed to verify config 
> files being pulled","\"Safe Name\"":"Security Vulnerability 
> Mgmt","header":"Mar 21 14:05:02 HHHPVATN1 CEF:0","event_name":"Retrieve 
> password","device_vendor":"Cyber-Ark","src_hostname":"10.44.134.78"}
> WAF CEF
>

[jira] [Assigned] (METRON-157) Create CEF Parser

2017-02-11 Thread Kyle Richardson (JIRA)


 [ 
https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kyle Richardson reassigned METRON-157:
--

Assignee: (was: Kyle Richardson)

> Create CEF Parser
> -
>
> Key: METRON-157
> URL: https://issues.apache.org/jira/browse/METRON-157
> Project: Metron
>  Issue Type: New Feature
>Reporter: Domenic Puzio
>Priority: Minor
>  Labels: platform
>
> Create a parser for CEF (Common Event Format). CEF is a very common 
> formatting for security data sources; it is used by FireEye, Adallom, Imperva 
> WAF, CyberArk, and others. The parser should be flexible enough to work for 
> any of these data sources. CEF uses shorthand field names, so field names 
> should be changed to human-readable and Metron-friendly equivalents. CEF 
> custom labels (cs1Label, flexString1Label, etc.) should be converted 
> appropriately.
> Below are sample messages and their expected parsed output.
> Adallom CEF
> 2016-04-01T09:29:11.356-0400 
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"]
>  services=["APPID_SXC"] users=["lva...@hotmail.com"] 
> cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a 
> cs6Label=consoleUrl
> ...
> {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity
>  policy 'User download\/view file' was triggered by 
> 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400
>  
> CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity
>  policy 'User download\/view file' was triggered by 'scolb...@gmail.com' 
> suser=wander...@rock.com start=1459517280810 end=1459517280810 
> audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]
>  services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] 
> cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a
>  cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 
> CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000}
> CyberArk CEF
> Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve 
> password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP 
> dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= 
> cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security 
> Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= 
> cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= 
> cn2Label="Ticket Id" cn2=Needed to verify config files being pulled  
> msg=Needed to verify config files being pulled
> ...
> {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC
>  phobos3 - COMP","src_username":"spilgrim","\"Other 
> info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being 
> pulled 
> ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve 
> password","message":"Needed to verify config files being 
> pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 
> CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password 
> suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 
> shost=10.44.134.78 dhost= duser= externalId= app= reason= cs1Label=\"Affected 
> User Name\" cs1= cs2Label=\"Safe Name\" cs2=Security Vulnerability Mgmt 
> cs3Label=\"Device Type\" cs3= cs4Label=\"Database\" cs4= cs5Label=\"Other 
> info\" cs5=101.198.70.93 cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" 
> cn2=Needed to verify config files being pulled  msg=Needed to verify config 
> files being pulled","\"Safe Name\"":"Security Vulnerability 
> Mgmt","header":"Mar 21 14:05:02 HHHPVATN1 CEF:0","event_name":"Retrieve 
> password","device_vendor":"Cyber-Ark","src_hostname":"10.44.134.78"}
> WAF CEF
> <14>CEF:0|Imperva Inc.|SecureSphere|10.0.0.4_16|ABC - Secure Login.vm Page 
> Rate Limit UK - Source IP||High|act=alert

[jira] [Commented] (METRON-342) Deployment - full-dev-platform - grok_upload - Error accessing file:/usr/metron/0.2.0BETA/patterns

2017-02-11 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/METRON-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862461#comment-15862461
 ] 

ASF GitHub Bot commented on METRON-342:
---

Github user 2xyo closed the pull request at:

https://github.com/apache/incubator-metron/pull/220


> Deployment - full-dev-platform - grok_upload - Error accessing 
> file:/usr/metron/0.2.0BETA/patterns
> --
>
> Key: METRON-342
> URL: https://issues.apache.org/jira/browse/METRON-342
> Project: Metron
>  Issue Type: Bug
> Environment: 
> yoyo@yoyo-desktop:~/incubator-metron/metron-deployment/scripts$ 
> ./platform-info.sh 
> Metron 0.2.0BETA
> --
> * master
> --
> commit 1182d1a87ce6a7b19b23a2d93bb72819835da102
> Author: justinleet 
> Date:   Wed Jul 20 11:15:34 2016 -0400
> METRON-334 Travis CI cache Maven dependencies (justinleet via dlyle65535) 
> closes apache/incubator-metron#196
> --
>  metron-deployment/vagrant/full-dev-platform/Vagrantfile | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> --
> ansible 2.0.0.2
>   config file = 
>   configured module search path = Default w/o overrides
> --
> Vagrant 1.8.5
> --
> Python 2.7.12
> --
> Apache Maven 3.3.9
> Maven home: /usr/share/maven
> Java version: 1.8.0_101, vendor: Oracle Corporation
> Java home: /usr/lib/jvm/java-8-oracle/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.4.0-31-generic", arch: "amd64", family: "unix"
> --
> Linux yoyo-desktop 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 
> 2016 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Yohann
>Priority: Trivial
>  Labels: deployment
>
> hdfs user can't access to /usr/metron/0.2.0BETA/patterns for the copy of grok 
> parsers to /apps/metron
> {code}
> yoyo@yoyo-desktop:~/incubator-metron/metron-deployment/vagrant/full-dev-platform$
>  vagrant provision
> ==> node1: Running provisioner: ansible...
> node1: Running ansible-playbook...
> ...
> 2016-07-22 14:34:23,656 p=9023 u=yoyo |  TASK [metron_streaming : include] 
> **
> 2016-07-22 14:34:23,715 p=9023 u=yoyo |  included: 
> /home/yoyo/incubator-metron/metron-deployment/roles/metron_streaming/tasks/grok_upload.yml
>  for node1
> 2016-07-22 14:34:23,775 p=9023 u=yoyo |  TASK [metron_streaming : Create HDFS 
> directory for grok patterns] **
> 2016-07-22 14:34:26,213 p=9023 u=yoyo |  changed: [node1]
> 2016-07-22 14:34:26,217 p=9023 u=yoyo |  TASK [metron_streaming : Assign hfds 
> user as owner of  /apps/metron/patterns HDFS directory] ***
> 2016-07-22 14:34:28,765 p=9023 u=yoyo |  changed: [node1]
> 2016-07-22 14:34:28,770 p=9023 u=yoyo |  TASK [metron_streaming : Assign 
> permissions of HDFS /apps/metron/patterns directory] ***
> 2016-07-22 14:34:31,281 p=9023 u=yoyo |  changed: [node1]
> 2016-07-22 14:34:31,287 p=9023 u=yoyo |  TASK [metron_streaming : Upload Grok 
> Patterns to hdfs:///apps/metron] **
> 2016-07-22 14:34:33,814 p=9023 u=yoyo |  fatal: [node1]: FAILED! => 
> {"changed": true, "cmd": ["hdfs", "dfs", "-put", "-f", 
> "/usr/metron/0.2.0BETA/patterns", "/apps/metron"], "delta": "0:00:02.107232", 
> "end": "2016-07-22 12:34:33.629405", "failed": true, "rc": 1, "start": 
> "2016-07-22 12:34:31.522173", "stderr": "put: Error accessing 
> file:/usr/metron/0.2.0BETA/patterns", "stdout": "", "stdout_lines": [], 
> "warnings": []}
> 2016-07-22 14:34:33,815 p=9023 u=yoyo |  PLAY RECAP 
> *
> 2016-07-22 14:34:33,816 p=9023 u=yoyo |  node1  : ok=172  
> changed=31   unreachable=0failed=1   
> {code}
> {code}
> [vagrant@node1 ~]$ ls -l /usr/metron/0.2.0BETA
> total 20
> d-.  2 root root 4096 22 juil. 12:19 bin
> d-.  3 root root 4096 22 juil. 12:19 config
> d-. 12 root root 4096 22 juil. 12:19 flux
> drwxr-xr-x.  2 root root 4096 22 juil. 12:34 lib
> d-.  2 root root 4096 22 juil. 12:19 patterns
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Created] (METRON-714) Migrate documentation into git

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Commented] (METRON-157) Create CEF Parser

[jira] [Assigned] (METRON-157) Create CEF Parser

[jira] [Commented] (METRON-342) Deployment - full-dev-platform - grok_upload - Error accessing file:/usr/metron/0.2.0BETA/patterns

10 matches

Site Navigation

Mail list logo

Footer information