This is an automated email from the ASF dual-hosted git repository. bridgetb pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push: new 0a0b93b add syslog format plugin doc 0a0b93b is described below commit 0a0b93bcea4ba8851001ab93c11bd41362cafb0b Author: Bridget Bevens <bbev...@maprtech.com> AuthorDate: Mon Apr 8 18:22:48 2019 -0700 add syslog format plugin doc --- .../035-plugin-configuration-basics.md | 4 +- .../plugins/116-sys-log-format-plugin.md | 58 ++++++++++++++++++++++ 2 files changed, 60 insertions(+), 2 deletions(-) diff --git a/_docs/connect-a-data-source/035-plugin-configuration-basics.md b/_docs/connect-a-data-source/035-plugin-configuration-basics.md index 2925d22..d058867 100644 --- a/_docs/connect-a-data-source/035-plugin-configuration-basics.md +++ b/_docs/connect-a-data-source/035-plugin-configuration-basics.md @@ -1,6 +1,6 @@ --- title: "Plugin Configuration Basics" -date: 2019-02-13 +date: 2019-04-09 parent: "Storage Plugin Configuration" --- There are several ways you can configure storage plugins. For example, you can configure storage plugins in the Drill Web UI, using REST API, or through configuration files. See [Configuring Storage Plugins]({{site.baseurl}}/docs/configuring-storage-plugins/) for more information. @@ -79,7 +79,7 @@ The following table describes the attributes you configure for storage plugins i </tr> <tr> <td>"formats" . . . "type"</td> - <td>"pcap"<br>"pcapng"<br>"text"<br>"parquet"<br>"json"<br>"maprdb"<br>"avro"<br>"image"<br>"sequencefile"<br>"httpd"</td> + <td>"pcap"<br>"pcapng"<br>"text"<br>"parquet"<br>"json"<br>"maprdb"<br>"avro"<br>"image"<br>"sequencefile"<br>"httpd"<br>"[syslog]({{site.baseurl}}/docs/sys-log-format-plugin/)"</td> <td>yes</td> <td>Format type. You can define two formats, csv and psv, as type "Text", but having different delimiters. </td> </tr> diff --git a/_docs/connect-a-data-source/plugins/116-sys-log-format-plugin.md b/_docs/connect-a-data-source/plugins/116-sys-log-format-plugin.md new file mode 100644 index 0000000..bb7211b --- /dev/null +++ b/_docs/connect-a-data-source/plugins/116-sys-log-format-plugin.md @@ -0,0 +1,58 @@ +--- +title: "Syslog Format Plugin" +date: 2019-04-08 +parent: "Connect a Data Source" +--- + +Starting in Drill 1.16, Drill provides a syslog format plugin, which enables Drill to query syslog formatted data as specified in RFC-5424, as shown: + + <165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][examplePriority@32473 class="high"] + +## Configuration Options +This syslog format plugin has the following configuration options: + + + +- **maxErrors** +Sets the maximum number of malformatted lines that the format plugin will tolerate before throwing an error and halting execution. +- **flattenStructuredData** +Syslog data optionally contains a series of key/value pairs known as the structured data. By default, Drill will parse these into a map. + + "syslog": { + "type": "syslog", + "extensions": [ "syslog" ], + "maxErrors": 10, + "flattenStructuredData": false + } + +## Fields + +In terms of data types, the `event_date` field is a datetime, the `severity_code`, `facility_code`, and `proc_id` are integers and all other fields are VARCHARs. + +**Note:** All fields, with the exception of the `event_date` field, are not required; therefore, all fields may not be present at all times. + +- **event_date** +This is the time of the event +- **severity_code** +The severity code of the event +- **facility_code** +The facility code of the incident +- **severity** +The severity of the event +- **facility** +- **ip** +The IP address or hostname of the source machine +- **app_name** +The name of the application that is generating the event +- **proc_id** +The process ID of the event that generated the event +- **msg_id** +The identifier of the message +- **message** +The actual message text of the event +- **raw** +The full text of the event + +## Structured Data + +Syslog data can contain a list of key/value pairs which Drill will extract in a field called `structured_data`. This field is a Drill map. \ No newline at end of file