This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new ff7634b edit httpd page - change to format plugin vs storage
ff7634b is described below
commit ff7634bf8099a09019acbffa57d7725bc9900227
Author: Bridget Bevens <[email protected]>
AuthorDate: Wed Feb 13 14:30:26 2019 -0800
edit httpd page - change to format plugin vs storage
---
.../035-plugin-configuration-basics.md | 6 +--
...torage-plugin.md => 111-httpd-format-plugin.md} | 43 +++++++++++++++++-----
2 files changed, 37 insertions(+), 12 deletions(-)
diff --git a/_docs/connect-a-data-source/035-plugin-configuration-basics.md
b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
index b8d8ce2..2925d22 100644
--- a/_docs/connect-a-data-source/035-plugin-configuration-basics.md
+++ b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
@@ -1,6 +1,6 @@
---
title: "Plugin Configuration Basics"
-date: 2018-09-28 21:35:20 UTC
+date: 2019-02-13
parent: "Storage Plugin Configuration"
---
There are several ways you can configure storage plugins. For example, you can
configure storage plugins in the Drill Web UI, using REST API, or through
configuration files. See [Configuring Storage
Plugins]({{site.baseurl}}/docs/configuring-storage-plugins/) for more
information.
@@ -73,13 +73,13 @@ The following table describes the attributes you configure
for storage plugins i
</tr>
<tr>
<td>"formats"</td>
- <td>"pcap"<br><a
href="https://pcapng.github.io/pcapng/";>"pcapng"</a><br>"psv"<br>"csv"<br>"tsv"<br>"parquet"<br>"json"<br>"avro"<br>"maprdb"<br>"image"<br>"sequencefile"</td>
+ <td>"pcap"<br><a
href="https://pcapng.github.io/pcapng/";>"pcapng"</a><br>"psv"<br>"csv"<br>"tsv"<br>"parquet"<br>"json"<br>"avro"<br>"maprdb"<br>"image"<br>"sequencefile"<br>"httpd"</td>
<td>yes</td>
<td>One or more valid file formats for reading. Drill detects formats of
some files; others require configuration. The maprdb format is in installations
of the mapr-drill package. </td>
</tr>
<tr>
<td>"formats" . . . "type"</td>
-
<td>"pcap"<br>"pcapng"<br>"text"<br>"parquet"<br>"json"<br>"maprdb"<br>"avro"<br>"image"<br>"sequencefile"</td>
+
<td>"pcap"<br>"pcapng"<br>"text"<br>"parquet"<br>"json"<br>"maprdb"<br>"avro"<br>"image"<br>"sequencefile"<br>"httpd"</td>
<td>yes</td>
<td>Format type. You can define two formats, csv and psv, as type "Text",
but having different delimiters. </td>
</tr>
diff --git a/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
b/_docs/connect-a-data-source/plugins/111-httpd-format-plugin.md
similarity index 75%
rename from _docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
rename to _docs/connect-a-data-source/plugins/111-httpd-format-plugin.md
index 8191b53..f690841 100644
--- a/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/111-httpd-format-plugin.md
@@ -1,24 +1,49 @@
---
-title: "HTTPD Storage Plugin"
+title: "HTTPD Format Plugin"
date: 2017-03-31 21:49:40 UTC
parent: "Connect a Data Source"
---
-As of version 1.9, Drill can natively ingest and query web server logs. To
configure Drill to read server logs, you must modify the extensions section in
the dfs storage plugin configuration, as shown below:
+As of version 1.9, Drill can natively ingest and query web server logs. To
configure Drill to read server logs, add the httpd format configuration to the
formats section of a dfs storage plugin configuration, as shown below:
- "httpd": {
- "type": "httpd",
- "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
- "timestampFormat": null
- }
+
+ {
+ "storage":{
+ dfs: {
+ type: "file",
+ connection: "file:///",
+ workspaces: {
+ "root" : {
+ location: "/",
+ writable: false,
+ allowAccessOutsideWorkspace: false
+ },
+ "tmp" : {
+ location: "/tmp",
+ writable: true,
+ allowAccessOutsideWorkspace: false
+ }
+ },
+ formats: {
+ "csv" : {
+ type: "text",
+ extensions: [ "csv" ],
+ delimiter: ","
+ },
+ "httpd" : {
+ type: "httpd",
+ logFormat: "%h %t \"%r\" %>s %b \"%{Referer}i\"
\"%{user-agent}i\"",
+ timestampFormat: "dd/MMM/yyyy:HH:mm:ss ZZ"
+ },
+ ...
{% include startnote.html %}The logFormat section must match the format of the
log files, otherwise Drill cannot correctly parse the logs.{% include
endnote.html %}
## HTTPD Format Strings
-The following table lists the fields that log files can include. The
`timestampformat` is optional, but you can include a format for the time stamp
and Drill will parse the times in the log files into Drill dates.
+The following table lists the fields that log files can include. The
`timestampformat` is optional, but you can include a format for the time stamp,
and Drill will parse the times in the log files into Drill dates.
-|Format String | Variable Name |
+|**Format String** | **Variable Name** |
|--------------|---------------|
|%a | connection.client.ip |
|%{c}a | connection.client.peerip |
