[
https://issues.apache.org/jira/browse/CHUKWA-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Graham updated CHUKWA-472:
-------------------------------
Status: Open (was: Patch Available)
Canceling this patch, since I want to add one more bit of functionality that I
think will be useful. The current implementation expects the date to be the
first set of characters in the record. In some cases (i.e. Apache logs) that's
not the case. Adding the ability to optionally specify a regular expression to
locate where the date string exists in the record.
For a record like this for example, you could use configs like below:
{{10.10.182.49 [22/Apr/2010:15:07:27 -0700] "" 200 "-" "Mozilla/5.0 (Macintosh;
U; Intel Mac OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"
"some.site.com:8076"}}
{noformat}
<property>
<name>TsProcessor.time.regex.some_data_type</name>
<value>^(?:[\\d.]+) \\[(\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2}
[-+]\\d{4})\\] .*</value>
</property>
{noformat}
{noformat}
<property>
<name>TsProcessor.default.time.regex</name>
<value>^(?:[\\d.]+) \\[(\\d{2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2}
[-+]\\d{4})\\] .*</value>
</property>
{noformat}
> TsProcessor: make date format configurable
> ------------------------------------------
>
> Key: CHUKWA-472
> URL: https://issues.apache.org/jira/browse/CHUKWA-472
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: CHUKWA-472.1.patch
>
>
> The TsProcessor's default date format and it's date format for a given data
> type should both be configurable.
> - To set time format for a given data type:
> {noformat}
> <property>
> <name>TsProcessor.time.format.some_data_type</name>
> <value>yyyy-MM-dd HH:mm:ss,SSS</value>
> </property>
> {noformat}
> - To set the default time format:
> {noformat}
> <property>
> <name>TsProcessor.default.time.format</name>
> <value>yyyy-MM-dd HH:mm:ss,SSS</value>
> </property>
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
