[
https://issues.apache.org/jira/browse/DRILL-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992699#comment-14992699
]
Jacques Nadeau commented on DRILL-3423:
---------------------------------------
I think everyone is focusing too much on what the parser is capable of doing.
That should be the last thing we focus on. We should start with the user API.
Let's take an example log file format and decide what the output table should
look like. Then let's talk about how we could vary things to provide more
flexibility.
I proposed a particular format. When you guys saw it, you thought that we
needed more flexibility. I then proposed a modification to provide flexibility
around mapping between log file fields and table fields.
[~nielsbasjes], I appreciate your statements about the flexibility of the
plugin and agree it is very powerful. What we need to figure out is what is the
right way to expose that power in a SQL context. It doesn't make sense for
Drill to support custom dissectors. If someone wanted to provide that
capability, they would implement a Drill UDF (a similarly easy thing to
implement).
[[email protected]], with regards to your comment "This model makes it
extremely difficult to support mapping of data types", my whole suggestion
there was to expose more flexibility by using the mapping suggestion above. I'm
thinking that maybe I wasn't clear enough in my recommendation and you
misunderstood what I was suggesting.
So let's start with what a user would want. Then figure out how to implement
that. I think that will make this discussion substantially less conceptual.
> Add New HTTPD format plugin
> ---------------------------
>
> Key: DRILL-3423
> URL: https://issues.apache.org/jira/browse/DRILL-3423
> Project: Apache Drill
> Issue Type: New Feature
> Components: Storage - Other
> Reporter: Jacques Nadeau
> Assignee: Jim Scott
> Fix For: 1.4.0
>
>
> Add an HTTPD logparser based format plugin. The author has been kind enough
> to move the logparser project to be released under the Apache License. Can
> find it here:
> <dependency>
> <groupId>nl.basjes.parse.httpdlog</groupId>
> <artifactId>httpdlog-parser</artifactId>
> <version>2.0</version>
> </dependency>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
