I installed drill-logfile-plugin-1.0.0 JAR file to
<drill_install>/jars/3rdParty/ directory, and configured dfs as the following,
but I got error: "Please retry: error (invalid JSON mapping)", in the
sqlline.log file, it shows an error: Unable to find constructor for storage
config named 'log' of type
'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I double
checked the drill-logfile-plugin-1.0.0.jar file is in the jars/3rdParty folder:
My config for dfs with log plugin suport is:
{
"type": "file",
"connection": "file:///",
"config": null,
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"test": {
"location": "/Users/tsd",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
}
},
"formats": {
"log" : {
"type" : "log",
"extensions" : [ "log" ],
"fieldNames" : [ "date", "time", "pid", "action", "query" ],
"dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
"dateFormat" : "yyMMdd",
"timeFormat" : "HH:mm:ss",
"pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
"errorOnMismatch" : false
}
},
"enabled": true
}
If I configured the log section as this to remove some fields, the error will
disappear, but some fields will be missing:
"log": {
"type": "log",
"extensions": [
"log"
],
"fieldNames": [
"date",
"time",
"pid",
"action",
"query"
],
"pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
}
