A follow up with what I've tried so far. I've tried setting the
fs.s3a.endpoint property in the S3 storage plugin configuration like this:
{
"type": "file",
"enabled": true,
"connection": "s3a://json",
"config": {
"fs.s3a.endpoint": "my-minio-server.com",
"fs.s3a.access.key": "<my-access-key>",
"fs.s3a.secret.key": "<my-secret-key>",
"fs.s3a.connection.ssl.enabled": "false"
},
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null
}
},
...
}
I'm sure that the access keys and secrect keys are correct and that the
bucket exists but when I try the query:
SELECT * FROM `s3.tmp`.`employee.json` LIMIT 20
I end up with this error:
Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IOException: Bucket json does not exist [Error Id:
> 6f5bd823-1ca1-471c-8a0d-04767986fe15 on cba194516ae4:31010]
I've even used wireshark to see if I any calls are made to the minio server
(when running locally) but this doesn't happen when I query using Drill
(but it does when I try to point the aws-cli towards Minio so I know that
Minio is working correctly).
Regads,
/Johan
On Sat, Jun 11, 2016 at 8:22 AM, Johan Haleby <[email protected]>
wrote:
> Hi,
>
> I'd like to use Drill to connect to Minio <https://www.minio.io/> which
> is S3 compatible. I wonder if, and if so how, you can configure the S3
> Storage Plugin <https://drill.apache.org/docs/s3-storage-plugin/> to
> connect to a non-AWS S3 storage system? What I suspect that I need to do is
> to be able to configure the DNS/IP to the storage system somehow but I
> can't seem to find it in the docs.
>
> Regards,
> /Johan
>
