Thanks for the suggestions. I experimented with the slash in the connection value (off, then on again — updating the conf every time), but didn’t change what I was seeing.
I also removed the config from the plugin and put it in a cross-site.xml file. This time I used the profile credentials provider. Experimented with the connection string as well. It seems to take longer to respond, but it did finally give me the same validation error (object not found). I know the object is there in the bucket in the connection string. I wish there was a way to see the http url it is trying to reach out to, to see where the problem might be. Maybe if I modify the source. Thanks again, -Renier > On Aug 7, 2025, at 01:23, Charles Givre <cgi...@gmail.com> wrote: > > Renier, > Thanks for sending. In this result set, you can see that the s3 connection > is present which means that Drill is successfully connecting to S3 with your > keys. > One thing you might try is removing the slash at the end of the connection > string. IE: "connection": "s3a://bucket-name-here“. If that doesn't work, > the next thing I'd try is explicitly creating a workspace with your folder. > I have a feeling this is something silly like an extra slash in a query or > path somewhere because it usually is pretty easy to set up Drill with S3. > Best, > -- C > > > > >> On Aug 6, 2025, at 20:35, Renier Morales <ren...@morales-rodriguez.net> >> wrote: >> >> Hi, thanks for taking the time. Here is the output of that command: >> >> apache drill> SHOW DATABASES; >> +--------------------+ >> | SCHEMA_NAME | >> +--------------------+ >> | dfs.default | >> | dfs.root | >> | dfs.tmp | >> | s3.default | >> | s3.root | >> | information_schema | >> | sys | >> | cp.default | >> +--------------------+ >> 8 rows selected (1.745 seconds) >> >>> On Aug 6, 2025, at 20:04, Charles Givre <cgi...@gmail.com> wrote: >>> >>> HI Renier, >>> Can you try a `SHOW DATABASES` query and see whether your s3 connection is >>> showing up? >>> >>>> On Aug 6, 2025, at 09:15, Renier Morales <ren...@morales-rodriguez.net> >>>> wrote: >>>> >>>> Hello, >>>> >>>> I’m having some trouble getting the AWS S3 storage plugin working and I’m >>>> wondering if I can get some pointers. I have been following the docs at >>>> https://drill.apache.org/docs/s3-storage-plugin/ and doing other searches >>>> online. >>>> >>>> After running drill-embedded and going to http://localhost:8047/storage, I >>>> configured (and enabled) the S3 plugin with this (renaming things in the >>>> pasted config for security reasons): >>>> >>>> { >>>> "type": "file", >>>> "connection": "s3a://bucket-name-here/“, >>>> "config": { >>>> "fs.s3a.secret.key": “secret access key here", >>>> "fs.s3a.access.key": “access key id here", >>>> "fs.s3a.endpoint": "s3.us-east-2.amazonaws.com", >>>> "fs.s3a.impl.disable.cache": "true" >>>> }, >>>> "workspaces": { >>>> "root": { >>>> "location": "/", >>>> "writable": false, >>>> "defaultInputFormat": null, >>>> "allowAccessOutsideWorkspace": false >>>> } >>>> }, >>>> "formats": { >>>> "parquet": { >>>> "type": "parquet" >>>> } >>>> }, >>>> "authMode": "SHARED_USER", >>>> "enabled": true >>>> } >>>> >>>> Then in the drill-embedded console, I did this: >>>> >>>> apache drill> SELECT * FROM s3.root.`/my-data.parquet`; >>>> Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 176: >>>> Object ‘/my-data.parquet' not found within 's3.root' >>>> >>>> [Error Id: a6691c3d-c5a7-47af-af51-3adb32c15b1b ] (state=,code=0) >>>> >>>> I also did this, but got no files: >>>> >>>> apache drill> USE s3.root; >>>> +------+-------------------------------------+ >>>> | ok | summary | >>>> +------+-------------------------------------+ >>>> | true | Default schema changed to [s3.root] | >>>> +------+-------------------------------------+ >>>> 1 row selected (0.188 seconds) >>>> apache drill (s3.root)> SHOW files; >>>> +------+-------------+--------+--------+-------+-------+-------------+------------+------------------+ >>>> | name | isDirectory | isFile | length | owner | group | permissions | >>>> accessTime | modificationTime | >>>> +------+-------------+--------+--------+-------+-------+-------------+------------+------------------+ >>>> No rows selected (0.397 seconds) >>>> >>>> I’m not seeing any 403 errors, so it can’t be permissions. I am basically >>>> copying the object key from the AWS S3 UI and pasting it between the `` >>>> with the forward slash added (tried without it, too). I don’t know if my >>>> commands are just incorrect. Anything else I can look into? >>>> >>>> Thank you. >>>> >>>> -Renier >>> >>> >>> <smime.p7s> >> >
