Can you share more details of how the data is structured within the S3 bucket, using some examples? Also some representative queries of what you are currently doing and what you hope was possible to do instead? I'm not clear on what your question is.
The drill special attributes - filename, dir0, dir1, ... does work for data within S3 storage plugin. On Tue, May 9, 2017 at 7:06 AM, Wesley Chow <[email protected]> wrote: > What is the recommended way to issue a query against a large number of > tables in S3? At the moment I'm aliasing the table as a giant UNION ALL, > but is there a better way to do this? > > Our data is stored as a time hierarchy, like YYYY/MM/DD/HH/MM in UTC, but > unfortunately I can't simply run the query recursively on an entire day of > data. I usually need a day of data in a non-UTC time zone. Is there some > elegant way to grab that data using the dir0, dir1 magic columns? > > Thanks, > Wes >
