Benchmark numbers using Drill

Hi, Is there any public performance benchmark that users have achieved using Drill in production scenarios ? It would be useful if someone can pass me any links for customer user stories. Regards

Re: S3 Connection Issues

Hi Padma, The bucket is is us-west-2. I also discovered that some of the variable names in the documentation on the main Drill site are incorrect. Do I need to specify the region in the configuration somewhere? As an update, after discovering that the variable names are incorrect and that

Re: S3 Connection Issues

Which AWS region are you trying to connect to ? We have a problem connecting to regions which support only v4 signature since the version of hadoop we include in Drill is old. Last time I tried, using Hadoop 2.8.1 worked for me. Thanks Padma > On Oct 18, 2017, at 8:14 PM, Charles Givre wrote

S3 Connection Issues

Hello all, I’m trying to use Drill to query data in an S3 bucket and running into some issues which I can’t seem to fix. I followed the various instructions online to set up Drill with S3, and put my keys in both the conf-site.xml and in the plugin config, but every time I attempt to do anythi

S3 with mixed files

Hi, Attempting to use Apache Drill to parse Open edX tracking log files I have stored on S3. I've successfully set up an S3 connection and I can see my different directories in the target S3 bucket when I type `show files;` in embedded drill. Hooray! However, I can't seem to do a query. I keep g

Re: describe query support? (catalog metadata, etc)

I’d like to second Alfredo’s request. I’ve been trying to get Drill to work with some open source visualization tools such as SqlPad and Metabase and the issue I keep running into is that Drill doesn’t have a convenient way to describe how it interprets flat files. This is really frustrating f

Re: describe query support? (catalog metadata, etc)

There were discussions on the need of building a catalog for drill. But I don't think that's the focus right now. And I am not sure the community will ever decide to go in that direction. For now, you best bet is to create views on top of your JSON/CSV data. Fro

Re: Date Conversion Question

Hi Julian, Alas, this doesn’t work in Drill since Drill uses Joda time formats. However, you got me thinking about this and I actually got it to work w/o using the substring or other weird string manipulation functions. SELECT to_timestamp ('2017-08-10T09:12:26.000Z', '-MM-dd''T''hh:mm:ss

describe query support? (catalog metadata, etc)

Hi I'm experimenting using Drill as a data virtualization component via JDBC and it generally works great for my needs. However some of the components connected via JDBC needs basic metadata/catalog informations, and they seems to be missing for JSON / CSV sources. For example the simple query D

Re: Date Conversion Question

FWIW I was doing very similar substring (etc) machinations until we started converting output from back-end data-generation tools directly into parquet (using other tools). IMO it's a common enough format (at least in the types of data you and I likely have to work with :-) that it'd be great if th

Date Conversion Question

Hello Drillers, I have a silly question which I’m a little stuck with. I have some data in CSV format with dates in the following format: 2017-08-10T09:12:26.000Z. I’m trying to convert this into a date time data field so that I have both the date and the hours, however I keep running into r

Re: Issue in executing query on Drill Cluster

Can you please see if you can access that file as the default user in both cases, because you mention that the default user is different in both cases. Try to do a hadoop fs -ls on that file, from both the different users and verify. Also, can you please share the JDBC connection string that y