drill wont open parquet files in s3 with _metadata and _common_metadata files

around them in some way? Thanks all Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and ma

Re: drill wont open parquet files in s3 with _metadata and _common_metadata files

Nevermind, this is because I changed the directory structure. Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information

Do i need hadoop installed to use dfs storage?

ultInputFormat": "json" }, ... And i get the same thing. I do not have hadoop installed. do i need it? Thanks, Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments c

Re: queries take over 2 min

Yep, I disabled all other storage plugins (default dfs, cp and custom s3 plugin) and the alter session took .5 sec as opposed to 169. Thank you! Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail

connecting to s3

, ResponseStatus: Forbidden, XML Error Message: AccessDeniedAccess Denied95BB171A886F3C27NLoKTo5jzzBS3j0mIRUeVwF8tGvTFUh/4jJrkPr+0U6kjo1e3t7uX3wv2Thkb1G/+/ZDODJXbVE= What am I doing wrong? Thanks all, Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.28

deploy dockerized drill cluster

etting the json all_text_mode and read_numbers_as_double. I have a drill-embeded instance that seems to pick these json setting from the drill-override file just fine. Can you see what I'm doing wrong? ________ Scott Kinney | DevOps stem <http://www.stem.com/> |

Re: Error creating view

Yep, I didn't set the workspace 'writable': true user error as usual Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information

Re: deploy dockerized drill cluster

it's still not picking up the store.json* config changes The only way I can see to set these is with running ALTER SYSTEM query after drill api is up. Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail

Error creating view

=0) dfsl is just a copy of the default dfs storage plugin. Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and propr

Best way to set schema to handle different json structures

will have out data in s3 in a name/year/month/day and I dont want to have to put this schema file in every directory is s3. that seems unmanageable. Is there a way to set the schema from a file before making a query via the REST api? Scott Kinney | DevOps stem <h

Re: Best way to set schema to handle different json structures

No i haven't. I'll give that a try. Thanks. Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole

gzipped json files not named .json.gz

Can I have drill open gzipped json files who's names do not end in .json.gz? We have a spark job generating these files and it just dosn't want to change the name or append the .json.gz. ? Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.28

Scaling Drill with s3 plugin

>From what I can tell the s3 plugin must be configured via the web interface. >We really need drill to be able to scale and cluster. Has anyone found a way >to configure drill to use s3 with xml (or whatever) files rather than the web >interface? ? __

Re: gzipped json files not named .json.gz

Well that's a bummer but I believe it setting "defaultInputFormat": "json" doesn't seem to have any effect. ________ Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachmen

Re: Scaling Drill with s3 plugin

Ah, ok. Thats something. Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended

Re: deploy dockerized drill cluster

Hm, i must have set those another way in embeded mode. I can't see where. Those settings persist between drill restarts. Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain

Re: gzipped json files not named .json.gz

. thanks! Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any

array in json with mixed values (int and float)

There is a key 'foo': [ 99, 99.1, 99.8 ]. Is there a way get drill to handle this? Maybe treat all ints as floats? ________ Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any att

Re: array in json with mixed values (int and float)

! Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has

Re: array in json with mixed values (int and float)

That looks promising but didn't work. Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Millbrae, California 94030 This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use

Re: missing data in json structure when using web / api

Not that I know of but I'm new to drill. I've done 'alter system' for json all_text_mode & read_numbers_as_double. Do you know of a setting that might cause something like this? ____ Scott Kinney | DevOps stem | m 510.282.1299 100 Rollins Road, Mill

missing data in json structure when using web / api

When i query from sqlline i can see all the data, very complicated / nested json structure but when i query with the api or the web ui a lot of the data is missing. ? Scott Kinney | DevOps stem <http://www.stem.com/> | m 510.282.1299 100 Rollin