Data-source/partition pruning with remotely stored non-parquet files

2019-02-19 Thread Lokendra Singh Panwar
Hi All, I am writing a custom storage plugin to read and query non-static json files stored on remote services and wanted to use something similar to Drill's partition pruning to optimise my queries. The files are looked dynamically within the plugin up via an external service based on the

Some queries related to writing a custom storage-plugin

2019-01-31 Thread Lokendra Singh Panwar
Hi All, I am relatively new to Drill and trying to write a custom storage plugin. I have couple of (naive sounding) queries, so mostly need some brief pointers: a) Why do a StoragePlugin have to implement a registerSchemas() (coming from SchemaFactory)? I assumed that drill would discover the