Thanks, Jacques. I might add your answers in a Javadoc and also maybe write an addendum to the documentation as I just told Ted.
It helps me learn and it will benefit others. On Saturday, August 29, 2015, Jacques Nadeau <[email protected]> wrote: > supportsRead -> Drill can read a table from this format (e.g. read a json > file) > supportsWrite -> Drill can write a table to this format (e.g. write a json > file) > > question: -- what does StoragePluginOptimizerRule do? > public Set<StoragePluginOptimizerRule> > getOptimizerRules(OptimizerRulesContext optimizerContext); > > You can generally return an empty set for this. You could possibly expose > one or more specialized rules for this format that the Drill query > optimizer would leverage. For example, if your plugin supported built-in > aggregations, you could expose a rule that rewrote a aggregate < your_scan > to be simply a your_scan_with_aggregation. > > public boolean isBlockSplittable() > Whether or not your format can be arbitrarily split and handle a random > start point. For example, a csv file supports this since our reader can > start in the middle of a line and stop at the next line beyond a given > split point. ON the flipside, Parquet can't use arbitrary split points. > JSON also doesn't support arbitrary split points since (in the most general > case), there is no way to search to the start of the next record. > > question: -- what criteria makes this truth; What does this mean? > public boolean isCompressible() > > Whether this format could also be in a compression container. For example: > csv.gz versus csv. This is an external compression container that would be > handled outside of your format code. Parquet also wouldn't support this > since it has an internal compression scheme. > > > question: -- is this configuration that is specified in the "Storage" > section on the web ui/config. files? > public StoragePluginConfig getConfig(); > > Yes, should be a jackson serializable object that extends > StoragePluginConfig. What is input when configuring in the web ui. > > > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Sat, Aug 29, 2015 at 7:01 PM, Edmon Begoli <[email protected] > <javascript:;>> wrote: > > > On *StoragePlugin*: > > > > question: -- I assume this means that sources of this storage format kind > > can be read from the original sources, correct? > > public boolean supportsRead(); > > > > question: -- Is this writing to the original source or does this mean > > creation of the table for this format? > > public boolean supportsWrite(); > > > > question: -- what does StoragePluginOptimizerRule do? > > public Set<StoragePluginOptimizerRule> > > getOptimizerRules(OptimizerRulesContext optimizerContext); > > > > question: -- is this configuration provided on the web ui or config. > files? > > public StoragePluginConfig getConfig(); > > > > > > On *EasyFormatPlugin*: > > > > question: -- what criteria makes this truth; What does this mean? > > public boolean isBlockSplittable() > > > > question: -- what criteria makes this truth; What does this mean? > > public boolean isCompressible() > > >
