I just want to say, there is a great JIRA already opened here: https://issues.apache.org/jira/browse/DRILL-4258
I added a comment, I would encourage others to add comments if they think this idea would be beneficial. On Wed, Mar 1, 2017 at 8:50 AM, John Omernik <j...@omernik.com> wrote: > So what would need to be done to get this process kick started? I see a > few components here: > > 1. Develop the table in sys (sys.functions) that stores the information > about the function. For this I propose this for discussions > > name - The name of the function > description - The Description of the function > docgroup- This should be the groupings under "SQL Functions" in the > documentation. See * below for proposal > tags - A way to tag functions for easy searching like select from > sys.functions where tags like '%string%' > arguments - The Ordered list of arguments and their types > return - What is returned (and it's type) > Examples: Usage examples that can display nicely in both select * from > sys.functions as well as provide enough information for use in HTML Docs. > > > * docgroup. > > Currently, in the documentation, the nesting is as below (I didn't expand > all the stuff on SQL Window and Nested Data). I want to outline my proposal > for to hand handle this... So I propose moving "SQL Window Functions", > "Nested Data Functions" and "Query Directory Functions" all under SQL > Functions and adding another group here "Common Functions". This will now > be the "Functions root". The Docgroup field in sys.function will have > levels and grouping separated by ":". Thus, firstlevel:second:level or > firstlevel:secondlevel:thirdlevel This will allow us to keep the > grouping in the sys.functions table, and make it so that on every release, > the documentation could be updated with a query. Note the level "SQL > Functions" would not be represented in the docgroup. > > Here are some examples of functions > > LOWER() > Current Doc Placement: SQL Functions -> String Manipulation > Proposed docgroup: 'Common Functions:String Manipulation' > How it would appear on the Doc page: SQL Functions -> Common Functions -> > String Manipulation > > COUNT() > Current Doc Placement: SQL Functions -> SQL Window Functions -> Aggregate > Window Functions > Proposed docgroup: 'SQL Window Functions:Aggregate Window Functions' > How it would appear on the Doc pages: SQL Functions -> SQL Window > Functions -> Aggregate Window Functions > > Basically it would be used to add the raw HTML to the final page in the > docgroup (Aggregate Window Functions) > > > Current Doc Layout: > > SQL Reference > SQL Reference Introduction > -> Data Types > Lexical Structure > Operators > -> SQL Functions > ->-> About SQL Function Examples > ->-> Math and Trig > ->->Data Type Conversion > ->->Date/Time Functions and Arithmetic > ->->String Manipulation > ->->Aggregate and Aggregate Statistical > ->-> Functions for handling Nulls > -> SQL Window Functions > ->-> ... > ->-> ... > -> Nested Data Functions > ->->... > ->->... > -> Query Directory Functions > > 2. Once we get to a certain point in development on sys.functions, we need > a call to arms. We need to come up with an initial "fill" of > sys.functions. For that we'll need to take current data fill it in, as > well as getting a list of all the ninja functions that have been added to > drill and not documented... not sure how get those with out an intense > code/jira review. > > 3. Come up with new function proposal guidelines. If you do a pull > request with a function, what will need to be included for your pull > request to be approved? We should not allow functions to be added to Drill > without a basic doc addition. > > 4. Update procedures? > > This is complicated, but done well, it could really put the knowledge and > analyst needs right in the system itself! > > John > > > > > On Tue, Feb 28, 2017 at 12:20 PM, John Omernik <j...@omernik.com> wrote: > >> You could also generate documentation updates via query at each release. >> This would be a great feature, move the information close to the analysts >> hands, I love how that would work. (I think I remember some talk about >> extending sys.options to be self documenting as well.... ) >> >> >> >> On Tue, Feb 28, 2017 at 12:11 PM, Jinfeng Ni <j...@apache.org> wrote: >> >>> Regarding the list of functions (build-in or UDF), someone once >>> suggested that we make the functions self-documented by adding a >>> sys.functions table. >>> >>> select * from sys.functions where name like '%SPLIT%'; >>> >>> return function_name, parameter_list, description etc. >>> >>> This way, use could simply query sys.functions using Drill. >>> >>> >>> >>> >>> >>> >>> >>> On Tue, Feb 28, 2017 at 9:02 AM, Jinfeng Ni <j...@apache.org> wrote: >>> >> 4. I think as part of developer review and pull requests that add >>> >> functions/functionality should require a pull request to also provide >>> a >>> >> documentation update. This helps to ensure that the docs keep up to >>> date, >>> >> as well as keeping users appraised of what is happening... i.e. it's >>> a good >>> >> "feeling" to see a great tool like Drill "improving" with new >>> >> functionality. >>> >> >>> >> Please, folks, we need to do some one time clean up (go back through >>> pull >>> >> requests to ensure all functions are documented up to now) and then >>> then >>> >> get processes in place to ensure ongoing updates. >>> >> >>> > >>> > That's a good suggestion. We should try our best to keep the code and >>> > doc in sync. >>> > >>> > +1 >>> >> >> >
