+1. Are you going to cater for column mapppings as well? It would be good if you can do it as well. As a guideline you can follow how the J2EE JPA related annotations have been defined and used. The user needs to define the query related things in a user friendly manner where the underlying annotation implementation will handle the actual implementation of the hive query which will make life so much easier for the developers.
On Tue, Jul 9, 2013 at 8:53 PM, Malith Dhanushka <[email protected]> wrote: > > Hi Sinthuja/Bhathiya, > > Thanks for the suggestions and please find my comments inline, > > On Tue, Jul 9, 2013 at 12:27 PM, Bhathiya Jayasekara <[email protected]>wrote: > >> Hi Malith, >> >> This is an essential feature. One question. Will this support using those >> variables inside TBLPROPERTIES values too? For example, inside SQL query of >> "hive.jdbc.table.create.query"? >> > > Yes, This is supported with annotations. > > >> >> And +1 for Sinthuja's suggestion. That will be very useful. >> >> Thanks. >> --Bhathiya >> >> >> On Tue, Jul 9, 2013 at 11:51 AM, Sinthuja Ragendran <[email protected]>wrote: >> >>> Whether the annotation variables' scope is within hive query? or it's >>> within the hive script? or it's global scope such that all hive scripts can >>> use it, >>> >>> For an example there might be situations where user is interested in >>> defining some variables to the entire script, and some are local to the >>> hive query. So it is useful to have different scope also for the annotation. >>> >>> May be we can say, >>> >>> @foo -> all scripts scope >>> @script.foo -> scope within the script >>> @query.foo -> scope within the query. >>> >> > Sure, will concern these in the implementation. > > >> >>> I believe you will be setting this annotation variables withing the >>> HiveContext, which basically holds the context for the current tenant. >>> >> > Yes, annotation variables will be available in HiveContext. > > Thanks, > Malith > > >> >>> Thanks, >>> Sinthuja. >>> >>> >>> On Tue, Jul 9, 2013 at 11:23 AM, Malith Dhanushka <[email protected]>wrote: >>> >>>> Hi all, >>>> >>>> I have started implementing the $Subject. The idea of having an >>>> annotation facility is to carryout some pre-processing of Hive queries >>>> before they are being passed to the Hive engine. Currently we already have >>>> a "class analyzer" which can be used execute some custom logic as a part of >>>> a Hive script. But the main use case of annotations is to inject run-time >>>> properties to Hive execution context before the actual queries are carried >>>> out by Hive. The annotation facility would be building upon this by having >>>> set of such common analyzers which can manipulate the Hive queries or Hive >>>> execution context which it is passed to Hive query engine. >>>> >>>> Annotation Syntax, >>>> >>>> *@script.foo(bar="value", bar1="value1",*)* >>>> >>>> >>>> Annotation scheme will be externalized by giving *abstract >>>> implementation of annotation* and *annotation-config.xml* file to >>>> provide the annotation configuration which allows third party annotations >>>> to be included to the system. >>>> >>>> *annotation-config.xml* >>>> >>>> <annotation> >>>> <name>foo</name> >>>> <class>org.wso2.carbon.analytics.hive.extension.annotation.foo</class> >>>> <analyzer>org.wso2.carbon.analytics.hive.extension.foo</analyzer> >>>> </annotation> >>>> >>>> <annotation> >>>> ................................ >>>> </annotation> >>>> >>>> >>>> Potential use case for this in incremental data processing where any >>>> query associated with "*@script.incremental(foo="value1", >>>> bar="value2",*)*" would flag and setup the properties those are >>>> required to present in order for that particular query to be executed in an >>>> incremental manner.There can be many other useful additions as well. >>>> >>>> Any suggestions, thoughts are welcome. >>>> >>>> -- >>>> Malith Dhanushka >>>> >>>> Engineer - Data Technologies >>>> *WSO2, Inc. : wso2.com* >>>> >>>> *Mobile* : +94 716 506 693 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *Sinthuja Rajendran* >>> Software Engineer <http://wso2.com/> >>> WSO2, Inc.:http://wso2.com >>> >>> Blog: http://sinthu-rajan.blogspot.com/ >>> Mobile: +94774273955 >>> >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Bhathiya Jayasekara* >> *Software Engineer,* >> *WSO2 inc., http://wso2.com* >> * >> * >> *Phone: +94715478185* >> *LinkedIn: http://www.linkedin.com/in/bhathiyaj* >> *Twitter: https://twitter.com/bhathiyax* >> > > > > -- > Malith Dhanushka > > Engineer - Data Technologies > *WSO2, Inc. : wso2.com* > > *Mobile* : +94 716 506 693 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Gayan Dhanushka Software Engineer WSO2 Inc. http://wso2.com Mobile : 0716662327
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
