Hi Sinthuja/Bhathiya, Thanks for the suggestions and please find my comments inline,
On Tue, Jul 9, 2013 at 12:27 PM, Bhathiya Jayasekara <[email protected]>wrote: > Hi Malith, > > This is an essential feature. One question. Will this support using those > variables inside TBLPROPERTIES values too? For example, inside SQL query of > "hive.jdbc.table.create.query"? > Yes, This is supported with annotations. > > And +1 for Sinthuja's suggestion. That will be very useful. > > Thanks. > --Bhathiya > > > On Tue, Jul 9, 2013 at 11:51 AM, Sinthuja Ragendran <[email protected]>wrote: > >> Whether the annotation variables' scope is within hive query? or it's >> within the hive script? or it's global scope such that all hive scripts can >> use it, >> >> For an example there might be situations where user is interested in >> defining some variables to the entire script, and some are local to the >> hive query. So it is useful to have different scope also for the annotation. >> >> May be we can say, >> >> @foo -> all scripts scope >> @script.foo -> scope within the script >> @query.foo -> scope within the query. >> > Sure, will concern these in the implementation. > >> I believe you will be setting this annotation variables withing the >> HiveContext, which basically holds the context for the current tenant. >> > Yes, annotation variables will be available in HiveContext. Thanks, Malith > >> Thanks, >> Sinthuja. >> >> >> On Tue, Jul 9, 2013 at 11:23 AM, Malith Dhanushka <[email protected]>wrote: >> >>> Hi all, >>> >>> I have started implementing the $Subject. The idea of having an >>> annotation facility is to carryout some pre-processing of Hive queries >>> before they are being passed to the Hive engine. Currently we already have >>> a "class analyzer" which can be used execute some custom logic as a part of >>> a Hive script. But the main use case of annotations is to inject run-time >>> properties to Hive execution context before the actual queries are carried >>> out by Hive. The annotation facility would be building upon this by having >>> set of such common analyzers which can manipulate the Hive queries or Hive >>> execution context which it is passed to Hive query engine. >>> >>> Annotation Syntax, >>> >>> *@script.foo(bar="value", bar1="value1",*)* >>> >>> >>> Annotation scheme will be externalized by giving *abstract >>> implementation of annotation* and *annotation-config.xml* file to >>> provide the annotation configuration which allows third party annotations >>> to be included to the system. >>> >>> *annotation-config.xml* >>> >>> <annotation> >>> <name>foo</name> >>> <class>org.wso2.carbon.analytics.hive.extension.annotation.foo</class> >>> <analyzer>org.wso2.carbon.analytics.hive.extension.foo</analyzer> >>> </annotation> >>> >>> <annotation> >>> ................................ >>> </annotation> >>> >>> >>> Potential use case for this in incremental data processing where any >>> query associated with "*@script.incremental(foo="value1", >>> bar="value2",*)*" would flag and setup the properties those are >>> required to present in order for that particular query to be executed in an >>> incremental manner.There can be many other useful additions as well. >>> >>> Any suggestions, thoughts are welcome. >>> >>> -- >>> Malith Dhanushka >>> >>> Engineer - Data Technologies >>> *WSO2, Inc. : wso2.com* >>> >>> *Mobile* : +94 716 506 693 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Sinthuja Rajendran* >> Software Engineer <http://wso2.com/> >> WSO2, Inc.:http://wso2.com >> >> Blog: http://sinthu-rajan.blogspot.com/ >> Mobile: +94774273955 >> >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Bhathiya Jayasekara* > *Software Engineer,* > *WSO2 inc., http://wso2.com* > * > * > *Phone: +94715478185* > *LinkedIn: http://www.linkedin.com/in/bhathiyaj* > *Twitter: https://twitter.com/bhathiyax* > -- Malith Dhanushka Engineer - Data Technologies *WSO2, Inc. : wso2.com* *Mobile* : +94 716 506 693
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
