Hi Gayan, On Tue, Jul 9, 2013 at 10:14 PM, Gayan Dhanushka <[email protected]> wrote:
> +1. Are you going to cater for column mapppings as well? > yes, This can be implemented as a built in annotation. > It would be good if you can do it as well. As a guideline you can follow > how the J2EE JPA related annotations have been defined and used. The user > needs to define the query related things in a user friendly manner where > the underlying annotation implementation will handle the actual > implementation of the hive query which will make life so much easier for > the developers. > > Of course that is the idea of having annotations. > On Tue, Jul 9, 2013 at 8:53 PM, Malith Dhanushka <[email protected]> wrote: > >> >> Hi Sinthuja/Bhathiya, >> >> Thanks for the suggestions and please find my comments inline, >> >> On Tue, Jul 9, 2013 at 12:27 PM, Bhathiya Jayasekara >> <[email protected]>wrote: >> >>> Hi Malith, >>> >>> This is an essential feature. One question. Will this support using >>> those variables inside TBLPROPERTIES values too? For example, inside SQL >>> query of "hive.jdbc.table.create.query"? >>> >> >> Yes, This is supported with annotations. >> >> >>> >>> And +1 for Sinthuja's suggestion. That will be very useful. >>> >>> Thanks. >>> --Bhathiya >>> >>> >>> On Tue, Jul 9, 2013 at 11:51 AM, Sinthuja Ragendran >>> <[email protected]>wrote: >>> >>>> Whether the annotation variables' scope is within hive query? or it's >>>> within the hive script? or it's global scope such that all hive scripts can >>>> use it, >>>> >>>> For an example there might be situations where user is interested in >>>> defining some variables to the entire script, and some are local to the >>>> hive query. So it is useful to have different scope also for the >>>> annotation. >>>> >>>> May be we can say, >>>> >>>> @foo -> all scripts scope >>>> @script.foo -> scope within the script >>>> @query.foo -> scope within the query. >>>> >>> >> Sure, will concern these in the implementation. >> >> >>> >>>> I believe you will be setting this annotation variables withing the >>>> HiveContext, which basically holds the context for the current tenant. >>>> >>> >> Yes, annotation variables will be available in HiveContext. >> >> Thanks, >> Malith >> >> >>> >>>> Thanks, >>>> Sinthuja. >>>> >>>> >>>> On Tue, Jul 9, 2013 at 11:23 AM, Malith Dhanushka <[email protected]>wrote: >>>> >>>>> Hi all, >>>>> >>>>> I have started implementing the $Subject. The idea of having an >>>>> annotation facility is to carryout some pre-processing of Hive queries >>>>> before they are being passed to the Hive engine. Currently we already have >>>>> a "class analyzer" which can be used execute some custom logic as a part >>>>> of >>>>> a Hive script. But the main use case of annotations is to inject run-time >>>>> properties to Hive execution context before the actual queries are carried >>>>> out by Hive. The annotation facility would be building upon this by having >>>>> set of such common analyzers which can manipulate the Hive queries or Hive >>>>> execution context which it is passed to Hive query engine. >>>>> >>>>> Annotation Syntax, >>>>> >>>>> *@script.foo(bar="value", bar1="value1",*)* >>>>> >>>>> >>>>> Annotation scheme will be externalized by giving *abstract >>>>> implementation of annotation* and *annotation-config.xml* file to >>>>> provide the annotation configuration which allows third party annotations >>>>> to be included to the system. >>>>> >>>>> *annotation-config.xml* >>>>> >>>>> <annotation> >>>>> <name>foo</name> >>>>> <class>org.wso2.carbon.analytics.hive.extension.annotation.foo</class> >>>>> <analyzer>org.wso2.carbon.analytics.hive.extension.foo</analyzer> >>>>> </annotation> >>>>> >>>>> <annotation> >>>>> ................................ >>>>> </annotation> >>>>> >>>>> >>>>> Potential use case for this in incremental data processing where any >>>>> query associated with "*@script.incremental(foo="value1", >>>>> bar="value2",*)*" would flag and setup the properties those are >>>>> required to present in order for that particular query to be executed in >>>>> an >>>>> incremental manner.There can be many other useful additions as well. >>>>> >>>>> Any suggestions, thoughts are welcome. >>>>> >>>>> -- >>>>> Malith Dhanushka >>>>> >>>>> Engineer - Data Technologies >>>>> *WSO2, Inc. : wso2.com* >>>>> >>>>> *Mobile* : +94 716 506 693 >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Sinthuja Rajendran* >>>> Software Engineer <http://wso2.com/> >>>> WSO2, Inc.:http://wso2.com >>>> >>>> Blog: http://sinthu-rajan.blogspot.com/ >>>> Mobile: +94774273955 >>>> >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *Bhathiya Jayasekara* >>> *Software Engineer,* >>> *WSO2 inc., http://wso2.com* >>> * >>> * >>> *Phone: +94715478185* >>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj* >>> *Twitter: https://twitter.com/bhathiyax* >>> >> >> >> >> -- >> Malith Dhanushka >> >> Engineer - Data Technologies >> *WSO2, Inc. : wso2.com* >> >> *Mobile* : +94 716 506 693 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > > Gayan Dhanushka > Software Engineer > WSO2 Inc. > http://wso2.com > > Mobile : 0716662327 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Dhanushka Engineer - Data Technologies *WSO2, Inc. : wso2.com* *Mobile* : +94 716 506 693
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
