OK. We'll do with JSONObject. ~ Chinmay On 22 Jan 2016 19:05, "Bhupesh Chawda" <[email protected]> wrote:
> Yes. So in the JDBC output case, a json object could be readily constructed > by the operator to get a POJO class rather than playing around with JSON > syntax. > > On Fri, Jan 22, 2016 at 6:54 PM, Chinmay Kolhatkar < > [email protected]> > wrote: > > > @bhupesh, would it help if JSONObject is taken as input to this utility > > rather than json string? > > > > Thanks, > > Chinmay > > On 22 Jan 2016 18:45, "Chinmay Kolhatkar" <[email protected]> > wrote: > > > > > Got it. I think approach 2 is still possible with json based schema. > > > So basically, the operator can create json schema as mention in first > > mail > > > based on column name and type. Then pass that json schema to this > utility > > > to get class to be used directly. > > > > > > Thanks, > > > Chinmay. > > > On 22 Jan 2016 18:38, "Priyanka Gugale" <[email protected]> > > wrote: > > > > > >> Say I need to emit pojo from JDBCInputOperator. I will use on fly POJO > > >> generator for this purpose instead of providing predefined class. > > >> Now to generate POJO on fly there are two options: > > >> 1. Get json as config from user as you specified. > > >> 2. Based on column name and type generate pojo fields and hence pojo > > class > > >> without much of user configuration. (I am only considering one level > > here, > > >> not the nested pojo) > > >> > > >> Option 1 is must have to provide user a way to control how we generate > > >> pojo. But option 2 is useful when user simply want to put column > values > > in > > >> some object. > > >> > > >> -Priyanka > > >> > > >> On Fri, Jan 22, 2016 at 6:30 PM, Chinmay Kolhatkar < > > >> [email protected]> > > >> wrote: > > >> > > >> > Hi Priyanka, > > >> > > > >> > Can you give an example? > > >> > > > >> > Thanks, > > >> > Chinmay. > > >> > > > >> > ~ Chinmay > > >> > On 22 Jan 2016 18:14, "Priyanka Gugale" <[email protected]> > > >> wrote: > > >> > > > >> > > We do need configuration in some form like json, but I was > thinking > > we > > >> > can > > >> > > have default wherever possible. This is an option and not > > replacement > > >> to > > >> > > json config. > > >> > > > > >> > > -Priyanka > > >> > > > > >> > > On Fri, Jan 22, 2016 at 4:34 PM, Chinmay Kolhatkar < > > >> > > [email protected]> > > >> > > wrote: > > >> > > > > >> > > > @priyanka > > >> > > > > > >> > > > Advantage with JSON schema is that one can represent nested POJO > > >> > > definition > > >> > > > in that. If you're referring to FieldInfo, I'm not sure if that > is > > >> > > possible > > >> > > > there. > > >> > > > > > >> > > > Even if we chose not to have nested POJO now, I think JSON > schema > > >> input > > >> > > to > > >> > > > this utility gives more chance to expand easily later. > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Fri, Jan 22, 2016 at 3:30 PM, Chinmay Kolhatkar < > > >> > > > [email protected]> > > >> > > > wrote: > > >> > > > > > >> > > > > Sure. We can make this utility to provide byte[] for compiled > > >> class. > > >> > > > > Though the utility will give a OutputStream object.. One can > get > > >> > byte[] > > >> > > > > out of that. > > >> > > > > > > >> > > > > On Fri, Jan 22, 2016 at 2:56 PM, Timothy Farkas < > > >> [email protected] > > >> > > > > >> > > > > wrote: > > >> > > > > > > >> > > > >> Suggestion: +1 for automatic generation of schema. The user > > >> should > > >> > be > > >> > > > able > > >> > > > >> to override the a manually specified schema though. > > >> > > > >> > > >> > > > >> Question: If the mechanism used to generate the class can > > >> produce a > > >> > > > byte[] > > >> > > > >> array, then you can send the byte[] array for the class to > > >> > downstream > > >> > > > >> operators and load the class from the byte[] array in each > > >> operator. > > >> > > An > > >> > > > >> example of how to do this is here: > > >> > > > >> > > >> > > > >> > > >> > > > >> > > >> > > > > > >> > > > > >> > > > >> > > > http://stackoverflow.com/questions/1781091/java-how-to-load-class-stored-as-byte-into-the-jvm > > >> > > > >> > > >> > > > >> Thanks, > > >> > > > >> Tim > > >> > > > >> > > >> > > > >> On Fri, Jan 22, 2016 at 12:59 AM, Priyanka Gugale < > > >> > > > >> [email protected]> > > >> > > > >> wrote: > > >> > > > >> > > >> > > > >> > Hi, > > >> > > > >> > > > >> > > > >> > Suggestion: > > >> > > > >> > This came up with one of the discussion with Pramod, will > it > > >> be a > > >> > > good > > >> > > > >> > idea, for database input operators to generate pojo based > on > > >> the > > >> > > > >> selected > > >> > > > >> > column field names and types? No need to accept json input > > from > > >> > > user. > > >> > > > >> > > > >> > > > >> > Question: > > >> > > > >> > How can we share this POJO among multiple operators when > > >> > application > > >> > > > is > > >> > > > >> > already launched and class is generated on the fly? > > >> > > > >> > > > >> > > > >> > -Priyanka > > >> > > > >> > > > >> > > > >> > On Thu, Jan 21, 2016 at 11:16 PM, Chinmay Kolhatkar < > > >> > > > >> > [email protected] > > >> > > > >> > > wrote: > > >> > > > >> > > > >> > > > >> > > Hi All, > > >> > > > >> > > > > >> > > > >> > > We're planning to add a utility in malhar-library for > > >> > generating a > > >> > > > >> POJO > > >> > > > >> > > class on the fly from given JSON schema. > > >> > > > >> > > > > >> > > > >> > > Use case is where the application is provided with schema > > and > > >> > that > > >> > > > >> needs > > >> > > > >> > to > > >> > > > >> > > be used in one or more operators either as a tuple over > the > > >> > stream > > >> > > > OR > > >> > > > >> for > > >> > > > >> > > processing. > > >> > > > >> > > > > >> > > > >> > > General Design: > > >> > > > >> > > 1. Utility will be provided with fqcn of the class and > > schema > > >> > > > >> definition > > >> > > > >> > > provided as json. > > >> > > > >> > > 2. The schema definition will look like following: > > >> > > > >> > > { > > >> > > > >> > > "fqcn":"<qualified class name>", > > >> > > > >> > > "fields": [ > > >> > > > >> > > { > > >> > > > >> > > "name":"field1", > > >> > > > >> > > "type":"long" > > >> > > > >> > > }, > > >> > > > >> > > { > > >> > > > >> > > "name":"field2", > > >> > > > >> > > "type": "string" > > >> > > > >> > > } > > >> > > > >> > > ] > > >> > > > >> > > } > > >> > > > >> > > 3. Supported types identified in "type" JSON field are: > > >> > > > >> > > boolean, char, byte, short, int, float, long, > double > > >> > > > >> > > 4. The output of this utility will be a generated .class > > >> file in > > >> > > the > > >> > > > >> form > > >> > > > >> > > of FSDataOutputStream. > > >> > > > >> > > 5. Xbean asm5 library will be used for this. > > >> > > > >> > > 6. Following methods will be added to the generated > class: > > >> > > > >> > > a. Getter/Setter methods for given fields. > > >> > > > >> > > b. simple toString - Generate string equivalent for > all > > >> the > > >> > > > fields > > >> > > > >> > > c. hashCode method - calculate the overall hashCode > > using > > >> > > > >> individual > > >> > > > >> > > field hashcodes, similar to how String.hashCode generates > > >> > hashCode > > >> > > > of > > >> > > > >> > > string using chars in it. > > >> > > > >> > > d. equals method - Similar to how String.equals > method > > >> has > > >> > > done > > >> > > > it > > >> > > > >> > > using individual char. > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > Questions I have about the functionality: > > >> > > > >> > > 1. Should the utility also support nested pojo definition > > via > > >> > > schema > > >> > > > >> > json? > > >> > > > >> > > In such case the field definition can look like > > >> following: > > >> > > > >> > > { "name":"nestedField", "type":"<fqcn of nested > > class>", > > >> > > > >> > > "fields":[....] } > > >> > > > >> > > The nested class will be provided > > >> > > > >> > > > > >> > > > >> > > 2. Should the utility also support giving out the > > >> > JarOutputStream > > >> > > > >> which > > >> > > > >> > > will have a generated jar? > > >> > > > >> > > > > >> > > > >> > > Please let me know your thoughts. > > >> > > > >> > > > > >> > > > >> > > Thanks, > > >> > > > >> > > Chinmay. > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
