-1 for putting source mapping in the object model, object model should be something we can use in the "Data Mapping Tooling" and elsewhere on the platform. Shouldn't we keep JSON in mind as a first-class citizen when designing this definition language. I think using things like XPath is a bit too tied-in to the XML ecosystem. Even though it's fine to use it in a translation layer (XML->WSO2OM), using it in object model itself don’t look nice.
Since we are already planning to introduce a data mapping tool, I think we should reuse the configuration language used by it to translate the data into the WSO2OM as well. On Thu, Jun 20, 2013 at 6:37 PM, Anjana Fernando <[email protected]> wrote: > Hi, > > On Wed, Jun 5, 2013 at 11:33 AM, Manuranga Perera <[email protected]> wrote: > >> I like to add that, having a format like this will be very helpful to >> generate dashboards and other visualizations in a generic way. >> >> 1) what are the basic types we should support? (besides string) >> > > I guess, all the primitive types we usually find. Will create a list later. > > >> 2) are all types null-able ? >> > > Yes, I guess, I don't think we need to go into the complexity of > specifying if it's nillable and so on. > > Also, I also had a chat with Srinath on how we map from/to this object > model. For example, a sample XML mapping scenarios are specified below: > > * Sample XML message:- > > <Employees> > <Employee> > <FirstName>Will</FirstName> > <LastName>Smith</LastName> > <Nicknames> > <Name>bigwilly</Name> > <Name>smithy</Name> > </Nicknames> > </Employee> > <Employee> > <FirstName>Sameera</FirstName> > <LastName>Jayasoma</LastName> > <Nicknames> > <Name>sam</Name> > <Name>jason</Name> > </Nicknames> > </Employee> > </Employees> > > * The intermediate object model:- > > > <struct name="Data"> > <list name="Employees"> > <struct> > <field name="FirstName" type="string" /> > <field name="LastName" type="string" /> > <field name="DOB" type="dateTime" /> > > <list name="Nicknames" type="string" /> > </struct> > </list> > </struct> > > So, now the mapping would be between the XML to the object model, which is > the scenario when we receive some message into a data service, and the > other is, object model to XML, which is when we are sending a response from > a data service request. > > The object model -> XML can be done by an XML templating method like the > following: > > <Employees> > <Employee __source__="$Data.Employees"> > <FirstName>$Data.Employees.FirstName</FirstName> > <LastName>$Data.Employees.LastName</LastName> > <DOB>1986/05/01</DOB> > <Nicknames> > <Name>$Data.Employees.Nicknames[0]</Name> > </Nicknames> > </Employee> > </Employees> > > Here, the "__source__" attribute is a special reserved attribute to > specify, from which list/array we are suppose to rotate the repeating > element. > > The other mapping, which is mapping XML to the object model is a bit > tricky, where we need to decide on, if we put the source of the values > inside the object model itself or in a separate mapping. To put it inline > the object model itself, we can follow something like the following:- > > > <struct name="Data"> > <list name="Employees"> > <struct> > <field name="FirstName" type="string" > source="/Employees/Employee/Firstname" /> > <field name="LastName" type="string" > source="/Employees/Employee/Lastname"/> > <field name="DOB" type="dateTime" > source="/Employees/Employee/DOB"/> > <list name="Nicknames" type="string" > source="/Employees/Employee/Nicknames"/> > </struct> > </list> > </struct> > > So that's basically putting a "source" attribute and give the XPath > expression (which is for XML) and provide how the value should come. Also, > we would also have a automatic generation of the XML according to the same > structure as the intermediate model, where it will simply map its fields to > element in an XML. We need that functionality basically to support input > XML streaming for this structure, where if we use XPath, we cannot achieve > streaming for this. > > So putting the mapping details directly in the object model means, this > object model is not independent, and we cannot re-use this for another > mapping. Or else, if we need the re-use also, we can define pure object > model separately, and again provide the same object model with the > mappings. Or else, we can provide a syntax like the following, to directly > assign the values to the fields. > > <input-mapping> > <target name="$Data.Employees.FirstName" > source="/Employees/Employee/Firstname" /> > <target name="$Data.Employees.LastName" > source="/Employees/Employee/Lastname" /> > <target name="$Data.Employees.DOB" source="/Employees/Employee/DOB" /> > <target name="$Data.Employees.Nicknames" > source="/Employees/Employee/Nicknames" /> > </input-mapping> > > So we thought, perhaps the first approach maybe more convenient to the > user, but then he loses the option to re-use the same object model. Any > feedback on this is appreciated. > > Cheers, > Anjana. > > >> >> On Wed, Jun 5, 2013 at 10:21 AM, Anjana Fernando <[email protected]> wrote: >> >>> Hi Srinath, >>> >>> On Wed, Jun 5, 2013 at 4:42 AM, Srinath Perera <[email protected]> wrote: >>> >>>> 1) why we have struct inside the <list> ? >>>> >>> >>> That basically says, this is a list of elements of a struct type. In >>> the same way, for primitive types, we directly mention the type as I've >>> shown in the "nicknames" list. >>> >>> >>>> 2) How do we query? Xpaths (from input message) >>>> >>> >>> Yeah, the input message is not mentioned here yet, the mapping from >>> input message to this intermediate model we have to figure out. Yeah, one >>> option to do is, fill in the field in the intermediate model by using XPath >>> for XML, and JSONPath for JSON and so on .. >>> >>> >>>> 3) Are we trying to do one data model that support both JSON and XML? >>>> So each service will have both. If we are OK for JSON or XML model (not >>>> both) >>>> did we consider the templates oppose to this model? (just thinking >>>> aloud. Templates are very clean if it can work) >>>> >>> >>> Yeah, basically one intermediate data model in the query section for all >>> wire formats, and in the operations, when defining the actual XML or JSON >>> formats, we do the mapping from our model from the query to that format, >>> for XML, JSON and so on, for example, for XML, we can use a templating >>> mechanism like the following (this would be the ouput section of the >>> operation) >>> >>> <operation name="getCustomerAddress" transports="http,https,thrift"> >>> ... >>> <output messageType="application/xml, text/xml"> >>> <message> >>> <status> >>> <code>$status.code</code> >>> <warns> >>> <entry>$status.warnings</entry> >>> <warns> >>> </status> >>> <customers> >>> <customer> >>> >>> <customerNumber>$customers.customer.customerNumber</customerNumber> >>> >>> <customerLastName>$customers.customer.customerLastName</customerLastName> >>> >>> <customerFirstName>$customers.customer.customerFirstName</customerFirstName> >>> </customer> >>> </customers> >>> </message> >>> </output> >>> >>> ... >>> </operation> >>> >>> So here, the field such as "$customers.customer.customerNumber" are >>> values read from the data model in the query. So here, there are some >>> things we have to figure out, like how the XML model here infers the >>> information as to how "//customers/customer" element should be a repeating >>> element and all, even though most of the information for this can be >>> retrieved from the intermediate data model in the query, but there are some >>> edge cases, which can be tricky. >>> >>> Cheers, >>> Anjana. >>> >>> >>>> >>>> --Srinath >>>> >>>> >>>> On Tue, Jun 4, 2013 at 6:25 PM, Anjana Fernando <[email protected]>wrote: >>>> >>>>> Hi, >>>>> >>>>> Today we had a meeting with Sanjiva, Sumedha, Harshana, Manuranga and >>>>> myself. The discussion was the new language changes for DSS 4.0, and also >>>>> how some of its features to be used as the general mapping format to be >>>>> used in other places in the platform as well, e.g. for the DevStudio data >>>>> mapping tool. >>>>> >>>>> Basically in the new format, in the data service queries, we are going >>>>> to make it's input and output data abstract. That is, it will no longer >>>>> define XML elements in the result sections, and it will have a generic >>>>> object model for the parameters and for the result section. >>>>> >>>>> So this is basically an intermediate object model populated from the >>>>> data received from the data sources, after a specific query is executed, >>>>> and this data structure will be serialized into a target wire format, such >>>>> as XML, JSON, Thrift and so on, which are defined in the operations and >>>>> resource definitions. And also in the same way, the input message will be >>>>> converted from the wire format to this intermediate data structure. >>>>> >>>>> With these functionality, data services can support complex data >>>>> service scenarios, such as complex input messages, which contains data for >>>>> many aggregated queries, and by supporting the new object model, it can >>>>> map >>>>> to many output types using the same data services query, so we can >>>>> efficiently support features like content negotiation. >>>>> >>>>> Following contains a sample configuration of the format we decided so >>>>> far to represent the object model:- >>>>> >>>>> <struct name="Data"> >>>>> <list name="Employees"> >>>>> <struct> >>>>> <field name="firstName" type="string" /> >>>>> <field name="lastName" type="string" /> >>>>> <field name="dob" type="dateTime" /> >>>>> <list name="nicknames" type="string" /> >>>>> </struct> >>>>> </list> >>>>> </struct> >>>>> >>>>> Notice the inner struct is anonymous, but it is also possible for a >>>>> named struct to be defined globally and it to be reference within another >>>>> type definition, like "<list name="Employees" type="Employee">", this is >>>>> in >>>>> the same way it works in XML schema. With this, we can re-use same type >>>>> definitions in different queries etc.. >>>>> >>>>> So basically something similar to above configuration will represent a >>>>> data service query parameters section (it doesn't have to be a single >>>>> parent item there, it can have multiple root field items), and also, same >>>>> type of configuration will be used for the result section as well, and >>>>> there will be a separate mapping configuration, which will be used to map >>>>> the data results of a query to this object model, that is we have to tell, >>>>> how the field "Data.Employees.firstName" will be populated, as in, which >>>>> record in the result set will fill that value, this mapping format is >>>>> still >>>>> not decided. I will work on a possible representation on that and update >>>>> this thread. >>>>> >>>>> Cheers, >>>>> Anjana. >>>>> >>>>> -- >>>>> *Anjana Fernando* >>>>> Technical Lead >>>>> WSO2 Inc. | http://wso2.com >>>>> lean . enterprise . middleware >>>>> >>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Srinath Perera, Ph.D. >>>> http://www.cs.indiana.edu/~hperera/ >>>> http://srinathsview.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> *Anjana Fernando* >>> Technical Lead >>> WSO2 Inc. | http://wso2.com >>> lean . enterprise . middleware >>> >> >> >> >> -- >> With regards, >> *Manu*ranga Perera. >> >> > > > -- > *Anjana Fernando* > Technical Lead > WSO2 Inc. | http://wso2.com > lean . enterprise . middleware > -- With regards, *Manu*ranga Perera.
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
