Hello All,
According to the requirement for supporting multiple namespaces in XML document in [1] according to [2] it is not possible by the single XSD schema approach we are taking ATM. Currently our approach to generate avro schema from XML is, XML ---> XSD schema ---> Avro schema. If we are to support multiple namespaces in the XML doc we should either generate multiple cascaded XSD schemas as explained in [2] or perform rigorous manipulation of the generated avro schema. IMO this is not a very good approach, Further generating avro schema object per xml complex type to support multiple namespaces will introduce additional overhead and lengthen the resulting avro schema. WDYT? [1] https://docs.google.com/spreadsheets/d/15eyRuxn9rFazULASqtB2DZaSAVznKqSNnIGrwvdpzic/edit?ts=56d5250b#gid=0 [2] http://stackoverflow.com/questions/9805359/how-to-handle-multiple-namespaces-with-different-uri-in-xsd Regards Awanthika Awanthika Senarath Software Engineer, WSO2 Inc. Mobile: +94717681791 On Fri, Feb 26, 2016 at 4:01 PM, Awanthika Senarath <[email protected]> wrote: > Hi all, > > This feature is implemented using the nokia avro scheme gen tol with > apache license[1]. It was modified to remove unnecessary class gen codes > with JAXB and also to manipulate the generated avro to insert child JSON > objects. > > I will insert this source into our datamapper tool to enable it within > dev-studio. > > > [1] https://github.com/Nokia/Avro-Schema-Generator > > > > Awanthika Senarath > Software Engineer, WSO2 Inc. > Mobile: +94717681791 > > > > On Fri, Feb 26, 2016 at 10:16 AM, Awanthika Senarath <[email protected]> > wrote: > >> Hi all, >> >> Is there any particular reason as to why we are sticking into avro schema >> for the datamapper implementation? >> >> AFAIS the only tool (open/non-open source) that exist for XSD to avro >> conversion, is the same that has been used by the team when we attempted >> the data mapper first 2 years back. >> >> This generates JAVA model from XSD schema using JAXB and generate the >> avro schema from the generated JAVA model. >> >> This has drawbacks : >> >> 1. Unable to gen avro for element children that are not wrapped by >> complex-type, >> 2. Unable to identify namespaces >> 3. Unable to generate single avro for a nested XSD, which I fixed by >> inserting the separate JSON objects generated accordingly to get the final >> nested avro schema. >> >> >> for (1) we need to manipulate the xsd give an insert complex-type tags. >> (2) cannot be supported with JAXB based generator, need to evaluate >> optional approach for this. >> >> >> >> meanwhile I am evaluating [2] as well. >> >> >> [2] https://github.com/elodina/xml-avro >> >> Regards >> Awanthika >> >> >> >> >> Awanthika Senarath >> Software Engineer, WSO2 Inc. >> Mobile: +94717681791 >> >> >> >> On Thu, Feb 25, 2016 at 5:42 PM, Sohani Weerasinghe <[email protected]> >> wrote: >> >>> Hi Awanthika, >>> >>> We have tried this tool sometime back and as I remember it has some >>> issues with namespaces and also it generates separate files for each >>> complex datatypes (which I think you are merging and creating a single >>> object) .If we are to use this it seems we need to improve this and fix the >>> above issues. >>> >>> Thanks, >>> Sohani >>> >>> Sohani Weerasinghe >>> Software Engineer >>> WSO2, Inc: http://wso2.com >>> >>> Mobile : +94 716439774 >>> Blog :http://christinetechtips.blogspot.com/ >>> Twitter : https://twitter.com/sohanichristine >>> >>> On Thu, Feb 25, 2016 at 5:04 PM, Awanthika Senarath <[email protected]> >>> wrote: >>> >>>> Hi Susinda, >>>> >>>> I was able to generate a nested avro schema from a nested xsd schema >>>> using the tool by doing some changes to the API, as it by default generated >>>> the nested schema Json objects as separate root level Json objects being >>>> mapped to the parent Json via namespaces. >>>> >>>> But it has an issue as it does not generate the avro schema correctly >>>> when the root xsd has simple fields as children and not complex items, >>>> >>>> For an example it will correctly parse [1] while it would fail to parse >>>> [2] it will only parse the complexElement in [3] ignoring the simple >>>> elements parallel to it. >>>> >>>> [1] <xs:schema attributeFormDefault="unqualified" elementFormDefault= >>>> "qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema";> >>>> >>>> <xs:element name="pizza"> >>>> <xs:complexType> >>>> <xs:sequence> >>>> <xs:element type="xs:string" name="name"/> >>>> <xs:element type="xs:float" name="price"/> >>>> <xs:element name="toppingsList"> >>>> <xs:complexType> >>>> <xs:sequence> >>>> <xs:element name="toppings" maxOccurs="unbounded" >>>> minOccurs="0"> >>>> <xs:complexType> >>>> <xs:sequence> >>>> <xs:element type="xs:short" name="id"/> >>>> <xs:element type="xs:string" name="name"/> >>>> <xs:element type="xs:float" name="extraPrice"/> >>>> <xs:element type="xs:string" name="category"/> >>>> </xs:sequence> >>>> </xs:complexType> >>>> </xs:element> >>>> </xs:sequence> >>>> </xs:complexType> >>>> </xs:element> >>>> </xs:sequence> >>>> </xs:complexType> >>>> </xs:element></xs:schema> >>>> >>>> >>>> >>>> [2]<?xml version="1.0"?> >>>> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"; >>>> attributeFormDefault="unqualified" elementFormDefault="qualified" >>>> targetNamespace="urn:sobject.partner.soap.sforce.com"> >>>> <xs:element name="type" type="xs:string"/> >>>> <xs:element name="Id" nillable="true" type="xs:string"/> >>>> <xs:element name="Name" type="xs:string"/> >>>> <xs:element name="Amount" type="xs:float"/> >>>> </xs:schema> >>>> >>>> [3] <xs:schema attributeFormDefault="unqualified" elementFormDefault= >>>> "qualified" targetNamespace="urn:sobject.partner.soap.sforce.com" >>>> xmlns:xs="http://www.w3.org/2001/XMLSchema";> >>>> >>>> <xs:element name="type" type="xs:string"/> >>>> <xs:element name="Id" nillable="true" type="xs:string"/> >>>> <xs:element name="Account"> >>>> <xs:complexType> >>>> <xs:sequence> >>>> <xs:element type="xs:string" name="type"/> >>>> <xs:element type="xs:string" name="Id" nillable="true"/> >>>> <xs:element type="xs:string" name="Name"/> >>>> </xs:sequence> >>>> </xs:complexType> >>>> </xs:element> >>>> <xs:element name="Name" type="xs:string"/> >>>> <xs:element name="Amount" type="xs:float"/></xs:schema> >>>> >>>> >>>> >>>> >>>> Regards >>>> Awanthika >>>> >>>> >>>> Awanthika Senarath >>>> Software Engineer, WSO2 Inc. >>>> Mobile: +94717681791 >>>> >>>> >>>> >>>> On Thu, Feb 25, 2016 at 11:18 AM, Awanthika Senarath < >>>> [email protected]> wrote: >>>> >>>>> Hi Susinda, >>>>> >>>>> Currently I am evaluating approaches for generating avro schema from >>>>> XSD keeping option 1 in mind. >>>>> >>>>> [1] seems to be a reliable option, but the generated avro schema is >>>>> not in the accepted format for us. >>>>> >>>>> >>>>> [1] https://github.com/Nokia/Avro-Schema-Generator >>>>> >>>>> >>>>> Regards >>>>> Awanthika >>>>> >>>>> Awanthika Senarath >>>>> Software Engineer, WSO2 Inc. >>>>> Mobile: +94717681791 >>>>> >>>>> >>>>> >>>>> On Thu, Feb 25, 2016 at 11:04 AM, Susinda Perera <[email protected]> >>>>> wrote: >>>>> >>>>>> For Datamapper we have to give an option to do data mapping starting >>>>>> from xml payload (that user may already have). For this we need to to >>>>>> create avro schema from xml instance. Possible approaches may be >>>>>> 1. Generate xml schema(XSD) from xml and then create avro schema >>>>>> from XSD. >>>>>> 2. Directly generate avro schema by looking at xml (we have to >>>>>> code from the scratch) >>>>>> >>>>>> I believe that going with option 1 would be good, so that we can >>>>>> handle the use case of mapping from xml-schema. Also there are some >>>>>> libraries[1] and online tools[2,3] available to generate XSD from xml >>>>>> with >>>>>> some limitations. (eg represent leaf values as strings). Also from bit of >>>>>> research i found that there are several types of XSD representation >>>>>> (Venetian Blind Design, Russian Doll Design, Salami Slice Design) and we >>>>>> may need to decide on which would best fit (or easy to convert to) avro >>>>>> schema. >>>>>> If someone has done similar xml<->XSD conversion, please share your >>>>>> ideas and suggestions. >>>>>> >>>>>> Thanks >>>>>> >>>>>> >>>>>> [1] http://xmlbeans.apache.org/docs/2.0.0/guide/tools.html#xsd2inst >>>>>> >>>>>> [2] https://devutilsonline.com/xsd-xml/generate-xsd-from-xml >>>>>> >>>>>> [3]http://www.freeformatter.com/xsd-generator.html >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Susinda Perera* >>>>>> Software Engineer >>>>>> B.Sc.(Eng), M.Sc(Computer Science), AMIE(SL) >>>>>> Mobile:(+94)716049075 >>>>>> Blog: susinda.blogspot.com >>>>>> WSO2 Inc. http://wso2.com/ >>>>>> Tel : 94 11 214 5345 Fax :94 11 2145300 >>>>>> >>>>>> >>>>> >>>> >>> >> >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
