Hi All, According to [1] since xml namespace is not equivalent to avro namespace, if we are to have different namespaces in XML the suggested approach in [1] is to use doc tag in avro to represent the namespaces.
But if we are going to use that, AFAIU we should include namespace tag as well in the avro schema to differentiate each Record. [1] https://docs.google.com/spreadsheets/d/15eyRuxn9rFazULASqtB2DZaSAVznKqSNnIGrwvdpzic/edit?ts=56d5250b#gid=0 Sohani Weerasinghe Software Engineer WSO2, Inc: http://wso2.com Mobile : +94 716439774 Blog :http://christinetechtips.blogspot.com/ Twitter : https://twitter.com/sohanichristine On Tue, Mar 1, 2016 at 1:26 PM, Awanthika Senarath <[email protected]> wrote: > Hello All, > > > > According to the requirement for supporting multiple namespaces in XML > document in [1] according to [2] it is not possible by the single XSD > schema approach we are taking ATM. > > Currently our approach to generate avro schema from XML is, > > XML ---> XSD schema ---> Avro schema. > > If we are to support multiple namespaces in the XML doc we should either > generate multiple cascaded XSD schemas as explained in [2] or perform > rigorous manipulation of the generated avro schema. IMO this is not a very > good approach, > > Further generating avro schema object per xml complex type to support > multiple namespaces will introduce additional overhead and lengthen the > resulting avro schema. > > WDYT? > > > [1] > https://docs.google.com/spreadsheets/d/15eyRuxn9rFazULASqtB2DZaSAVznKqSNnIGrwvdpzic/edit?ts=56d5250b#gid=0 > > [2] > http://stackoverflow.com/questions/9805359/how-to-handle-multiple-namespaces-with-different-uri-in-xsd > > Regards > Awanthika > > Awanthika Senarath > Software Engineer, WSO2 Inc. > Mobile: +94717681791 > > > > On Fri, Feb 26, 2016 at 4:01 PM, Awanthika Senarath <[email protected]> > wrote: > >> Hi all, >> >> This feature is implemented using the nokia avro scheme gen tol with >> apache license[1]. It was modified to remove unnecessary class gen codes >> with JAXB and also to manipulate the generated avro to insert child JSON >> objects. >> >> I will insert this source into our datamapper tool to enable it within >> dev-studio. >> >> >> [1] https://github.com/Nokia/Avro-Schema-Generator >> >> >> >> Awanthika Senarath >> Software Engineer, WSO2 Inc. >> Mobile: +94717681791 >> >> >> >> On Fri, Feb 26, 2016 at 10:16 AM, Awanthika Senarath <[email protected]> >> wrote: >> >>> Hi all, >>> >>> Is there any particular reason as to why we are sticking into avro >>> schema for the datamapper implementation? >>> >>> AFAIS the only tool (open/non-open source) that exist for XSD to avro >>> conversion, is the same that has been used by the team when we attempted >>> the data mapper first 2 years back. >>> >>> This generates JAVA model from XSD schema using JAXB and generate the >>> avro schema from the generated JAVA model. >>> >>> This has drawbacks : >>> >>> 1. Unable to gen avro for element children that are not wrapped by >>> complex-type, >>> 2. Unable to identify namespaces >>> 3. Unable to generate single avro for a nested XSD, which I fixed by >>> inserting the separate JSON objects generated accordingly to get the final >>> nested avro schema. >>> >>> >>> for (1) we need to manipulate the xsd give an insert complex-type tags. >>> (2) cannot be supported with JAXB based generator, need to evaluate >>> optional approach for this. >>> >>> >>> >>> meanwhile I am evaluating [2] as well. >>> >>> >>> [2] https://github.com/elodina/xml-avro >>> >>> Regards >>> Awanthika >>> >>> >>> >>> >>> Awanthika Senarath >>> Software Engineer, WSO2 Inc. >>> Mobile: +94717681791 >>> >>> >>> >>> On Thu, Feb 25, 2016 at 5:42 PM, Sohani Weerasinghe <[email protected]> >>> wrote: >>> >>>> Hi Awanthika, >>>> >>>> We have tried this tool sometime back and as I remember it has some >>>> issues with namespaces and also it generates separate files for each >>>> complex datatypes (which I think you are merging and creating a single >>>> object) .If we are to use this it seems we need to improve this and fix the >>>> above issues. >>>> >>>> Thanks, >>>> Sohani >>>> >>>> Sohani Weerasinghe >>>> Software Engineer >>>> WSO2, Inc: http://wso2.com >>>> >>>> Mobile : +94 716439774 >>>> Blog :http://christinetechtips.blogspot.com/ >>>> Twitter : https://twitter.com/sohanichristine >>>> >>>> On Thu, Feb 25, 2016 at 5:04 PM, Awanthika Senarath <[email protected] >>>> > wrote: >>>> >>>>> Hi Susinda, >>>>> >>>>> I was able to generate a nested avro schema from a nested xsd schema >>>>> using the tool by doing some changes to the API, as it by default >>>>> generated >>>>> the nested schema Json objects as separate root level Json objects being >>>>> mapped to the parent Json via namespaces. >>>>> >>>>> But it has an issue as it does not generate the avro schema correctly >>>>> when the root xsd has simple fields as children and not complex items, >>>>> >>>>> For an example it will correctly parse [1] while it would fail to >>>>> parse [2] it will only parse the complexElement in [3] ignoring the >>>>> simple >>>>> elements parallel to it. >>>>> >>>>> [1] <xs:schema attributeFormDefault="unqualified" elementFormDefault= >>>>> "qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema";> >>>>> >>>>> <xs:element name="pizza"> >>>>> <xs:complexType> >>>>> <xs:sequence> >>>>> <xs:element type="xs:string" name="name"/> >>>>> <xs:element type="xs:float" name="price"/> >>>>> <xs:element name="toppingsList"> >>>>> <xs:complexType> >>>>> <xs:sequence> >>>>> <xs:element name="toppings" maxOccurs="unbounded" >>>>> minOccurs="0"> >>>>> <xs:complexType> >>>>> <xs:sequence> >>>>> <xs:element type="xs:short" name="id"/> >>>>> <xs:element type="xs:string" name="name"/> >>>>> <xs:element type="xs:float" name="extraPrice"/> >>>>> <xs:element type="xs:string" name="category"/> >>>>> </xs:sequence> >>>>> </xs:complexType> >>>>> </xs:element> >>>>> </xs:sequence> >>>>> </xs:complexType> >>>>> </xs:element> >>>>> </xs:sequence> >>>>> </xs:complexType> >>>>> </xs:element></xs:schema> >>>>> >>>>> >>>>> >>>>> [2]<?xml version="1.0"?> >>>>> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"; >>>>> attributeFormDefault="unqualified" elementFormDefault="qualified" >>>>> targetNamespace="urn:sobject.partner.soap.sforce.com"> >>>>> <xs:element name="type" type="xs:string"/> >>>>> <xs:element name="Id" nillable="true" type="xs:string"/> >>>>> <xs:element name="Name" type="xs:string"/> >>>>> <xs:element name="Amount" type="xs:float"/> >>>>> </xs:schema> >>>>> >>>>> [3] <xs:schema attributeFormDefault="unqualified" elementFormDefault= >>>>> "qualified" targetNamespace="urn:sobject.partner.soap.sforce.com" >>>>> xmlns:xs="http://www.w3.org/2001/XMLSchema";> >>>>> >>>>> <xs:element name="type" type="xs:string"/> >>>>> <xs:element name="Id" nillable="true" type="xs:string"/> >>>>> <xs:element name="Account"> >>>>> <xs:complexType> >>>>> <xs:sequence> >>>>> <xs:element type="xs:string" name="type"/> >>>>> <xs:element type="xs:string" name="Id" nillable="true"/> >>>>> <xs:element type="xs:string" name="Name"/> >>>>> </xs:sequence> >>>>> </xs:complexType> >>>>> </xs:element> >>>>> <xs:element name="Name" type="xs:string"/> >>>>> <xs:element name="Amount" type="xs:float"/></xs:schema> >>>>> >>>>> >>>>> >>>>> >>>>> Regards >>>>> Awanthika >>>>> >>>>> >>>>> Awanthika Senarath >>>>> Software Engineer, WSO2 Inc. >>>>> Mobile: +94717681791 >>>>> >>>>> >>>>> >>>>> On Thu, Feb 25, 2016 at 11:18 AM, Awanthika Senarath < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Susinda, >>>>>> >>>>>> Currently I am evaluating approaches for generating avro schema from >>>>>> XSD keeping option 1 in mind. >>>>>> >>>>>> [1] seems to be a reliable option, but the generated avro schema is >>>>>> not in the accepted format for us. >>>>>> >>>>>> >>>>>> [1] https://github.com/Nokia/Avro-Schema-Generator >>>>>> >>>>>> >>>>>> Regards >>>>>> Awanthika >>>>>> >>>>>> Awanthika Senarath >>>>>> Software Engineer, WSO2 Inc. >>>>>> Mobile: +94717681791 >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Feb 25, 2016 at 11:04 AM, Susinda Perera <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> For Datamapper we have to give an option to do data mapping starting >>>>>>> from xml payload (that user may already have). For this we need to to >>>>>>> create avro schema from xml instance. Possible approaches may be >>>>>>> 1. Generate xml schema(XSD) from xml and then create avro schema >>>>>>> from XSD. >>>>>>> 2. Directly generate avro schema by looking at xml (we have to >>>>>>> code from the scratch) >>>>>>> >>>>>>> I believe that going with option 1 would be good, so that we can >>>>>>> handle the use case of mapping from xml-schema. Also there are some >>>>>>> libraries[1] and online tools[2,3] available to generate XSD from xml >>>>>>> with >>>>>>> some limitations. (eg represent leaf values as strings). Also from bit >>>>>>> of >>>>>>> research i found that there are several types of XSD representation >>>>>>> (Venetian Blind Design, Russian Doll Design, Salami Slice Design) and we >>>>>>> may need to decide on which would best fit (or easy to convert to) avro >>>>>>> schema. >>>>>>> If someone has done similar xml<->XSD conversion, please share your >>>>>>> ideas and suggestions. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> [1] http://xmlbeans.apache.org/docs/2.0.0/guide/tools.html#xsd2inst >>>>>>> >>>>>>> [2] https://devutilsonline.com/xsd-xml/generate-xsd-from-xml >>>>>>> >>>>>>> [3]http://www.freeformatter.com/xsd-generator.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Susinda Perera* >>>>>>> Software Engineer >>>>>>> B.Sc.(Eng), M.Sc(Computer Science), AMIE(SL) >>>>>>> Mobile:(+94)716049075 >>>>>>> Blog: susinda.blogspot.com >>>>>>> WSO2 Inc. http://wso2.com/ >>>>>>> Tel : 94 11 214 5345 Fax :94 11 2145300 >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
