Hi all, Is there any particular reason as to why we are sticking into avro schema for the datamapper implementation?
AFAIS the only tool (open/non-open source) that exist for XSD to avro conversion, is the same that has been used by the team when we attempted the data mapper first 2 years back. This generates JAVA model from XSD schema using JAXB and generate the avro schema from the generated JAVA model. This has drawbacks : 1. Unable to gen avro for element children that are not wrapped by complex-type, 2. Unable to identify namespaces 3. Unable to generate single avro for a nested XSD, which I fixed by inserting the separate JSON objects generated accordingly to get the final nested avro schema. for (1) we need to manipulate the xsd give an insert complex-type tags. (2) cannot be supported with JAXB based generator, need to evaluate optional approach for this. meanwhile I am evaluating [2] as well. [2] https://github.com/elodina/xml-avro Regards Awanthika Awanthika Senarath Software Engineer, WSO2 Inc. Mobile: +94717681791 On Thu, Feb 25, 2016 at 5:42 PM, Sohani Weerasinghe <soh...@wso2.com> wrote: > Hi Awanthika, > > We have tried this tool sometime back and as I remember it has some issues > with namespaces and also it generates separate files for each complex > datatypes (which I think you are merging and creating a single object) .If > we are to use this it seems we need to improve this and fix the above > issues. > > Thanks, > Sohani > > Sohani Weerasinghe > Software Engineer > WSO2, Inc: http://wso2.com > > Mobile : +94 716439774 > Blog :http://christinetechtips.blogspot.com/ > Twitter : https://twitter.com/sohanichristine > > On Thu, Feb 25, 2016 at 5:04 PM, Awanthika Senarath <awanth...@wso2.com> > wrote: > >> Hi Susinda, >> >> I was able to generate a nested avro schema from a nested xsd schema >> using the tool by doing some changes to the API, as it by default generated >> the nested schema Json objects as separate root level Json objects being >> mapped to the parent Json via namespaces. >> >> But it has an issue as it does not generate the avro schema correctly >> when the root xsd has simple fields as children and not complex items, >> >> For an example it will correctly parse [1] while it would fail to parse >> [2] it will only parse the complexElement in [3] ignoring the simple >> elements parallel to it. >> >> [1] <xs:schema attributeFormDefault="unqualified" elementFormDefault= >> "qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema";> >> >> <xs:element name="pizza"> >> <xs:complexType> >> <xs:sequence> >> <xs:element type="xs:string" name="name"/> >> <xs:element type="xs:float" name="price"/> >> <xs:element name="toppingsList"> >> <xs:complexType> >> <xs:sequence> >> <xs:element name="toppings" maxOccurs="unbounded" >> minOccurs="0"> >> <xs:complexType> >> <xs:sequence> >> <xs:element type="xs:short" name="id"/> >> <xs:element type="xs:string" name="name"/> >> <xs:element type="xs:float" name="extraPrice"/> >> <xs:element type="xs:string" name="category"/> >> </xs:sequence> >> </xs:complexType> >> </xs:element> >> </xs:sequence> >> </xs:complexType> >> </xs:element> >> </xs:sequence> >> </xs:complexType> >> </xs:element></xs:schema> >> >> >> >> [2]<?xml version="1.0"?> >> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"; >> attributeFormDefault="unqualified" elementFormDefault="qualified" >> targetNamespace="urn:sobject.partner.soap.sforce.com"> >> <xs:element name="type" type="xs:string"/> >> <xs:element name="Id" nillable="true" type="xs:string"/> >> <xs:element name="Name" type="xs:string"/> >> <xs:element name="Amount" type="xs:float"/> >> </xs:schema> >> >> [3] <xs:schema attributeFormDefault="unqualified" elementFormDefault= >> "qualified" targetNamespace="urn:sobject.partner.soap.sforce.com" >> xmlns:xs="http://www.w3.org/2001/XMLSchema";> >> >> <xs:element name="type" type="xs:string"/> >> <xs:element name="Id" nillable="true" type="xs:string"/> >> <xs:element name="Account"> >> <xs:complexType> >> <xs:sequence> >> <xs:element type="xs:string" name="type"/> >> <xs:element type="xs:string" name="Id" nillable="true"/> >> <xs:element type="xs:string" name="Name"/> >> </xs:sequence> >> </xs:complexType> >> </xs:element> >> <xs:element name="Name" type="xs:string"/> >> <xs:element name="Amount" type="xs:float"/></xs:schema> >> >> >> >> >> Regards >> Awanthika >> >> >> Awanthika Senarath >> Software Engineer, WSO2 Inc. >> Mobile: +94717681791 >> >> >> >> On Thu, Feb 25, 2016 at 11:18 AM, Awanthika Senarath <awanth...@wso2.com> >> wrote: >> >>> Hi Susinda, >>> >>> Currently I am evaluating approaches for generating avro schema from XSD >>> keeping option 1 in mind. >>> >>> [1] seems to be a reliable option, but the generated avro schema is not >>> in the accepted format for us. >>> >>> >>> [1] https://github.com/Nokia/Avro-Schema-Generator >>> >>> >>> Regards >>> Awanthika >>> >>> Awanthika Senarath >>> Software Engineer, WSO2 Inc. >>> Mobile: +94717681791 >>> >>> >>> >>> On Thu, Feb 25, 2016 at 11:04 AM, Susinda Perera <susi...@wso2.com> >>> wrote: >>> >>>> For Datamapper we have to give an option to do data mapping starting >>>> from xml payload (that user may already have). For this we need to to >>>> create avro schema from xml instance. Possible approaches may be >>>> 1. Generate xml schema(XSD) from xml and then create avro schema from >>>> XSD. >>>> 2. Directly generate avro schema by looking at xml (we have to code >>>> from the scratch) >>>> >>>> I believe that going with option 1 would be good, so that we can handle >>>> the use case of mapping from xml-schema. Also there are some libraries[1] >>>> and online tools[2,3] available to generate XSD from xml with some >>>> limitations. (eg represent leaf values as strings). Also from bit of >>>> research i found that there are several types of XSD representation >>>> (Venetian Blind Design, Russian Doll Design, Salami Slice Design) and we >>>> may need to decide on which would best fit (or easy to convert to) avro >>>> schema. >>>> If someone has done similar xml<->XSD conversion, please share your >>>> ideas and suggestions. >>>> >>>> Thanks >>>> >>>> >>>> [1] http://xmlbeans.apache.org/docs/2.0.0/guide/tools.html#xsd2inst >>>> >>>> [2] https://devutilsonline.com/xsd-xml/generate-xsd-from-xml >>>> >>>> [3]http://www.freeformatter.com/xsd-generator.html >>>> >>>> >>>> >>>> -- >>>> *Susinda Perera* >>>> Software Engineer >>>> B.Sc.(Eng), M.Sc(Computer Science), AMIE(SL) >>>> Mobile:(+94)716049075 >>>> Blog: susinda.blogspot.com >>>> WSO2 Inc. http://wso2.com/ >>>> Tel : 94 11 214 5345 Fax :94 11 2145300 >>>> >>>> >>> >> >
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
