Hi team, I am posting this question here since it is related to my previous one. Now I am trying to figure out how to group segments in pairs (or more) of elements, in the XML output from parsing EDI.
Now, let's say that within the <Order> element (from my previous example), there can be multiple <order-info> elements, so having just one <order-info> wrapper doesn't work. In turn, each <order-info> must group segments HDR and CUS in pairs. Also, some CUS segments are optional: there can be one or none in each pair. Example EDI file: HDR+1+0+59.97+64.92+4.95+Wed Nov 15 13?:45?:28 EST 2006' HDR+1+0+59.97+64.92+4.95+Wed Nov 15 13?:45?:28 EST 2006' HDR+1+0+59.97+64.92+4.95+Wed Nov 15 13?:45?:28 EST 2006' CUS+user1+Harry:Fletcher+SD' ORD+1+1+364+The 40-Year-Old Virgin+29.98' ORD+2+1+299+Pulp Fiction+29.99' Example output (simplified): <order-info> <header/> </order-info> <order-info> <header/> </order-info> <order-info> <header/> <customer-details/> </order-info> Is it possible to do such grouping with DFDL? If it is, could you please provide some example or guidance on this? I have tried some things but I haven't managed to make it work properly yet. We are migrating some legacy code-based transformation to DFDL, and its output xml contains such wrappings and groupings, and I am trying to do it without help from xslt at the moment. Cheers PS: This is the previous schema for our simplified Order example. <?xml version="1.0" encoding="UTF-8" standalone="no"?> <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dfdl=" http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmEdiFmt=" http://www.ibm.com/dfdl/EDI/Format"> <xsd:import namespace="http://www.ibm.com/dfdl/EDI/Format" schemaLocation="IBM_EDI_Format.xsd"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format ref="ibmEdiFmt:EDIFormat"/> </xsd:appinfo> </xsd:annotation> <xsd:complexType name="OrderInfo"> <xsd:sequence dfdl:initiatedContent="yes"> <xsd:element dfdl:initiator="HDR" dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="header" type="HDR"/> <xsd:element dfdl:initiator="CUS" dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="customer-details" type="CUS"/> </xsd:sequence> </xsd:complexType> <xsd:element name="Order"> <xsd:complexType> <xsd:sequence> <xsd:element name="order-info" type="OrderInfo"/> <xsd:sequence dfdl:initiatedContent="yes"> <xsd:element dfdl:initiator="ORD" dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="order-item" maxOccurs="unbounded" type="ORD"/> </xsd:sequence> </xsd:sequence> </xsd:complexType> </xsd:element> <xsd:complexType name="HDR"> <xsd:sequence dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> <xsd:element name="order-id" type="xsd:string"/> <xsd:element name="status-code" type="xsd:string"/> <xsd:element name="net-amount" type="xsd:string"/> <xsd:element name="total-amount" type="xsd:string"/> <xsd:element name="tax" type="xsd:string"/> <xsd:element name="date" type="xsd:string"/> </xsd:sequence> </xsd:complexType> <xsd:complexType name="CUS"> <xsd:sequence dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> <xsd:element name="username" type="xsd:string"/> <xsd:element name="name"> <xsd:complexType> <xsd:sequence dfdl:ref="ibmEdiFmt:EDICompositeSequenceFormat"> <xsd:element name="firstname" type="xsd:string"/> <xsd:element name="lastname" type="xsd:string"/> </xsd:sequence> </xsd:complexType> </xsd:element> <xsd:element name="state" type="xsd:string"/> </xsd:sequence> </xsd:complexType> <xsd:complexType name="ORD"> <xsd:sequence dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> <xsd:element name="position" type="xsd:string"/> <xsd:element name="quantity" type="xsd:string"/> <xsd:element name="product-id" type="xsd:string"/> <xsd:element name="title" type="xsd:string"/> <xsd:element name="price" type="xsd:string"/> </xsd:sequence> </xsd:complexType> </xsd:schema> On Wed, Dec 1, 2021 at 7:23 PM Nestor Fernandez < nestor.fernan...@chakray.co.uk> wrote: > You are right, I just missed that attribute. I will try to follow this > approach to build the XML, thanks a lot! > > On Wed, Dec 1, 2021 at 5:41 PM Steve Lawrence <slawre...@apache.org> > wrote: > >> Looks like when moving things around you accidentally removed the >> >> dfdl:ref="ibmEdiFmt:EDISegmentFormat" >> >> from the "order-item" element. That defines the terminator for a >> segment, so without that Daffodil didn't know when the segment ended and >> parsing broke. Adding that back gets things working for me with the >> desired nesting. >> >> - Steve >> >> On 12/1/21 11:26 AM, Nestor Fernandez wrote: >> > This is the updated schema I tried >> > >> > <?xml version="1.0" encoding="UTF-8" standalone="no"?> >> > <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema >> > <http://www.w3.org/2001/XMLSchema>" >> > xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/ >> > <http://www.ogf.org/dfdl/dfdl-1.0/>" >> > xmlns:ibmEdiFmt="http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format>"> >> > <xsd:import namespace="http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format>" >> schemaLocation="IBM_EDI_Format.xsd"/> >> > <xsd:annotation> >> > <xsd:appinfo source="http://www.ogf.org/dfdl/ < >> http://www.ogf.org/dfdl/>"> >> > <dfdl:format ref="ibmEdiFmt:EDIFormat"/> >> > </xsd:appinfo> >> > </xsd:annotation> >> > <xsd:complexType name="OrderInfo"> >> > <xsd:sequence dfdl:initiatedContent="yes"> >> > <xsd:element dfdl:initiator="HDR" >> > dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="header" type="HDR"/> >> > <xsd:element dfdl:initiator="CUS" >> > dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="customer-details" >> type="CUS"/> >> > </xsd:sequence> >> > </xsd:complexType> >> > <xsd:element name="Order"> >> > <xsd:complexType> >> > <xsd:sequence> >> > <xsd:element name="order-info" type="OrderInfo"/> >> > <xsd:sequence dfdl:initiatedContent="yes"> >> > <xsd:element dfdl:initiator="ORD" >> name="order-item" >> > maxOccurs="unbounded" type="ORD"/> >> > </xsd:sequence> >> > </xsd:sequence> >> > </xsd:complexType> >> > </xsd:element> >> > <xsd:complexType name="HDR"> >> > <xsd:sequence dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> >> > <xsd:element name="order-id" type="xsd:string"/> >> > <xsd:element name="status-code" type="xsd:string"/> >> > <xsd:element name="net-amount" type="xsd:string"/> >> > <xsd:element name="total-amount" type="xsd:string"/> >> > <xsd:element name="tax" type="xsd:string"/> >> > <xsd:element name="date" type="xsd:string"/> >> > </xsd:sequence> >> > </xsd:complexType> >> > <xsd:complexType name="CUS"> >> > <xsd:sequence dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> >> > <xsd:element name="username" type="xsd:string"/> >> > <xsd:element name="name"> >> > <xsd:complexType> >> > <xsd:sequence >> dfdl:ref="ibmEdiFmt:EDICompositeSequenceFormat"> >> > <xsd:element name="firstname" >> type="xsd:string"/> >> > <xsd:element name="lastname" >> type="xsd:string"/> >> > </xsd:sequence> >> > </xsd:complexType> >> > </xsd:element> >> > <xsd:element name="state" type="xsd:string"/> >> > </xsd:sequence> >> > </xsd:complexType> >> > <xsd:complexType name="ORD"> >> > <xsd:sequence dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> >> > <xsd:element name="position" type="xsd:string"/> >> > <xsd:element name="quantity" type="xsd:string"/> >> > <xsd:element name="product-id" type="xsd:string"/> >> > <xsd:element name="title" type="xsd:string"/> >> > <xsd:element name="price" type="xsd:string"/> >> > </xsd:sequence> >> > </xsd:complexType> >> > </xsd:schema> >> > >> > >> > On Wed, Dec 1, 2021 at 5:18 PM Steve Lawrence <slawre...@apache.org >> > <mailto:slawre...@apache.org>> wrote: >> > >> > Can you send your updated schema? >> > >> > On 12/1/21 10:58 AM, Nestor Fernandez wrote: >> > > Thank you Steve. >> > > >> > > I believe I have already tried something like that. >> > > >> > > Unless I am missing something from your response, this is what >> I get now, >> > which >> > > includes the wrapper element but it is wrong (order-items are >> not parsed >> > properly). >> > > >> > > <?xml version="1.0" encoding="UTF-8"?> >> > > <Order> >> > > <order-info> >> > > <header> >> > > <order-id>1</order-id> >> > > <status-code>0</status-code> >> > > <net-amount>59.97</net-amount> >> > > <total-amount>64.92</total-amount> >> > > <tax>4.95</tax> >> > > <date>Wed Nov 15 13:45:28 EST 2006</date> >> > > </header> >> > > <customer-details> >> > > <username>user1</username> >> > > <name> >> > > <firstname>Harry</firstname> >> > > <lastname>Fletcher</lastname> >> > > </name> >> > > <state>SD</state> >> > > </customer-details> >> > > </order-info> >> > > <order-item> >> > > <position>1</position> >> > > <quantity>1</quantity> >> > > <product-id>364</product-id> >> > > <title>The 40-Year-Old Virgin</title> >> > > <price>29.98'ORD</price> >> > > </order-item> >> > > </Order> >> > > [warning] Left over data. Consumed 1024 bit(s) with at least >> 232 bit(s) >> > remaining. >> > > Left over data (Hex) starting at byte 129 is: >> (0x2b322b312b323939...) >> > > Left over data (UTF-8) starting at byte 129 is: (+2+1+299...) >> > > >> > > The output format is right, but the EDI parsing is not. >> > > >> > > >> > > On Wed, Dec 1, 2021 at 4:19 PM Steve Lawrence < >> slawre...@apache.org >> > <mailto:slawre...@apache.org> >> > > <mailto:slawre...@apache.org <mailto:slawre...@apache.org>>> >> wrote: >> > > >> > > You should be able to add a new complex type and move >> existing elements >> > > to be children of that new type, and then reference that >> type where your >> > > children were referenced. >> > > >> > > For example, in your schema you could create a new >> OrderInfo complex >> > > type and move your "header" and "customer-details" >> elements to children >> > > of that: >> > > >> > > <xsd:complexType name="OrderInfo"> >> > > <xsd:sequence dfdl:initiatedContent="yes"> >> > > <xsd:element dfdl:initiator="HDR" >> > > dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="header" >> type="HDR"/> >> > > <xsd:element dfdl:initiator="CUS" >> > > dfdl:ref="ibmEdiFmt:EDISegmentFormat" >> name="customer-details" >> > type="CUS"/> >> > > </xsd:sequence> >> > > </xsd:complexType> >> > > >> > > Note that we also copy over the dfdl:initiatedContent >> property from the >> > > previous containing sequence. >> > > >> > > We now need to modify the Order element to reference this >> new complex >> > > type. You might think we can simply replace the header and >> customer >> > > details elements with our new type, like so: >> > > >> > > <xsd:element name="Order"> >> > > <xsd:complexType> >> > > <xsd:sequence dfdl:initiatedContent="yes"> >> > > <xsd:element name="order-info" >> type="OrderInfo" /> >> > > <xsd:element dfdl:initiator="ORD" >> name="order-item" >> > > maxOccurs="unbounded" type="ORD"/> >> > > </xsd:sequence> >> > > </xsd:complexType> >> > > </xsd:element> >> > > >> > > But this doesn't actually work because the order-info >> element doesn't >> > > have an initiator (and it shouldn't, it's just a wrapper >> that doesn't >> > > consume any data itself). So it can't be inside the >> sequence with >> > > dfdl:initiatedContent="yes". To deal with this, we just >> need to move it >> > > outside the initiated content sequence into a new sequence >> that doesn't >> > > have initiated content. So we end up with nested >> sequences, something >> > > like this: >> > > >> > > <xsd:element name="Order"> >> > > <xsd:complexType> >> > > <xsd:sequence> >> > > <xsd:element name="order-info" >> type="OrderInfo" /> >> > > <xsd:sequence >> dfdl:initiatedContent="yes"> >> > > <xsd:element dfdl:initiator="ORD" >> > name="order-item" >> > > maxOccurs="unbounded" type="ORD"/> >> > > </xsd:sequence> >> > > </xsd:sequence> >> > > </xsd:complexType> >> > > </xsd:element> >> > > >> > > With those two changes, this should add a new order-info >> wrapper element >> > > around the header and customer-details elements like in >> your desired >> > XML. >> > > >> > > And this same pattern should work for most cases where you >> want a new >> > > wrapper element. Create a new complex type, move elements >> into that >> > > type, and reference that new type where your elements used >> to be. This >> > > gets a bit more complicated if the sequence containing the >> elements had >> > > properties that shouldn't apply to the new complex type, >> but in that >> > > case you can probably just move the new complex reference >> outside that >> > > sequence. >> > > >> > > >> > > >> > > On 12/1/21 9:36 AM, Nestor Fernandez wrote: >> > > > Hi team, >> > > > >> > > > We have a requirement that some of the resulting XML >> elements after >> > > parsing an >> > > > EDIFACT file should be nested or wrapped by some >> grouping element. >> > > > >> > > > Given an edi file with a flat structure like this: >> > > > >> > > > segment1 >> > > > segment2 >> > > > segment3 >> > > > segment4 >> > > > >> > > > our final goal would be to get an xml file with a more >> complex >> > structure >> > > such as >> > > > this: >> > > > >> > > > <xml> >> > > > <segment1/> >> > > > <segment-wrapper> >> > > > <segment-wrapper> >> > > > <segment2/> >> > > > <segment3/> >> > > > </segment-wrapper> >> > > > <segment4/> >> > > > </segment-wrapper> >> > > > </xml> >> > > > >> > > > I know we could just use XSLT, but is it even possible >> just with >> > Daffodil >> > > > libraries? I am new to DFDL, and thought this would be >> very >> > simple, but all >> > > > things I tried have failed (either XSD is not valid, or >> I get >> > daffodil >> > > runtime >> > > > errors). I am using apache-daffodil-3.1.0 command line >> interface. >> > > > >> > > > >> > > > I put together a more practical example: >> > > > >> > > > INPUT EDI file: >> > > > >> > > > HDR+1+0+59.97+64.92+4.95+Wed Nov 15 13?:45?:28 EST >> > > > 2006'CUS+user1+Harry:Fletcher+SD'ORD+1+1+364+The >> 40-Year-Old >> > > > Virgin+29.98'ORD+2+1+299+Pulp Fiction+29.99' >> > > > >> > > > >> > > > DFDL XSD Schema file: >> > > > >> > > > <?xml version="1.0" encoding="UTF-8" standalone="no"?> >> > > > <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema >> > <http://www.w3.org/2001/XMLSchema> >> > > <http://www.w3.org/2001/XMLSchema < >> http://www.w3.org/2001/XMLSchema>> >> > > > <http://www.w3.org/2001/XMLSchema >> > <http://www.w3.org/2001/XMLSchema> < >> http://www.w3.org/2001/XMLSchema >> > <http://www.w3.org/2001/XMLSchema>>>" >> > > > xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/ >> > <http://www.ogf.org/dfdl/dfdl-1.0/> >> > > <http://www.ogf.org/dfdl/dfdl-1.0/ < >> http://www.ogf.org/dfdl/dfdl-1.0/>> >> > > > <http://www.ogf.org/dfdl/dfdl-1.0/ >> > <http://www.ogf.org/dfdl/dfdl-1.0/> < >> http://www.ogf.org/dfdl/dfdl-1.0/ >> > <http://www.ogf.org/dfdl/dfdl-1.0/>>>" >> > > > xmlns:ibmEdiFmt="http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format> >> > > <http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format>> >> > > > <http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format> < >> http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format>>>"> >> > > > <xsd:import namespace=" >> http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format> >> > > <http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format>> >> > > > <http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format> >> > > <http://www.ibm.com/dfdl/EDI/Format >> > <http://www.ibm.com/dfdl/EDI/Format>>>" >> schemaLocation="IBM_EDI_Format.xsd"/> >> > > > <xsd:annotation> >> > > > <xsd:appinfo source="http://www.ogf.org/dfdl/ >> > <http://www.ogf.org/dfdl/> >> > > <http://www.ogf.org/dfdl/ <http://www.ogf.org/dfdl/>> >> > <http://www.ogf.org/dfdl/ <http://www.ogf.org/dfdl/> >> > > <http://www.ogf.org/dfdl/ <http://www.ogf.org/dfdl/>>>"> >> > > > <dfdl:format ref="ibmEdiFmt:EDIFormat"/> >> > > > </xsd:appinfo> >> > > > </xsd:annotation> >> > > > <xsd:element name="Order"> >> > > > <xsd:complexType> >> > > > <xsd:sequence dfdl:initiatedContent="yes"> >> > > > <xsd:element dfdl:initiator="HDR" >> > > > dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="header" >> type="HDR"/> >> > > > <xsd:element dfdl:initiator="CUS" >> > > > dfdl:ref="ibmEdiFmt:EDISegmentFormat" >> name="customer-details" >> > type="CUS"/> >> > > > <xsd:element dfdl:initiator="ORD" >> > > > dfdl:ref="ibmEdiFmt:EDISegmentFormat" name="order-item" >> > maxOccurs="unbounded" >> > > > type="ORD"/> >> > > > </xsd:sequence> >> > > > </xsd:complexType> >> > > > </xsd:element> >> > > > <xsd:complexType name="HDR"> >> > > > <xsd:sequence >> > dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> >> > > > <xsd:element name="order-id" >> type="xsd:string"/> >> > > > <xsd:element name="status-code" >> type="xsd:string"/> >> > > > <xsd:element name="net-amount" >> type="xsd:string"/> >> > > > <xsd:element name="total-amount" >> type="xsd:string"/> >> > > > <xsd:element name="tax" >> type="xsd:string"/> >> > > > <xsd:element name="date" >> type="xsd:string"/> >> > > > </xsd:sequence> >> > > > </xsd:complexType> >> > > > <xsd:complexType name="CUS"> >> > > > <xsd:sequence >> > dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> >> > > > <xsd:element name="username" >> type="xsd:string"/> >> > > > <xsd:element name="name"> >> > > > <xsd:complexType> >> > > > <xsd:sequence >> > > dfdl:ref="ibmEdiFmt:EDICompositeSequenceFormat"> >> > > > <xsd:element name="firstname" >> > type="xsd:string"/> >> > > > <xsd:element name="lastname" >> > type="xsd:string"/> >> > > > </xsd:sequence> >> > > > </xsd:complexType> >> > > > </xsd:element> >> > > > <xsd:element name="state" >> type="xsd:string"/> >> > > > </xsd:sequence> >> > > > </xsd:complexType> >> > > > <xsd:complexType name="ORD"> >> > > > <xsd:sequence >> > dfdl:ref="ibmEdiFmt:EDISegmentSequenceFormat"> >> > > > <xsd:element name="position" >> type="xsd:string"/> >> > > > <xsd:element name="quantity" >> type="xsd:string"/> >> > > > <xsd:element name="product-id" >> type="xsd:string"/> >> > > > <xsd:element name="title" >> type="xsd:string"/> >> > > > <xsd:element name="price" >> type="xsd:string"/> >> > > > </xsd:sequence> >> > > > </xsd:complexType> >> > > > </xsd:schema> >> > > > >> > > > >> > > > OUTPUT: >> > > > >> > > > <Order> >> > > > <header> >> > > > <order-id>1</order-id> >> > > > <status-code>0</status-code> >> > > > <net-amount>59.97</net-amount> >> > > > <total-amount>64.92</total-amount> >> > > > <tax>4.95</tax> >> > > > <date>Wed Nov 15 13:45:28 EST 2006</date> >> > > > </header> >> > > > <customer-details> >> > > > <username>user1</username> >> > > > <name> >> > > > <firstname>Harry</firstname> >> > > > <lastname>Fletcher</lastname> >> > > > </name> >> > > > <state>SD</state> >> > > > </customer-details> >> > > > <order-item> >> > > > <position>1</position> >> > > > <quantity>1</quantity> >> > > > <product-id>364</product-id> >> > > > <title>The 40-Year-Old Virgin</title> >> > > > <price>29.98</price> >> > > > </order-item> >> > > > <order-item> >> > > > <position>2</position> >> > > > <quantity>1</quantity> >> > > > <product-id>299</product-id> >> > > > <title>Pulp Fiction</title> >> > > > <price>29.99</price> >> > > > </order-item> >> > > > </Order> >> > > > >> > > > >> > > > Let's say we want to nest XML elements (originally EDI >> segments), >> > header and >> > > > custom-details, and get something similar to this: >> > > > >> > > > EXAMPLE OF DESIRED OUTPUT: >> > > > >> > > > <Order> >> > > > <order-info> >> > > > <header> >> > > > <order-id>1</order-id> >> > > > <status-code>0</status-code> >> > > > <net-amount>59.97</net-amount> >> > > > <total-amount>64.92</total-amount> >> > > > <tax>4.95</tax> >> > > > <date>Wed Nov 15 13:45:28 EST 2006</date> >> > > > </header> >> > > > <customer-details> >> > > > <username>user1</username> >> > > > <name> >> > > > <firstname>Harry</firstname> >> > > > <lastname>Fletcher</lastname> >> > > > </name> >> > > > <state>SD</state> >> > > > </customer-details> >> > > > </order-info> >> > > > <order-item> >> > > > <position>1</position> >> > > > <quantity>1</quantity> >> > > > <product-id>364</product-id> >> > > > <title>The 40-Year-Old Virgin</title> >> > > > <price>29.98</price> >> > > > </order-item> >> > > > <order-item> >> > > > <position>2</position> >> > > > <quantity>1</quantity> >> > > > <product-id>299</product-id> >> > > > <title>Pulp Fiction</title> >> > > > <price>29.99</price> >> > > > </order-item> >> > > > </Order> >> > > > >> > > > >> > > > Any ideas would be appreciated. >> > > > >> > > > Thanks and regards >> > > > >> > > >> > >> >>