Mathew, I assume you are looking at the Daffodil/DFDL/VMF_S2S on bitbucket.di2e.net
It appears that this schema has 2 elements of interest: urn:VMF_DFDL:messageSet - which computes a single set of messages. urn:VMF_DFDL:messageSets - which parses urn:VMF_DFDL:messageSet an unbounded number of times, with dfdl:occursCountKind="implicit" I suspect that you want to be using urn:VMF_DFDL:messageSet instead of urn:VMF_DFDL:messageSets. The issue is that Daffodil needs to know when it reaches the end of the schema. With urn:VMF_DFDL:messageSet, Daffodil can parse the header to know how many messages to parse. Once that number of messages has been parsed, Daffodil knows that it is finished, and has no reason to attempt to read more from the input stream. With urn:VMF_DFDL:messageSets, Daffodil will first parse a single messageSet as above. However, when that is done, Daffodil does not know if it is finished, since there is a possibility that it is supposed to parse more messageSets. What it will do is speculatively parse an additional messageSet. In the case of non-blocking IO, this speculative parse will fail, as there is no data to consume. Once this speculative parse fails, Daffodil knows there is no more messageSets, and returns the complete parse of the first one. When blocking IO is used, Daffodil again attempts to parse a second messageSet. In this case, instead of failing, Daffodil blocks on the attempt to read it. ________________________________ From: Beckerle, Mike <[email protected]> Sent: Friday, October 25, 2019 3:28 PM To: Magliocca, Matthew - 1004 - MITLL <[email protected]>; [email protected] <[email protected]> Cc: Shannon, Tod - 1004 - MITLL <[email protected]>; Ledder, William - 1004 - MITLL <[email protected]>; Chiang, Carol - 1004 - MITLL <[email protected]> Subject: Re: DFDL VMF 0.0.7 Question Forwarding to standard support email, as I am tied up on other matters today. ________________________________ From: Magliocca, Matthew - 1004 - MITLL <[email protected]> Sent: Friday, October 25, 2019, 11:33 AM To: Beckerle, Mike Cc: Shannon, Tod - 1004 - MITLL; Ledder, William - 1004 - MITLL; Chiang, Carol - 1004 - MITLL Subject: DFDL VMF 0.0.7 Question Hi Mike, I had a question for you. I’ve taken the updated 0.0.4 and 0.0.7 versions of the VMF DFDL schema and I’m trying to integration-test it against our code. Most things seem to work fine but when we use blocking input streams to supply DFDL with binary data, I’m noticing something kind of odd. When I debug our code I see that DFDL reads the full number of bytes in our binary messages but then seems to go back to the stream and ask for more. If the InputStream it is reading is nonblocking and returns a “-1” character after the message, the system works fine. However if we use blocking input streams the system seems to hang; with DFDL waiting forever for more data without ever returning a completed infoset. I was wondering if there had been a paradigm shift in DFDL dealing with blocking inputstreams in underlying code. The reason I ask if because I noticed you had added a “messageSets” element to the schema and I thought you might have been attempting to use that to group messages rather than handing them back individually. Could the changes we’ve seeing be related to any paradigm shifts in DFDL or is that unlikely? Thanks a lot Matt Mags
