Well the separator suppression code has not had a lot of scrutiny. I wrote this a *long time* ago, and honestly have not revisited it since. I assume you figured out that separatorSuppressionPolicy replaced the separatorPolicy property. This happened after IBM released it's first DFDL product, as a result handling both the old and new property names was required.
For any of these packed numbers, if you are using delimited lengthKind, then zero-length is possible, and it means "absent", meaning that if optional, the element is not present. If required, it's an error unless zero-length triggers a nil value. If an element is both optional, and empty is a legitimate value, then I think empty->optional not present is the winner, but I have to look it up. I wasn't sure what you meant below by "....for IBM4690 and other packed binary formats the associated separators aren't processed,...". Probably best for us to talk this through on phone tomorrow (Tuesday). Look for me on the instant messenger. ________________________________ From: Joshua Adams Sent: Monday, November 27, 2017 3:59:14 PM To: Mike Beckerle; [email protected] Cc: Stephen Lawrence Subject: Issue with separatorSuppressionPolicy and empty elements in IBM4690-TLOG scheams Hey Mike, Wanted to get your opinion on the issue I've been running into with IBM4690-TLOG schemas. I talked with Steve for a while trying to figure out what was going on and we came to the opinion that there is either an issue with the TLOG schemas, or (perhaps more likely) there is an error in the separatorSuppressionPolicy code when dealing with infix separators in Daffodil. In the TlogAce.xsd file (https://github.com/DFDLSchemas/IBM4690-TLOG/blob/master/ACE/TlogAce.xsd#L155) it seems that the way the schema and data files were written assumed that the IBM4690 packed format could have a valid zero length representation, ie an optional element that doesn't occur would just be an empty string surrounded by separators. While this works just fine for strings or hex binary that have valid zero length representations, for IBM4690 and other packed binary formats the associated separators aren't processed, and in the TlogAce.xsd file, when the element SpecialTime is missing all subsequent parsed data in the sequence become CustomUserField's as that is the only element that matches the separators (I think). So, just wanted to get your opinion on whether or not this is an issue with the current Daffodil separator suppression policy code or if this is a case of an incorrectly formed schema. Steve may jump in to clarify anything I didn't explain correctly, as he is a bit more familiar with the separatorSuppression code in Daffodil. Thanks, Josh
