The first portion of one of these Signal PDUs is in BE+MSBF order up to and including a header for the LINK-16 portion. Then there is the BE+LSBF data which is a JTIDS header followed by the contents of the J message.
The pre/post processing feature does seem like a better hammer for the problem. I believe we could contribute a pre/post processor that can handle swapping. Do you have an idea on when this new feature will start to be implemented? On Thu, Jan 11, 2018 at 8:11 PM, Mike Beckerle <[email protected]> wrote: > The byte-order swapping sounds right Marshall. > > > One of the features for DFDL/Daffodil for which there is much demand is a > feature that enables pre-processing passes like what you are describing. > Things like base64 encoding, line-folding, byte-order/word swapping, > decompress, are all things people need to be able to do on the underlying > data stream; not necessarily of the entire data stream, but on just the > section of the data stream holding the representation of a specific element > within a greater format. E.g., VMF messages are often carried as a compressed > payload within a MIL-STD-2045 envelope. Internet messages have attachments > that are base64 encoded commonly, etc etc. > > > In your case, this word/byte swapping would be applicable to the SISO-J > headers, but would it apply to the Link16 data being carried by it? If it > applies only to the SISO-J packaging part, then you'd need this feature. > > > The current proposal for this feature (which I'm calling the "layering" or > "base64" feature) was actually circulated on this mailing list last week for > initial feedback. Implementing this feature in Daffodil is within our 6 month > roadmap/plan. > > > I like this better than trying to come up with an endless proliferation of > word, byte, and bit order properties, the interactions of which are not very > orthogonal. > > > As for the way this or any other feature gets incorporated into the DFDL > language spec. Basically the cycle is 1) propose and discuss feature 2) > prototype feature in Daffodil 3) get experience with feature 4) officially > propose for inclusion in DFDL. That's if you are working with Daffodil. If > you worked for IBM you would perhaps prototype in their implementation. > > > If we implement base64 and/or compress maybe we could get you to develop the > word-swapping pre/post processor which would would work analogously in > Daffodil? > > ________________________________ > From: Marshall Miller <[email protected]> > Sent: Thursday, January 11, 2018 5:18:26 PM > To: [email protected] > Subject: Re: Help implementing schema for SISO-J > > Also, it occurred to me that it might be possible to perform a stream > transformation on the data before it is parsed. If each 32 bit word > were byte-swapped before parsing fields, then I believe the fields > could be properly parsed in littleEndian byte order with LSBF > bitOrder. > > On Thu, Jan 11, 2018 at 4:50 PM, Marshall Miller > <[email protected]> wrote: >> Mike, >> >> Thanks for the helpful and quick response. I was worried we might >> have to extract the fields with mathematical operations as you >> described. >> >> Hopefully the DFDL Workgroup will add support for this type of packing >> so that LINK-16 schemas can be reused in SISO-J messages. When does >> the workgroup meet next? What do you think the likelihood is for a >> change like this to be accepted? >> >> >> Marshall Miller >> >> On Thu, Jan 11, 2018 at 3:48 PM, Mike Beckerle <[email protected]> wrote: >>> Marshall, >>> >>> >>> Looks like we'll have to give some thought to whether bigEndian and LSBF >>> can go together. We've had no actual examples of this until now. I will >>> bring that up with the OGF DFDL Workgroup. >>> >>> >>> But to deal with this today, these kinds of mixed bit order situations >>> generally require use of the dfdl:inputValueCalc and dfdl:outputValueCalc >>> properties. >>> >>> >>> A simple example is here: >>> >>> >>> https://github.com/rchampag/dfdl/blob/master/arinc429-label/localizer-deviation-label.dfdl.xsd >>> >>> >>> This was for individual bits, but in your case they would be wider integers. >>> >>> >>> For your 3 fields in the example I suggest you parse as 4 fields >>> >>> >>> h_slot2LSBs - 20 bits >>> >>> h_slot1 - 12 bits >>> >>> h_slot3 - 20 bits >>> >>> h_slot2MSBs - 12 bits >>> >>> >>> You would do this all bigEndian and mostSignificantBitFirst. >>> >>> >>> The above would go into a hidden group so they wouldn't show up in the >>> logical data. >>> >>> >>> Then you would have 3 logical ordinary, not hidden elements >>> >>> slot1 dfdl:inputValueCalc='{ ../h_slot1 }' >>> >>> slot2 dfdl:inputValueCalc='{ ( ../h_slot2MSBs * dfdl:int('x80000' ) + >>> ../h_slot2LSBs }' >>> >>> slot3 dfdl:inputValueCalc='{ ../h_slot3 }' >>> >>> >>> dfdl:int('x80000') constructs a integer from a hex string, and I think >>> that's the right value. Or maybe it's 'x100000'. >>> >>> >>> You can also use math.pow function as in >>> >>> >>> xs:int(math:pow(2, 19)) >>> >>> >>> (you need to add this namespace to your xs:schema element >>> xmlns:math='http://www.w3.org/2005/xpath-functions/math') >>> >>> >>> The tricky part is that for unparsing you must invert this using >>> dfdl:outputValueCalc. So the 4 hidden elements would be like this: >>> >>> >>> h_slot2LSBs dfdl:outputValueCalc="{ ../slot2 mod dfdl:int('x80000') }" >>> >>> h_slot1 dfdl:outputValueCalc='{ ../slot1 }' >>> >>> h_slot3 dfdl:outputValueCalc='{ ../slot3 }' >>> >>> h_slot2MSBs dfdl:outputValueCalc="{ ../slot2 idiv dfdl:int('x80000') }" >>> >>> >>> or something like that. My details here are probably off by 1 or something. >>> >>> >>> DFDL's Xpath-like expression language has the operators +, -, *, div, idiv, >>> mod, and the math.pow() function. Alas there's no shift nor a bit-wise & >>> and | operator. Generally it is possible to figure out how to achieve what >>> you want by way of mathematical operations. >>> >>> >>> Also note: If you want Link16 specific support we have a channel for FOUO >>> email support which is [email protected]. >>> >>> >>> ...mike beckerle >>> >>> Tresys >>> >>> >>> >>> ________________________________ >>> From: Marshall Miller <[email protected]> >>> Sent: Thursday, January 11, 2018 2:34:46 PM >>> To: [email protected] >>> Subject: Help implementing schema for SISO-J >>> >>> Hello, >>> >>> I am trying to implement a schema to describe a LINK-16 message >>> embedded within a DIS Signal PDU as described by SISO-STD-002 [1], >>> paragraph 5.2.2.9 and table 5.2.3; commonly referred to as SISO-J. >>> The problem is that fields are packed into 32 bit words with big >>> endian byte order, and little endian bitfield order. It is currently >>> an error to specify a byteOrder of bigEndian with a bitOrder of >>> leastSignificantBitFirst. Also, I have not found a way to set word >>> size; it appears that word size is implicitly set to the size of each >>> field. >>> >>> Following is a simple example to demonstrate how fields need to be >>> interpreted. It is just an example and does not represent any part of >>> a SISO-J message. >>> >>> Data is comprised of 3 unsigned integer bitfields: >>> slot1 is 12 bits >>> slot2 is 32 bits >>> slot3 is 20 bits >>> >>> Given an input with the following bytes in hex: >>> 01 23 45 67 89 AB CD EF >>> >>> Parsed results need to be (in hex): >>> slot1: 0567 >>> slot2: DEF01234 >>> slot3: 089ABC >>> >>> Explanation of results: >>> >>> Since the bitfields need to consume data in a leastSignificantBitFirst >>> order, it is easier to represent the input in right-to-left order >>> where bits are consumed starting at the right and moving toward the >>> left. Since words are 4 bytes, and in big endian order, the lowest >>> bit in the 4th byte is the first bit to be consumed. Reordering the >>> input data and consuming data from right to left yields the correct >>> field boundaries. >>> >>> 89 AB CD EF 01 23 45 67 parse fields right to left >>> 05 06 07 08 01 02 03 04 input byte position >>> >>> slot1 consumes the right 3 nibbles, slot2 consumes the next 8 nibbles, >>> and slot3 consumes the left 5 nibbles. >>> >>> Could you please let me know if there is a way to parse fields in this >>> manner? >>> >>> >>> >>> Marshall Miller >>> >>> >>> [1] >>> https://www.sisostds.org/DesktopModules/Bring2mind/DMX/Download.aspx?Command=Core_Download&EntryId=43048&PortalId=0&TabId=105
