[
https://issues.apache.org/jira/browse/DAFFODIL-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486507#comment-17486507
]
Mike Beckerle commented on DAFFODIL-2626:
-----------------------------------------
Global group references DO combine properties from reference site with those on
the referenced group.
Only complex type references do not. combine the properties.
This design note
[https://daffodil.apache.org/dev/design-notes/term-sharing-in-schema-compiler/]
talks about this whole issue of shared terms versus unique aspects of a term. I
have to review it just to think about this stuff again.
There seem to be two problems here.
One is that the schema compiler is not proving that the alignmentFill region is
not required. This has nothing to do with sharing the global definition,
because the alignmentFill region is part of the "uniqueBefore" region (per
above page). But it seems we are not keeping track of the abstract bit position
(at schema compile time) sufficiently to prove the alignment will in fact
always be a multiple of 8 in this case.
For a root element, we should be starting with absolute knowledge that we are
at bitPos0b 0, so perfectly aligned. Someplace we're not properly abstractly
adding the various widths of elements/terms, and we're losing track of the fact
that the widths add up to a byte boundary. Likely as soon as one adds a
bits-length element that is not a mulitple of 8 wide, the abstract addition at
that point makes the pessimistic assumption that you are on a bit boundary, and
then adding more bits to it doesn't recover byte alignment ever. Fixing this
so that we really are able to recover when the various bit fields lengths add
up to a multiple of 8 again seems pretty tricky.
So I'm actually kind of pessimistic that we can optimize out this alignment.
One way to optimize it out would be to use 1-bit alignment for everything, and
introduce a 1-bit aligned character set, so that the mandatory text alignment
regions also evaporate.
Or perhaps a tunable/property to say that the encoding should be treated as 1
bit aligned.
Or a tunable or property allowing one to state that the schema is handling all
alignment manually, and that the schema compiler should just not insert any
alignmentFill or mandatory text alignments at all.
These seem heavy handed, but alignment problems are a thorny area in DFDL, and
I rather wish we had not put the alignment stuff into DFDL, forcing DFDL
schemas to be explicit about alignment regions with leadingSkip, trailingSkip,
etc.
Onto the second problem:
The second problem is this. even with an alignmentFill region, the prior stuff
is not variable-width unknown. It is fixed width stuff.
Hence, that information should propagate at runtime, the alignmentFill region
should at some point (on a retry of the suspension perhaps) detect that the
starting position is at a multiple of 8, and therefore it's length is 0,
freeing the deadlock.
This second problem is more fundamental. The runtime is somehow not propagating
the position information for the buffered data output streams as it is figured
out. Upon some evaluation of that alignmentFill suspension, it should look at
the underlying buffered data output stream, and see that its starting position
is known, and so that alignmentFill can compute how big it is (which should be
zero in this case).
The algorithms here are really snarly though. I believe it navigates backward
recursively through the chain of buffered data output streams, and pulls
forward information about length, and starting position of each buffered data
output stream. Somehow this is not finding the information and propagating it
forward.
> Circular deadlock when computing stored length around prefixed-length elements
> ------------------------------------------------------------------------------
>
> Key: DAFFODIL-2626
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2626
> Project: Daffodil
> Issue Type: Bug
> Components: Back End
> Affects Versions: 3.2.1
> Reporter: Mike Beckerle
> Assignee: Steve Lawrence
> Priority: Critical
>
> I have a small schema with a messageLength element as the first element,
> which has outputValueCalc on the message payload.
> The message payload contains prefixed-length strings.
> I get a deadlock on unparsing:
> {code}
> org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodil)
> SuspensionDeadlockException: Runtime Schema Definition Error:
> Expressions/Unparsers are circularly deadlocked (mutually defined):
> - SimpleTypeRetryUnparserSuspendableOperation for messageLength
> - ElementUnusedUnparserSuspendableOperation for payload
> - AlignmentFillUnparserSuspendableOperation for address
> Schema context: messageLength Location line 59 column 18 in
> file:/tmp/s__3044777217024912267.dfdl.xsd
> Data location was preceding byte 0
> {code}
> See test test_computedLengthAroundPrefixedLengths1u
> Test test_computedLengthAroundPrefixedLengths1p is the same thing, just
> testing the parse direction.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
