This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-daffodil-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1d43545 Publishing from ed8a08ca591896c4bfe3c978b083993736215729
1d43545 is described below
commit 1d43545445da0279ba6b65b0481d337f06cf810b
Author: Mike Beckerle <[email protected]>
AuthorDate: Mon Feb 10 23:26:35 2020 +0000
Publishing from ed8a08ca591896c4bfe3c978b083993736215729
---
content/dev/aboutAsciiDoc/index.html | 56 +-
content/dev/design-notes/hidden-groups/index.html | 272 ++++++
.../namespace-binding-minimization/index.html | 763 +++++++++++++++++
.../term-sharing-in-schema-compiler/index.html | 922 +++++++++++++++++++++
.../diag-1313a13d91a6f8c668ade63f72bbb61d.png | Bin 0 -> 10901 bytes
.../diag-19f9d885d07ef92b8922b13dfaa30484.png | Bin 0 -> 2970 bytes
.../diag-45bcd5e9c7484340ba070e865b4dd7ed.png | Bin 0 -> 14195 bytes
.../diag-51b412adf272104af315f2f6958aeb2d.png | Bin 0 -> 2912 bytes
.../diag-a233f3edc65e7c991774d9233c23f186.png | Bin 0 -> 18689 bytes
.../diag-e8cdc9d803fb5e074386a0e096177241.png | Bin 0 -> 11274 bytes
content/images/test_packet_diagram_1.svg | 181 ----
content/images/unshared-shared.png | Bin 0 -> 14323 bytes
12 files changed, 1987 insertions(+), 207 deletions(-)
diff --git a/content/dev/aboutAsciiDoc/index.html
b/content/dev/aboutAsciiDoc/index.html
index 3240466..026954d 100644
--- a/content/dev/aboutAsciiDoc/index.html
+++ b/content/dev/aboutAsciiDoc/index.html
@@ -185,22 +185,6 @@ and here’s the same image as an image block:</p>
<div class="paragraph">
<p>There are numerous kinds of diagrams possible.</p>
</div>
-</div>
-</div>
-<div class="sect1">
-<h2 id="graphviz-record-based-nodes-diagram">GraphViz Record-Based Nodes
Diagram</h2>
-<div class="sectionbody">
-<div class="paragraph">
-<p><a href="https://graphviz.gitlab.io/documentation/";>GraphViz</a> is
probably the most powerful of the various diagram tools, but with that power
comes complexity that some of the other diagram types are able to overcome by
being more restrictive.</p>
-</div>
-<div class="paragraph">
-<p>This is an example of graphViz <a
href="https://graphviz.gitlab.io/_pages/doc/info/shapes.html#record";>Record-based
Nodes</a> which are very useful for box diagrams showing data layouts when a
packetdiag is too rigid.</p>
-</div>
-<div class="imageblock">
-<div class="content">
-<img src="/images/diag-67c737a775095ab65bc0eea2b6f973e4.png" alt="diag
67c737a775095ab65bc0eea2b6f973e4" width="349" height="201">
-</div>
-</div>
<div class="sect2">
<h3 id="plantuml-diagram">PlantUML Diagram</h3>
<div class="paragraph">
@@ -272,6 +256,11 @@ and here’s the same image as an image block:</p>
</ul>
</div>
<div class="paragraph">
+<p>An www.plantuml.com/plantuml/uml[online PlantUML authoring tool] is
available.
+Though just saving and refreshing the page in a web browser (usually they
auto-refresh
+after 1 second) is also pretty easy.</p>
+</div>
+<div class="paragraph">
<p>Here is a class diagram example:</p>
</div>
<div class="imageblock">
@@ -289,29 +278,44 @@ and here’s the same image as an image block:</p>
</div>
</div>
<div class="sect2">
-<h3 id="packet-diagram-easier-but-limited-to-msbf-left-to-right">Packet
Diagram - Easier but Limited to MSBF, Left-to-Right</h3>
+<h3 id="ditaa-diagram">Ditaa Diagram</h3>
<div class="paragraph">
-<p><a href="http://blockdiag.com/en/nwdiag/packetdiag-examples.html";>Packet
diagrams</a> are useful for any time you want to show data layouts at the level
of bits, bytes, and words. It assumes bit order is <em>most significant bit
first</em> and that you want to number the bits from left to right.</p>
+<p>The <a href="http://ditaa.sourceforge.net/";>DITAA</a> graphics are
ASCII-Art converted into smoother looking drawings. They have the advantage of
being visual in the text file.</p>
</div>
<div class="paragraph">
-<p>This example supplies a target name for the graphic, which allows it to be
reused not just in this document, but in others also. We specify SVG as the
format since there appears to be bugs in PNG format for packetdiag.</p>
+<p>These diagrams are quite easily drawn then cut/pasted to/from actual
asciidoc digrams using an online tool called <a
href="http://asciiflow.com/";>asciiflow</a>.</p>
</div>
<div class="imageblock">
<div class="content">
-<img src="/images/test_packet_diagram_1.svg" alt="test packet diagram 1"
width="896" height="312">
-</div>
+<img src="/images/diag-8ac9b322bf3e2e1e7b0df17f6f64d0a6.png" alt="diag
8ac9b322bf3e2e1e7b0df17f6f64d0a6" width="540" height="280">
</div>
</div>
-<div class="sect2">
-<h3 id="ditaa-diagram">Ditaa Diagram</h3>
<div class="paragraph">
-<p>The <a href="http://ditaa.sourceforge.net/";>DITAA</a> graphics are
ASCII-Art converted into smoother looking drawings. They have the advantage of
being visual in the text file, but the challenge of needing to be
-laid out by hand.</p>
+<p>Note: below is the widest ditaa diagram you can draw that will fit in the
line-length that
+github’s code-review/pull-request window can display without line-wrap.
+If the lines wrap in your ditaa diagram you really lose the ability to comment
on them
+in their textual form in the asciidoc:</p>
</div>
<div class="imageblock">
<div class="content">
-<img src="/images/diag-8ac9b322bf3e2e1e7b0df17f6f64d0a6.png" alt="diag
8ac9b322bf3e2e1e7b0df17f6f64d0a6" width="540" height="280">
+<img src="/images/diag-19f9d885d07ef92b8922b13dfaa30484.png" alt="diag
19f9d885d07ef92b8922b13dfaa30484" width="1220" height="98">
+</div>
+</div>
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="graphviz-record-based-nodes-diagram">GraphViz Record-Based Nodes
Diagram</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p><a href="https://graphviz.gitlab.io/documentation/";>GraphViz</a> is
probably the most powerful of the various diagram tools, but with that power
comes complexity that some of the other diagram types are able to overcome by
being more restrictive.</p>
+</div>
+<div class="paragraph">
+<p>This is an example of graphViz <a
href="https://graphviz.gitlab.io/_pages/doc/info/shapes.html#record";>Record-based
Nodes</a> which are very useful for box diagrams showing data layouts when a
packetdiag is too rigid.</p>
</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/diag-67c737a775095ab65bc0eea2b6f973e4.png" alt="diag
67c737a775095ab65bc0eea2b6f973e4" width="349" height="201">
</div>
</div>
</div>
diff --git a/content/dev/design-notes/hidden-groups/index.html
b/content/dev/design-notes/hidden-groups/index.html
new file mode 100644
index 0000000..5f5527d
--- /dev/null
+++ b/content/dev/design-notes/hidden-groups/index.html
@@ -0,0 +1,272 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Daffodil (incubating) | </title>
+
+ <meta name="author" content="">
+
+ <!-- Enable responsive viewport -->
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <!-- HTML5 shim, for IE6-8 support of HTML elements -->
+ <!--[if lt IE 9]>
+ <script
src="http://html5shim.googlecode.com/svn/trunk/html5.js";></script>
+ <![endif]-->
+
+ <link href="/assets/themes/apache/bootstrap/css/bootstrap.css"
rel="stylesheet">
+ <link href="/assets/themes/apache/css/style.css?body=1" rel="stylesheet"
type="text/css">
+ <link href="/assets/themes/apache/css/syntax.css" rel="stylesheet"
type="text/css" media="screen" />
+
+ </head>
+
+ <body>
+
+ <div class="navbar navbar-inverse" role="navigation">
+ <div class="container">
+ <div class="navbar-header"><a class="navbar-brand" href="/"><img
src="/assets/themes/apache/img/apache-daffodil-logo.png" alt="Apache
Daffodil"/></a></div>
+ <nav role="navigation">
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/releases">Releases</a></li>
+ <li id="documentation">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Docs<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a href="/getting-started/">Getting Started</a></li>
+ <li><a href="/examples/">Examples</a></li>
+ <li><a href="/docs/latest/javadoc/">Java API</a></li>
+ <li><a href="/docs/latest/scaladoc/">Scala API</a></li>
+ <li><a href="/docs/dfdl/">DFDL Specification</a></li>
+ <li><a href="/unsupported/">Unsupported Features</a></li>
+ <li><a href="/faq/">Frequently Asked Questions</a></li>
+ </ul>
+ </li>
+ <li id="community">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Community<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a href="/community">Get Involved</a></li>
+ <li><a href="/people">People</a></li>
+ </ul>
+ </li>
+ <li id="development">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Development<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a class="external"
href="https://cwiki.apache.org/confluence/display/DAFFODIL/";>Wiki</a></li>
+ <li><a class="external"
href="https://github.com/search?q=repo%3Aapache%2Fincubator-daffodil+repo%3Aapache%2Fincubator-daffodil-site&type=Repositories";>GitHub</a></li>
+ <li><a class="external"
href="https://issues.apache.org/jira/projects/DAFFODIL/";>JIRA</a></li>
+ </ul>
+ </li>
+ <li id="apache">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Apache<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a class="external" href="https://www.apache.org/";>Apache
Software Foundation</a></li>
+ <li><a class="external"
href="https://www.apache.org/licenses/";>License</a></li>
+ <li><a class="external"
href="https://www.apache.org/security";>Security</a></li>
+ <li><a class="external"
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a class="external"
href="https://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ </ul>
+ </li>
+ </ul>
+ </nav>
+ </div>
+ </div>
+
+
+<div class="title">
+ <div class="container">
+ <h2></h2>
+ </div>
+</div>
+
+
+
+ <div class="container">
+ <div class="row">
+ <div class="col-md-12">
+ <div class="sect1">
+<h2 id="hidden-groups">Hidden Groups</h2>
+<div class="sectionbody">
+<div class="sect2">
+<h3 id="introduction">Introduction</h3>
+<div class="paragraph">
+<p>The DFDL language enables groups to be hidden, meaning not part of the
logical infoset.</p>
+</div>
+<div class="paragraph">
+<p>This is achieved by way of <em>hidden group references</em>.
+This implies that a global group definition can be referenced from some group
references which are ordinary, non-hidden, group references, and from other
hidden group references.
+Hence, the contents of the group definition are not themselves inherently
hidden or not, as the hiddenness is based on how the group is referenced.</p>
+</div>
+<div class="paragraph">
+<p>Of course if all uses of a particular group definition are hidden group
references, then the schema compiler could provide that all elements within
that group are always hidden.
+This is expected to be fairly common, as often hidden groups are designed to
contain the hidden reprsentation details of data that has a simpler logical
appearance in the infoset.</p>
+</div>
+<div class="paragraph">
+<p>Nevertheless, Daffodil must accomodate that the same group definition can
be used both in hidden and non-hidden manners.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="static-analysis-by-the-schema-compiler-i-e-there-is-none">Static
Analysis by the Schema Compiler (i.e., there is none)</h3>
+<div class="paragraph">
+<p>The schema compiler could provide information about whether a given term of
the schema is always hidden, sometimes hidden, or never hidden.
+However, the runtime overheads are anticipated to be so small that
optimizations based on this information are expected to be unnecessary.
+Hence, no static analysis of whether a term is always/sometimes/never hidden
is performed.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="runtime-principles-of-operation">Runtime Principles of Operation</h3>
+<div class="paragraph">
+<p>The implementation takes advantage of the processor (parser/unparser)
traversal of the DFDL schema components following an in-order traversal
pattern, so that a stack-discpline can be used when traversing hidden-group
references. No actual stack is needed, only a counter.
+The details are:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>The PState/UState contains a counter (initial value 0) of the number of
hidden group references currently in the nest.</p>
+</li>
+<li>
+<p>When parsing/unparsing a hidden group ref, this counter is incremented.
+When unwinding from that unparse, the counter is decremented.
+No action is required for non-hidden group references.</p>
+</li>
+<li>
+<p>When an infoset element is created, the setHidden() method is called if the
counter is non-zero.</p>
+</li>
+<li>
+<p>At the end of processing, the counter should be zero.</p>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect2">
+<h3 id="runtime-data-structures-runtime-1">Runtime Data Structures - Runtime
1</h3>
+<div class="paragraph">
+<p>In Daffodil’s "Runtime 1" backend, these are the runtime data
structure features to support hidden group references:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>Infoset elements have an isHidden() boolean method, and a setHidden()
setter.</p>
+</li>
+<li>
+<p>The PState/UState contains the counter described above.</p>
+</li>
+<li>
+<p>The ModelGroupRuntimeData object contains an isHidden flag that is true if
the term corresponds to a hidden group reference.
+It is this flag that is inspected to determine if the PState/UState counter
should be incremented/decremented or not.</p>
+</li>
+<li>
+<p>The increment/decrement logic is centralized in Parser.parse1() method and
Unparser.unparse1() method.
+These inspect the current TermRuntimeData object, and when it is a
ModelGroupRuntimeData, they check the isHidden flag.</p>
+</li>
+<li>
+<p>At the end of processing where checking that the various runtime-stacks and
pools are properly emptied/restored, an additional check is done to insure the
hidden counter of the PState/UState has been returned to zero.
+This check need only be performed on normal exits. If the processor terminates
abnormally, we needn’t check.</p>
+</li>
+<li>
+<p>The debugger/trace should display, as part of the state, whether the
current context isHidden (counter > 0) or not.</p>
+</li>
+<li>
+<p>Parser backtracking must save/restore the counter.</p>
+</li>
+<li>
+<p>Unparser suspended UStates do not need to copy the counter value. That is,
this counter need only exist in UStateMain.</p>
+<div class="ulist">
+<ul>
+<li>
+<p>This holds because infoset elements are created as part of the state of
suspensions, and those infoset elements will have setHidden() called or not
depending on the counter. Hence, the counter value of a suspended UState is not
needed when evaluating a suspension.</p>
+</li>
+</ul>
+</div>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect2">
+<h3 id="testing">Testing</h3>
+<div class="ulist">
+<ul>
+<li>
+<p>Tests cover where the same group definition is used both hidden and
non-hidden within the same schema.</p>
+</li>
+<li>
+<p>Tests cover this for hidden sequences and hidden choices.
+Note that a hidden group references always uses an xs:sequence element, but
the referenced group can be a choice group definition.</p>
+</li>
+</ul>
+</div>
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="transition-plan-from-daffodil-2-5-0">Transition Plan From Daffodil
2.5.0</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>(Delete this section once implementation is complete.)</p>
+</div>
+<div class="paragraph">
+<p>The existing v2.5.0 isHidden implementation consists of code that populates
a member of class ElementRuntimeData (Runtime 1) isHidden member.</p>
+</div>
+<div class="paragraph">
+<p>All code for populating that member, and the member itself, can be removed.
Anything it calls which is used only for that purpose can be removed.</p>
+</div>
+<div class="paragraph">
+<p>Any code that assumes the isHidden characteristic of elements is static
information must be removed.</p>
+</div>
+<div class="paragraph">
+<p>All access to whether an element isHidden must call the isHidden method of
InfosetElement (aka DIElement) object instead of on the ERD
(ElementRuntimeData) object.</p>
+</div>
+<div class="paragraph">
+<p>There is no conversion of existing code to a new design, as the entire old
mechanism, and its underlying assumptions, are being removed, and fully
replaced by the new mechanism.</p>
+</div>
+</div>
+</div>
+ </div>
+</div>
+
+
+ <footer>
+ <footer class="site-footer">
+ <div class="wrapper">
+ <div class="footer-col-wrapper" style="font-size: .85em;">
+ <hr>
+ <div class="container">
+ <div class="row">
+ <div class="col-xs-3" style="margin-top: 15px;">
+ <a href="https://incubator.apache.org";><img
src="/assets/themes/apache/img/incubator_feather_egg_logo.png"
+ alt="Apache
Incubator" style="width:100%;"/></a>
+ </div>
+ <div class="col-xs-9">
+ Apache Daffodil is an effort undergoing <a
href="https://incubator.apache.org/index.html";>Incubation</a>
+ at The Apache Software Foundation (ASF), sponsored by
the Incubator. Incubation is required of all newly
+ accepted projects until a further review indicates
that the infrastructure, communications, and decision
+ making process have stabilized in a manner consistent
with other successful ASF projects. While incubation
+ status is not necessarily a reflection of the
completeness or stability of the code, it does indicate that
+ the project has yet to be fully endorsed by the ASF.
+ </div>
+ </div>
+ </div>
+ <hr>
+ <div>
+ <div style="text-align: center;">
+ Copyright © 2020 <a href="https://www.apache.org";>The
Apache Software Foundation</a>.
+ Licensed under the <a
href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version
+ 2.0</a>.
+ <br>
+ Apache, the Apache Incubator project logo, Apache
Daffodil, Daffodil, and the
+ Apache Daffodil logo are trademarks of The Apache Software
Foundation.
+ </div>
+ </div>
+ </div>
+ </div>
+</footer>
+
+ </footer>
+ </div>
+
+ <script src="/assets/themes/apache/jquery/jquery-2.1.1.min.js"></script>
+
+ <script src="/assets/themes/apache/bootstrap/js/bootstrap.min.js"></script>
+
+
+ </body>
+</html>
+
diff --git a/content/dev/design-notes/namespace-binding-minimization/index.html
b/content/dev/design-notes/namespace-binding-minimization/index.html
new file mode 100644
index 0000000..77fcd7a
--- /dev/null
+++ b/content/dev/design-notes/namespace-binding-minimization/index.html
@@ -0,0 +1,763 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Daffodil (incubating) | </title>
+
+ <meta name="author" content="">
+
+ <!-- Enable responsive viewport -->
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <!-- HTML5 shim, for IE6-8 support of HTML elements -->
+ <!--[if lt IE 9]>
+ <script
src="http://html5shim.googlecode.com/svn/trunk/html5.js";></script>
+ <![endif]-->
+
+ <link href="/assets/themes/apache/bootstrap/css/bootstrap.css"
rel="stylesheet">
+ <link href="/assets/themes/apache/css/style.css?body=1" rel="stylesheet"
type="text/css">
+ <link href="/assets/themes/apache/css/syntax.css" rel="stylesheet"
type="text/css" media="screen" />
+
+ </head>
+
+ <body>
+
+ <div class="navbar navbar-inverse" role="navigation">
+ <div class="container">
+ <div class="navbar-header"><a class="navbar-brand" href="/"><img
src="/assets/themes/apache/img/apache-daffodil-logo.png" alt="Apache
Daffodil"/></a></div>
+ <nav role="navigation">
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/releases">Releases</a></li>
+ <li id="documentation">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Docs<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a href="/getting-started/">Getting Started</a></li>
+ <li><a href="/examples/">Examples</a></li>
+ <li><a href="/docs/latest/javadoc/">Java API</a></li>
+ <li><a href="/docs/latest/scaladoc/">Scala API</a></li>
+ <li><a href="/docs/dfdl/">DFDL Specification</a></li>
+ <li><a href="/unsupported/">Unsupported Features</a></li>
+ <li><a href="/faq/">Frequently Asked Questions</a></li>
+ </ul>
+ </li>
+ <li id="community">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Community<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a href="/community">Get Involved</a></li>
+ <li><a href="/people">People</a></li>
+ </ul>
+ </li>
+ <li id="development">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Development<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a class="external"
href="https://cwiki.apache.org/confluence/display/DAFFODIL/";>Wiki</a></li>
+ <li><a class="external"
href="https://github.com/search?q=repo%3Aapache%2Fincubator-daffodil+repo%3Aapache%2Fincubator-daffodil-site&type=Repositories";>GitHub</a></li>
+ <li><a class="external"
href="https://issues.apache.org/jira/projects/DAFFODIL/";>JIRA</a></li>
+ </ul>
+ </li>
+ <li id="apache">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Apache<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a class="external" href="https://www.apache.org/";>Apache
Software Foundation</a></li>
+ <li><a class="external"
href="https://www.apache.org/licenses/";>License</a></li>
+ <li><a class="external"
href="https://www.apache.org/security";>Security</a></li>
+ <li><a class="external"
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a class="external"
href="https://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ </ul>
+ </li>
+ </ul>
+ </nav>
+ </div>
+ </div>
+
+
+<div class="title">
+ <div class="container">
+ <h2></h2>
+ </div>
+</div>
+
+
+
+ <div class="container">
+ <div class="row">
+ <div class="col-md-12">
+ <div class="sect1">
+<h2 id="namespace-binding-minimization">Namespace Binding Minimization</h2>
+<div class="sectionbody">
+<div class="sect2">
+<h3 id="introduction">Introduction</h3>
+<div class="paragraph">
+<p>DFDL schemas are XML schemas and so DFDL inherits the namespace system of
XML and XML Schema for composing large schemas from smaller ones, for reusing
schema files, and for managing name conflicts.</p>
+</div>
+<div class="paragraph">
+<p>A DFDL Infoset isn’t necessarily represented as XML however.
+Some representations won’t have any ability to deal with namespaces
(JSON for example), and so Daffodil will sometimes issue warnings when
compiling a schema if the namespace usage will not allow unambiguous
representation without namespaces.</p>
+</div>
+<div class="paragraph">
+<p>Most representations of DFDL Infosets will, like XML, use some
representation of the namespaces of elements, and in textual forms this will
most commonly be by way of namespace prefixes.
+XML is not the only representation that uses namespaces, however, so this
should not be taken as an entirely XML-specific discussion.</p>
+</div>
+<div class="paragraph">
+<p>There are these goals for namespace-binding minimization.</p>
+</div>
+<div class="olist arabic">
+<ol class="arabic">
+<li>
+<p>Clarity: Infosets that have redundant namespace bindings are very hard to
read and understand, and require namespace-binding-aware tooling to compare
them, or clumsy post-processing to remove the excess bindings.</p>
+</li>
+<li>
+<p>Performance: Attaching an element to the infoset at runtime should take
constant time.</p>
+</li>
+<li>
+<p>Consistency: The prefix-to-namespace bindings used should be drawn from
those expressed on the DFDL schema by the schema author, and the prefixes used
and bindings introduced when an element is attached to the infoset should be
consistent with the set of namespace prefix definitions in place at the point
where the element’s declaration lexically appears in the DFDL schema.</p>
+</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>These goals are in some tension.
+Consider 4 elements named A, B, C, and Q.
+Suppose element A contains element B, which contains element Q.
+Suppose elsewhere in the same infoset element A contains element C which
contains element Q.
+From the perspective of element Q, the set of namespace bindings surrounding
it are those from (A, B) or those from (A, C).
+Suppose element Q requires, and introduces, a namespace with prefix "qns"
bound to namespace "urn:Q_Namespace".
+Suppose element C also introduces this same namespace binding.
+Then when element Q appears inside element B, its namespace binding for "qns"
is needed.
+But when element Q appears inside element C, its namespace binding for "qns"
is redundant with one already provided by element C.</p>
+</div>
+<div class="paragraph">
+<p>The conclusion is that the minmal set of namespace bindings introduced by
an element depends on the nesting of elements.</p>
+</div>
+<div class="paragraph">
+<p>The basic technique is to store, on the runtime element data structure
(DPathElementCompileInfo), the complete set of lexical namespace bindings
present for the element declaration in the DFDL schema document.</p>
+</div>
+<div class="sect3">
+<h4
id="namespace-bindings-come-from-the-element-declarations-not-element-references">Namespace
Bindings come from the Element Declarations, not Element References</h4>
+<div class="paragraph">
+<p>Consider two schema files:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><!-- file
foo.dfdl.xsd -->
+<schema
+ xmlns:pre1="namespace1"
+ xmlns:ns1="differentNamespace">
+ <import namespace="namespace1" schemaFileLocation="bar.dfdl.xsd"/>
+ ...
+ <element name="root">
+ <complexType>
+ <sequence>
+ <element ref="pre1:foo" maxOccurs="unbounded"/>
+ </sequence>
+ </complexType>
+ </element>
+
+</schema>
+
+<!-- file bar.dfdl.xsd -->
+<schema targetNamespace="namespace1"
+ xmlns:ns1="namespace1"
+ xmlns:pre1="someOtherNamespace">
+
+ <element name="foo" ..../>
+</schema></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>In the above we have a conflict over the use of the prefix "pre1".
+Now consider an XML document corresponding to this with element 'foo' inside
the 'root' element:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><root
xmlns:pre1="namespace1"
+ xmlns:ns1="differentNamespace">
+ ...
+ <ns1:foo
+ xmlns:ns1="namespace1"
+ xmlns:pre1="someOtherNamespace">
+ ...
+ </ns1:foo>
+ ...
+</root></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>Notice that element 'foo' appears inside 'root' using the "ns1" prefix but
it also introduces a binding for that prefix which supercedes that of the
enclosing environment.
+The prefix "pre1" cannot be used for element 'foo' because in the namespace
bindings of the bar.dfdl.xsd schema document, the "pre1" prefix is bound to
"someOtherNamespace".</p>
+</div>
+<div class="paragraph">
+<p>This example illustrates that each element must use, and introduce, the
lexically defined prefixes from the point where the element is declared.
+Not from the point of element reference.</p>
+</div>
+<div class="paragraph">
+<p>Since element 'foo' is recurring, it’s unfortunate, but every single
instance will, textualized, carry these namespace bindings.
+E.g.,</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><root
xmlns:pre1="namespace1"
+ xmlns:ns1="differentNamespace">
+ ...
+ <ns1:foo
+ xmlns:ns1="namespace1"
+ xmlns:pre1="someOtherNamespace">
+ ...
+ </ns1:foo>
+ <ns1:foo
+ xmlns:ns1="namespace1"
+ xmlns:pre1="someOtherNamespace">
+ ...
+ </ns1:foo>
+ <ns1:foo
+ xmlns:ns1="namespace1"
+ xmlns:pre1="someOtherNamespace">
+ ...
+ </ns1:foo>
+
+ ...
+</root></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>This problem is not one Daffodil strives to solve.
+The schema author can simply avoid these sorts of name clashes and this
problem will not occur.
+Automatic renaming of prefixes to avoid this problem is considered
unwarranted, as it will confuse users.</p>
+</div>
+</div>
+</div>
+<div class="sect2">
+<h3 id="namespace-minimization">Namespace Minimization</h3>
+<div class="sect3">
+<h4 id="only-element-namespace-prefix-bindings">Only Element Namespace Prefix
Bindings</h4>
+<div class="paragraph">
+<p>Only namespace definitions associated with element declarations need to
ever be considered for the infoset.
+Namespace definitions that define prefixes used for type, group, format, or
escapeScheme references are not included
+in the namespace definitions carried on infoset elements.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="avoid-prefix-tns-or-other-common-ambiguous-names-when-possible">Avoid
Prefix "tns" (or Other Common Ambiguous Names) When Possible</h4>
+<div class="paragraph">
+<p>Many DFDL schemas will define prefix "tns" to be that schema
document’s target namespace.</p>
+</div>
+<div class="paragraph">
+<p>This same problem could occur for other prefixes. The "tns" convention is
just a common one.</p>
+</div>
+<div class="paragraph">
+<p>If the prefix "tns" is ambiguous across the schema set (also used by other
schema documents, but for different namespaces),
+then its use is undesirable.</p>
+</div>
+<div class="paragraph">
+<p>If a schema document defines both "tns" and other prefixes for the target
namespace, then another prefix is preferred for
+use as the prefix of elements created from declarations in that schema
document.</p>
+</div>
+<div class="paragraph">
+<p>This situation arises commonly for the default namespace (no prefix,
defined by <code>xmlns="namespaceURI"</code>). If
+this is ambiguous across the schema set (highly likely), then an available
alternative prefix (from that schema document)
+is preferred.
+There is actually no difference between using "tns" and the default namespace.
Both are just commonly used, and frequently ambiguous across the schema set.</p>
+</div>
+<div class="paragraph">
+<p>(This all generalizes to any prefix which is ambiguous across the schema
set.)</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="corner-cases">Corner Cases</h4>
+<div class="paragraph">
+<p>There are numerous ways schema authors can use and reuse namespace prefixes
that can lead to cluttered infosets.</p>
+</div>
+<div class="paragraph">
+<p>Other than minor heuristics to choose among alternative available prefix
definitions, Daffodil does not try to improve on the
+namespace prefix problem on behalf of schema authors.</p>
+</div>
+<div class="sect4">
+<h5 id="no-prefix-at-all">No Prefix At All</h5>
+<div class="paragraph">
+<p>A legal schema document can define a target namespace and define no prefix
for it at all.</p>
+</div>
+<div class="paragraph">
+<p>In this case, the only way elements of that schema document can be used is
some other schema document must provide a prefix definition.
+Daffodil chooses a prefix from those available in the schema set
(deterministically - e.g., shortest prefix, with ties resolved by alphanumeric
order, avoiding ambiguous prefixes like "tns").</p>
+</div>
+<div class="admonitionblock caution">
+<table>
+<tr>
+<td class="icon">
+<img src="/images/icons/caution.png" alt="Caution">
+</td>
+<td class="content">
+TBD: Should we issue a warning or even make this a schema definition error?
+</td>
+</tr>
+</table>
+</div>
+</div>
+<div class="sect4">
+<h5 id="only-tns-or-only-the-default-namespace">Only "tns" or Only the Default
Namespace</h5>
+<div class="paragraph">
+<p>A schema defines "tns" (or other very common prefix like "pre" or "p") for
its target namespace, but defines no other prefix that can be used as an
alternative.</p>
+</div>
+<div class="paragraph">
+<p>Daffodil does nothing here to improve on the situation where there will be
many inner namespace re-bindings of the "tns" like:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><tns:foo
xmlns:tns="namespace1">
+ <tns:bar xmlns:tns="namespace2">
+ <tns:quux xmlns:tns="namespace3">
+ ...
+ </tns:quux>
+ </tns:bar>
+</tns:foo></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>This sort of thing can happen if schema authors make extensive use of the
default namespace (no prefix).
+For example, a schema document can define a target namespace, then define that
namespace to be the default, with no other namespace prefix defined.
+In that case you can have infosets like this:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><foo
xmlns="namespace1">
+ <bar xmlns="namespace2">
+ <quux xmlns="namespace3">
+ ...
+ </quux>
+ </bar>
+</foo></code></pre>
+</div>
+</div>
+</div>
+</div>
+<div class="sect3">
+<h4 id="undefining-the-default-namespace">Undefining the Default Namespace</h4>
+<div class="paragraph">
+<p>Many schemas will not define the default namespace.</p>
+</div>
+<div class="paragraph">
+<p>If an element is defined in a schema which defines the default namespace to
a URI, and nested with that element are other elements from schemas that
+do NOT have a definition for the default namespace, then if there are
unqualified names in the latter schema that are supposed to be in <em>no
namespace</em>,
+the default namespace must be explicitly undefined.</p>
+</div>
+<div class="paragraph">
+<p>Consider two schema files:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><!-- file
foo.dfdl.xsd -->
+<xs:schema
+ xmlns="default1"
+ xmlns:ns2="namespace2" >
+
+ <xs:import namespace="namespace2" schemaFileLocation="bar.dfdl.xsd"/>
+ ...
+ <xs:element name="root">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element ref="ns2:foo" maxOccurs="unbounded"/>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+
+</xs:schema>
+
+<!-- file bar.dfdl.xsd -->
+<schema targetNamespace="namespace2"
+ xmlns:ns2="namespace2"
+ elementFormDefault="unqualified"
+ xmlns="http://www.w3.org/2001/XMLSchema"> <!-- default namespace used
for schema -->
+
+ <element name="foo">
+ <complexType>
+ <sequence>
+ <element name="bar" .../><!-- no namespace -->
+ </sequence>
+ </complexType>
+ </element>
+</schema></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>In this case, an instance of root, containing 'foo' containing 'bar'
requires:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><root
+ xmlns="default1"
+ xmlns:ns2="namespace2">
+ <ns2:foo>
+ <bar xmlns=""> <!-- undefine default -->
+ ...
+ </bar>
+ </ns2:foo>
+</root></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>This undefine shown above is needed even though the bar.dfdl.xsd schem has
a default namespace definition, because the local element names are in
no-namespace.
+The default namespace in bar.dfdl.xsd is actually not used for reference to
elements.
+So it is tantamount to not having the default namespace defined in
bar.dfdl.xsd at all.</p>
+</div>
+<div class="admonitionblock caution">
+<table>
+<tr>
+<td class="icon">
+<img src="/images/icons/caution.png" alt="Caution">
+</td>
+<td class="content">
+Some of this minimization may happen upon conversion to XML, and may happen
automatically depending on XML libraries.
+That is, an element with no namespace displayed in a context which has a
default namespace definition may automatically insert <code>xmlns=""</code>.
+</td>
+</tr>
+</table>
+</div>
+</div>
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="namespace-binding-minimization-algorithm">Namespace Binding
Minimization Algorithm</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>The technique described here assumes that one needs to render the infoset
to XML text using standard printing, i.e., using no special XML library.
+Hence, every namespace prefix binding must be explicitly represented in the
output XML text.</p>
+</div>
+<div class="paragraph">
+<p>The basics are:</p>
+</div>
+<div class="olist arabic">
+<ol class="arabic">
+<li>
+<p>For every element declaration, capture the lexical namespace scope
(<code>scala.xml.NamespaceBinding</code>) from its element declaration XML in
its schema document.
+Save this on the runtime data structure for the element. In Runtime 1, this
would be the DPathElementCompileInfo. (This is longstanding functionality in
Daffodil since before version 1.0.0)</p>
+</li>
+<li>
+<p>Excepting on the Root, remove any namespace binding that is unambiguous
across the schema, and which appears on the root.</p>
+</li>
+<li>
+<p>For each element declaration, the remaining namespace bindings and assigned
prefix to be used are assigned based on the minimization rules describe above
(e.g., about avoiding "tns" when possible.)</p>
+</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>That is all that is done at schema compile time and at parse time up to the
point where a textual representation (such as XML) needs to be output.</p>
+</div>
+<div class="paragraph">
+<p>The DFDL Infoset tree is constructed with InfosetElement nodes that point
to this compile time DPathElementCompileInfo structure, and no processing of
namespace bindings occurs.</p>
+</div>
+<div class="paragraph">
+<p>However when converting an infoset element into XML examine the namespace
bindings of the element and those of the enclosing parent element.</p>
+</div>
+<div class="olist loweralpha">
+<ol class="loweralpha" type="a">
+<li>
+<p>Any that are redundant across the two are dropped.</p>
+</li>
+<li>
+<p>New definitions introduced by the child are output as bindings</p>
+</li>
+<li>
+<p>Redefinitions are output as bindings</p>
+</li>
+<li>
+<p>If the element has no namespace, and the parent (or any super-parent) has a
default namespace binding, then add an undefine binding for the default
namespace.</p>
+</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>This algorithm requires non-constant-time (worst case) processing at
runtime; however, there is no overhead unless there are ambiguities among the
namespace bindings and when namespace bindings at nodes beneath the root are
required.
+In addition, the number of such cases in any <em>real</em> schema will be
small, so the algorithmic complexity worst-case here is far less important than
the constant factor here.
+Attaching an element to the infoset is a common operation.
+These namespace binding machinations have the potential to be equally costly,
per binding, to the general overhead of attaching the infoset element node.</p>
+</div>
+<div class="paragraph">
+<p>Our standard design principle is, however, to not worry about overheads
like this which are often not going to occur in real schemas, unless
performance profiling shows them to be a hot-spot.</p>
+</div>
+<div class="paragraph">
+<p>Sensibly-designed schemas will have no overhead from this namespace-binding
combining.</p>
+</div>
+<div class="sect2">
+<h3 id="converting-the-dfdl-infoset-to-xml-in-one-pass-streaming">Converting
the DFDL Infoset to XML in One Pass (Streaming)</h3>
+<div class="paragraph">
+<p>Note that eliminating prefix definitions that are unused in a particular
XML document is not compatible with streaming.
+It requires two passes to determine if a prefix is ever used to decide whether
it can be omitted or must be included.</p>
+</div>
+<div class="paragraph">
+<p>The only alternative to this is to introduce new namespace prefix
definitions only at their point of use.
+That would, however, be inconsistent with our goal of clarity and avoiding
namespace prefix clutter in the schema.</p>
+</div>
+<div class="paragraph">
+<p>It is preferable to output extra namespace bindings on the root element
than to litter the document with namespace bindings at
+interior XML elements.</p>
+</div>
+<div class="paragraph">
+<p>Daffodil aspires to streaming parsing and unparsing. A streaming parser
will output parts of the infoset without waiting to
+know if children will eventually appear that require the namespace prefix
definitions.
+As a result, all namespace prefix definitions which <em>may</em> be required
are included.
+Most commonly this will result in extra unused namespace prefix definitions
having been output on the start tag of the root element.</p>
+</div>
+</div>
+<div class="sect2">
+<h3
id="api-xml-fragment-mode-for-clarity-avoid-namespace-bindings-on-the-root">API
XML-Fragment Mode - For Clarity: Avoid Namespace Bindings on the Root</h3>
+<div class="paragraph">
+<p>When using the message streaming API and converting the parse Infoset to
XML, each message is created as XML text by the parser and associated
InfosetOutputter, and converting one relatively small message to XML may result
in far more characters used to represent the namespace bindings on the root of
the message than the rest of the message occupies in XML text.</p>
+</div>
+<div class="paragraph">
+<p>The API provides a method to enable XML-fragment mode.
+In this mode a method can be called to retrieve namespace prefix bindings that
would appear on the root (i.e, on the root XML element of each message) if a
complete XML data document were being created.
+Subsequent calls to parse in XML-Fragment mode create XML which has no
namespace bindings on the root element of each message.
+This is an XML fragment because it lacks the namespace bindings needed for it
to be a complete XML document.
+This is in effect leaving it up to the caller whether and when to append the
namespace bindings to the XML text.</p>
+</div>
+<div class="paragraph">
+<p>This option is provided because the caller may not want to construct
complete XML documents, and the namespace bindings in use for the XML may be
well-known by the processing system.</p>
+</div>
+<div class="paragraph">
+<p>This is less a performance optimization (XML text is really verbose, and
this optimization is only scratching the surface).
+This feature is about clarity and coping with XML when using it for small data
documents corresponding to small communications messages.
+Small XML data documents can be overwealmed by the volume of namespace
definitions.
+This is particularly likely if they are for small data messages created from
large DFDL schemas with many schema documents and many namespaces.</p>
+</div>
+<div class="paragraph">
+<p>As an example consider:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml"
data-lang="xml"><gn:Feature xmlns:cc="http://creativecommons.org/ns#";
xmlns:dcterms="http://purl.org/dc/terms/";
xmlns:foaf="http://xmlns.com/foaf/0.1/";
xmlns:gn="http://www.geonames.org/ontology#";
xmlns:owl="http://www.w3.org/2002/07/owl#";
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#";
xmlns:wgs84_pos="http://www.w3.org/2003/01/geo/wgs84_pos#"><rdfs:isDefinedBy>sws/3/
[...]
+</div>
+</div>
+<div class="paragraph">
+<p>This is almost impossible to understand, given that the first 1/3 of it is
just namespace bindings.</p>
+</div>
+<div class="paragraph">
+<p>Without the namespace bindings it is easier. It looks like:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml"
data-lang="xml"><gn:Feature><rdfs:isDefinedBy>sws/3/about.rdf</rdfs:isDefinedBy><gn:name>Zamīn
Sūkhteh</gn:name><gn:alternateName><lang>fa</lang><name>Zamīn
Sūkhteh</name></gn:alternateName><gn:alternateName><lang>fa</lang><name>زمين
سوخته</name></gn:alternateName><gn:featureClass>ontology#S</gn:featureClass><gn:featureCode
[...]
+</div>
+</div>
+<div class="paragraph">
+<p>With line endings after each element end tag, it is quite easy to
understand.</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml"
data-lang="xml"><gn:Feature><rdfs:isDefinedBy>sws/3/about.rdf</rdfs:isDefinedBy>
+<gn:name>Zamīn Sūkhteh</gn:name>
+<gn:alternateName><lang>fa</lang><name>Zamīn
Sūkhteh</name></gn:alternateName>
+<gn:alternateName><lang>fa</lang><name>زمين
سوخته</name></gn:alternateName>
+<gn:featureClass>ontology#S</gn:featureClass>
+<gn:featureCode>ontology#S.CRRL</gn:featurecode>
+<gn:countryCode>IR</gn:countryCode>
+<wgs84_pos:lat>32.45831</wgs84_pos:lat>
+<wgs84_pos:long>48.96335</wgs84_pos:long>
+<gn:parentFeature>sws/3202991/</gn:parentFeature>
+<gn:parentCountry>sws/130758/</gn:parentCountry>
+<gn:parentADM1>sws/127082/</gn:parentADM1>
+<gn:nearbyFeatures>sws/3/nearby.rdf</gn:nearbyFeatures>
+<gn:locationMap>3/zamin-sukhteh.html</gn:locationMap>
+</gn:Feature></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>The XML Infoset Inputter also has a feature allowing an API method to
supply the root-level namespace bindings once, not on the root element of every
XML-fragment delivered for unparsing.</p>
+</div>
+<div class="paragraph">
+<p>The symmetry of the API insures that one can unparse the XML output from a
parse that is creating XML in this fragment mode.</p>
+</div>
+<div class="paragraph">
+<p>The Daffodil CLI has an option to add XML-fragment mode to message
streaming behavior for parsing and unparsing.</p>
+</div>
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="transition-plan-from-daffodil-2-5-0">Transition Plan from Daffodil
2.5.0</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>(Delete this section once implementation is complete.)</p>
+</div>
+<div class="paragraph">
+<p>The existing design in Daffodil 2.5.0 assumes that every element
declaration is unique, not shared, and so the entire path from that
element’s declaration object to the root element is well known and
unique.</p>
+</div>
+<div class="sect2">
+<h3 id="qnames-and-name-resolution">QNames and Name Resolution</h3>
+<div class="paragraph">
+<p>The QNameBase.scala and ResolvesQNames traits do not need modification, but
their function needs to be understood to know what <strong>does</strong> have
to change.</p>
+</div>
+<div class="paragraph">
+<p>Below shows the classes used for reprsenting the names of named things in a
DFDL schema, and for referring to named things in a DFDL schema:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/diag-a233f3edc65e7c991774d9233c23f186.png" alt="diag
a233f3edc65e7c991774d9233c23f186" width="613" height="310">
+</div>
+</div>
+<div class="paragraph">
+<p>We distinguish names given to objects by their definitions/declarations
from names referring to those by way of the NamedQName and RefQName distinction.
+A NamedQName is created for a named thing. A RefQName is created for points of
reference to it. The two can never be confused because you can’t create a
RefQName from a declaration/definition (that would create a NamedQName), and
you cannot create a NamedQName from the "ref" property (that will create a
RefQName). Furthermore, you can’t test two NamedQNames to see if they
match, you have to test if a RefQName refers to a NamedQName.</p>
+</div>
+<div class="paragraph">
+<p>All these objects are created via methods on the singleton QName object.</p>
+</div>
+<div class="sect3">
+<h4 id="resolvesqnames-trait">ResolvesQNames trait</h4>
+<div class="paragraph">
+<p>This trait provides the resolveQName(qnString: String) : RefQName method
which calls the QName.resolveRef() method supplying the necessary namespace
binding information from the XML of the schema component that is needed to
resolve QName prefixes to specific namespaces.</p>
+</div>
+<div class="paragraph">
+<p>This is done by wau of the public namespaces member. The scala.xml.Node
class has a scope member which returns type NamespaceBinding. While singular,
this is not a single binding, but a chained data structure where a first
namespace binding object is chained to a second and subsequent one. This not an
ordinary Seq() style collection. Hence the ResolvesQNames trait provides
member:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-scala" data-lang="scala">val
namespaces = xml.scope</code></pre>
+</div>
+</div>
+</div>
+</div>
+<div class="sect2">
+<h3 id="elementbase-changes">ElementBase Changes</h3>
+<div class="paragraph">
+<p>The existing ElementBase class has members, all of which are subject to
revision/removal.</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>Outputs</p>
+<div class="ulist">
+<ul>
+<li>
+<p>thisElementsNamespace - target namespace of defining schema or no-namespace
if a local element decl with elementFormDefault = "unqualified", or no target
namespace.</p>
+</li>
+<li>
+<p>thisElementsNamespacePrefix - This is the prefix that will be used for this
element when converting to XML. This will change to be different from the
provided namespace prefix only if we choose a non-"tns" equivalent, or generate
a namespace prefix for particular cases.</p>
+</li>
+<li>
+<p>thisElementsRequiredNamespaceBindings - This will be removed. This
can’t be statically computed like this anymore.</p>
+</li>
+<li>
+<p>protected minimizedScope - This will be removed or made optional. Ths
can’t always be statically computed like this anymore. This will be
computed instead at runtime.</p>
+</li>
+</ul>
+</div>
+</li>
+<li>
+<p>Minimization algorithm machinery - these members are likely no longer
needed. What they were computing statically and probably very inefficiently
needs to be done at runtime (sometimes), and far more efficiently.</p>
+<div class="ulist">
+<ul>
+<li>
+<p>private emptyNSPairs</p>
+</li>
+<li>
+<p>private myOwnNSPairs</p>
+</li>
+<li>
+<p>private myParentNSPairs</p>
+</li>
+<li>
+<p>private myUniquePairs</p>
+</li>
+<li>
+<p>private def pairsToNSBinding</p>
+</li>
+<li>
+<p>private parentMinimizedScope</p>
+</li>
+</ul>
+</div>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect2">
+<h3 id="runtime-1-changes">Runtime 1 changes</h3>
+<div class="ulist">
+<ul>
+<li>
+<p>DPathElementCompileInfo</p>
+<div class="ulist">
+<ul>
+<li>
+<p>Cleanup: <code>val name: String</code> should be removed as a constructor
argument and be obtained from the namedQName via <code>def name =
namedQName.local</code></p>
+</li>
+</ul>
+</div>
+</li>
+<li>
+<p>ElementRuntimeData</p>
+<div class="ulist">
+<ul>
+<li>
+<p>namespaces - should stay the same.</p>
+</li>
+<li>
+<p>targetNamespace - should stay the same</p>
+</li>
+<li>
+<p>targetNamespacePrefix - computation of this may/will change in case of,
e.g., choosing one that is not "tns" or other ambiguous prefix.</p>
+</li>
+<li>
+<p>thisElementsNamespacePrefix - computation of this may/will change similarly
to avoid "tns" or other undesriable/ambiguous prefix.</p>
+</li>
+<li>
+<p>minimizedScope - likely changes per the "namespace binding minimization
algorithm" above. Should be renamed to reflect proper usage as a statically
computed part of namespaces that must be incorporated at runtime when an
infoset element is attached to a parent. The new name might be
"nonRootMovedScopes" or soemthing to reflect that these are bindings which
could not be statically just be relocated onto the root element. This will be a
primary input to the new runtime algorithm that a [...]
+</li>
+</ul>
+</div>
+</li>
+</ul>
+</div>
+</div>
+</div>
+</div>
+ </div>
+</div>
+
+
+ <footer>
+ <footer class="site-footer">
+ <div class="wrapper">
+ <div class="footer-col-wrapper" style="font-size: .85em;">
+ <hr>
+ <div class="container">
+ <div class="row">
+ <div class="col-xs-3" style="margin-top: 15px;">
+ <a href="https://incubator.apache.org";><img
src="/assets/themes/apache/img/incubator_feather_egg_logo.png"
+ alt="Apache
Incubator" style="width:100%;"/></a>
+ </div>
+ <div class="col-xs-9">
+ Apache Daffodil is an effort undergoing <a
href="https://incubator.apache.org/index.html";>Incubation</a>
+ at The Apache Software Foundation (ASF), sponsored by
the Incubator. Incubation is required of all newly
+ accepted projects until a further review indicates
that the infrastructure, communications, and decision
+ making process have stabilized in a manner consistent
with other successful ASF projects. While incubation
+ status is not necessarily a reflection of the
completeness or stability of the code, it does indicate that
+ the project has yet to be fully endorsed by the ASF.
+ </div>
+ </div>
+ </div>
+ <hr>
+ <div>
+ <div style="text-align: center;">
+ Copyright © 2020 <a href="https://www.apache.org";>The
Apache Software Foundation</a>.
+ Licensed under the <a
href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version
+ 2.0</a>.
+ <br>
+ Apache, the Apache Incubator project logo, Apache
Daffodil, Daffodil, and the
+ Apache Daffodil logo are trademarks of The Apache Software
Foundation.
+ </div>
+ </div>
+ </div>
+ </div>
+</footer>
+
+ </footer>
+ </div>
+
+ <script src="/assets/themes/apache/jquery/jquery-2.1.1.min.js"></script>
+
+ <script src="/assets/themes/apache/bootstrap/js/bootstrap.min.js"></script>
+
+
+ </body>
+</html>
+
diff --git
a/content/dev/design-notes/term-sharing-in-schema-compiler/index.html
b/content/dev/design-notes/term-sharing-in-schema-compiler/index.html
new file mode 100644
index 0000000..9176548
--- /dev/null
+++ b/content/dev/design-notes/term-sharing-in-schema-compiler/index.html
@@ -0,0 +1,922 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Daffodil (incubating) | </title>
+
+ <meta name="author" content="">
+
+ <!-- Enable responsive viewport -->
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <!-- HTML5 shim, for IE6-8 support of HTML elements -->
+ <!--[if lt IE 9]>
+ <script
src="http://html5shim.googlecode.com/svn/trunk/html5.js";></script>
+ <![endif]-->
+
+ <link href="/assets/themes/apache/bootstrap/css/bootstrap.css"
rel="stylesheet">
+ <link href="/assets/themes/apache/css/style.css?body=1" rel="stylesheet"
type="text/css">
+ <link href="/assets/themes/apache/css/syntax.css" rel="stylesheet"
type="text/css" media="screen" />
+
+ </head>
+
+ <body>
+
+ <div class="navbar navbar-inverse" role="navigation">
+ <div class="container">
+ <div class="navbar-header"><a class="navbar-brand" href="/"><img
src="/assets/themes/apache/img/apache-daffodil-logo.png" alt="Apache
Daffodil"/></a></div>
+ <nav role="navigation">
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/releases">Releases</a></li>
+ <li id="documentation">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Docs<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a href="/getting-started/">Getting Started</a></li>
+ <li><a href="/examples/">Examples</a></li>
+ <li><a href="/docs/latest/javadoc/">Java API</a></li>
+ <li><a href="/docs/latest/scaladoc/">Scala API</a></li>
+ <li><a href="/docs/dfdl/">DFDL Specification</a></li>
+ <li><a href="/unsupported/">Unsupported Features</a></li>
+ <li><a href="/faq/">Frequently Asked Questions</a></li>
+ </ul>
+ </li>
+ <li id="community">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Community<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a href="/community">Get Involved</a></li>
+ <li><a href="/people">People</a></li>
+ </ul>
+ </li>
+ <li id="development">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Development<b class="caret"></b></a>
+ <ul class="dropdown-menu dropdown-left">
+ <li><a class="external"
href="https://cwiki.apache.org/confluence/display/DAFFODIL/";>Wiki</a></li>
+ <li><a class="external"
href="https://github.com/search?q=repo%3Aapache%2Fincubator-daffodil+repo%3Aapache%2Fincubator-daffodil-site&type=Repositories";>GitHub</a></li>
+ <li><a class="external"
href="https://issues.apache.org/jira/projects/DAFFODIL/";>JIRA</a></li>
+ </ul>
+ </li>
+ <li id="apache">
+ <a href="#" data-toggle="dropdown"
class="dropdown-toggle">Apache<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a class="external" href="https://www.apache.org/";>Apache
Software Foundation</a></li>
+ <li><a class="external"
href="https://www.apache.org/licenses/";>License</a></li>
+ <li><a class="external"
href="https://www.apache.org/security";>Security</a></li>
+ <li><a class="external"
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+ <li><a class="external"
href="https://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+ </ul>
+ </li>
+ </ul>
+ </nav>
+ </div>
+ </div>
+
+
+<div class="title">
+ <div class="container">
+ <h2></h2>
+ </div>
+</div>
+
+
+
+ <div class="container">
+ <div class="row">
+ <div class="col-md-12">
+ <div class="sect1">
+<h2 id="term-sharing-in-the-schema-compiler">Term Sharing in the Schema
Compiler</h2>
+<div class="sectionbody">
+<div class="sect2">
+<h3 id="introduction">Introduction</h3>
+<div class="paragraph">
+<p>The DFDL language has a composition property known as <em>referential
transparency</em>.
+It is not supposed to matter whether you lift a part of the schema out
+and create a reusable group, reusable type, or reusable element from
+it.</p>
+</div>
+<div class="paragraph">
+<p>Because DFDL (version 1.0) does not allow recursive definitions of any
+kind, it is theoretically possible to implement this by treating all
+reusable definitions in the language like macros.
+I.e., inline-expand all definitions at their point of use.
+This will work for schemas up to some size. However, it leads to an
+unacceptable expansion in schema compilation time and space, as the
+size of the schema once all these inline substitutions have been
+performed can be exponentially larger than the original
+non-substituted schema.</p>
+</div>
+<div class="paragraph">
+<p>To avoid this undesirable combinatorial explosion, the Daffodil schema
+compiler arranges to achieve the same macro-like semantics, while
+still sharing the core parts of the reusable components so they need
+only be compiled once for each unique way the component is used.</p>
+</div>
+<div class="paragraph">
+<p>This is also part of eventually enabling an extension of the DFDL
+language to allow recursive definitions.</p>
+</div>
+<div class="paragraph">
+<p>The dfdl:alignment property is one of the largest contributors to
+complexity in sharing objects by the schema compiler.</p>
+</div>
+<div class="paragraph">
+<p>Alignment will be used as the example in this design note to
+illustrate the schema compiler behavior.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="dsom-term-objects">DSOM Term Objects</h3>
+<div class="paragraph">
+<p>DSOM is the Daffodil Schema Object Model.
+Within this model, the components from a DFDL schema that actually have
+representations in the data stream or are computed, are called <em>terms</em>
and DSOM models them as
+subclasses of the class/trait Term.
+Terms which are computed (via dfdl:inputValueCalc) are said to have <em>no
representation</em> in the data stream.</p>
+</div>
+<div class="paragraph">
+<p>For this discussion we are concerned only with <em>concrete</em> terms,
meaning those that do have representation.</p>
+</div>
+<div class="paragraph">
+<p>The classes in Daffodil that are used for concrete terms are:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>ElementRef</p>
+</li>
+<li>
+<p>Root (A degenerate element ref to the root element)</p>
+</li>
+<li>
+<p>LocalElementDecl</p>
+</li>
+<li>
+<p>the quasi-elements:</p>
+<div class="ulist">
+<ul>
+<li>
+<p>PrefixLengthQuasiElementDecl (Fake local element decl used for the
+length fields of Prefixed-length types)</p>
+</li>
+<li>
+<p>RepTypeQuasiElementDecl (Fake local element decl used as the
+representation of elements that are computed via the dfdl:repType
+mechanism.)</p>
+</li>
+</ul>
+</div>
+</li>
+<li>
+<p>Sequence</p>
+</li>
+<li>
+<p>SequenceGroupRef</p>
+</li>
+<li>
+<p>ChoiceBranchImpliedSequence (The sequence that is inserted when a
+choice branch is a single element decl/ref)</p>
+</li>
+<li>
+<p>Choice</p>
+</li>
+<li>
+<p>ChoiceGroupRef</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>A <a
href="https://cwiki.apache.org/confluence/display/DAFFODIL/DFDL+Schema+Object+Model+%28DSOM%29+with+UML";>UML
Diagram</a> is available showing these classes.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="term-representation-regions">Term Representation Regions</h3>
+<div class="paragraph">
+<p>Consider the representation of a term, which we call the term’s
+<em>region</em>, as appearing between two other term regions in the data
+stream as illustrated below:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/diag-51b412adf272104af315f2f6958aeb2d.png" alt="diag
51b412adf272104af315f2f6958aeb2d" width="630" height="154">
+</div>
+</div>
+<div class="paragraph">
+<p>In the above, lower position bits are to the left.
+Higher position bits to the right.
+In the diagram, we see that the term’s region consists of 3
+sub-regions, named <em>unique before</em>, <em>shared</em>, and <em>shared
after</em>. Each
+term appears in a context of possible additional terms before it and
+after it which are shown with ellipsis above.</p>
+</div>
+<div class="paragraph">
+<p>A term can be an element or a model-group (sequence or choice).
+By far the most common situation is that a term appears inside a model group.
+There are two exceptions. The <em>root</em>, and the model-group of a complex
type.</p>
+</div>
+<div class="paragraph">
+<p>The core idea here is that we are separating the representation of a
+term into unique and shared parts.
+The unique region is affected by the surrounding model group context.
+The shared regions are, in many cases, sharable across instances of
+this term and is independent of the surrounding model-group context.
+So if the term was defined by way of a reusable global definition,
+then the goal is that there need be only one implementation of the
+shared part(s).</p>
+</div>
+<div class="paragraph">
+<p>There are some limits to this sharing.
+A single implementation is not always achievable, but generally one or
+a small number of implementations are possible.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="limits-to-sharing">Limits to Sharing</h3>
+<div class="paragraph">
+<p>Consider if this term above was defined by way of a XSD group
+reference.
+The DFDL properties expressed on the group reference must
+be combined with those expressed on the group definition, to form the
+complete set of properties associated with the term.
+This combining creates situations where a given shared group definition can
have
+wildly different representations for different uses.
+Consider this:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml"><group
name="g">
+ <sequence>
+ <element name="a" type="xs:int"/>
+ <element name="b" type="xs:int"/>
+ <element name="c" type="xs:int"/>
+ </sequence>
+</group>
+
+....
+
+<element name="e">
+ <complexType>
+ <sequence>
+ ... some stuff here ...
+ <group ref="tns:g" dfdl:separator="|"/> <!-- separated -->
+ ... more stuff here ...
+ <group ref="tns:g"/> <!-- not separated -->
+ ... yet more stuff here ...
+ </sequence>
+ </complexType>
+</element></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>The two uses of the group definition for 'g' are wildly different, in
+that one has separators, the other does not.</p>
+</div>
+<div class="paragraph">
+<p>This is a rather extreme example, but DFDL implementations must allow
+for this.</p>
+</div>
+<div class="paragraph">
+<p>Experience with DFDL schemas to-date (2020-01-21) is that this is very
+rare and appears only in test cases designed to exercise it.
+However, less extreme versions of this are possible, such as different
+uses all being separated and delimited textual data, but where the
+distinct uses do not all use the same separator characters, so the
+separator to be used would be specified differently on each group
+reference. Another expected variation could be separated sequence
+groups where some uses are infix separator and others are prefix or
+postfix separator.</p>
+</div>
+<div class="paragraph">
+<p>Anecdotally, the vast majority of schemas seen to-date reuse groups
+entirely, that is specifying no properties at all on the group
+reference.</p>
+</div>
+<div class="sect3">
+<h4 id="property-environment-or-propenv">Property Environment or PropEnv</h4>
+<div class="paragraph">
+<p>We define the <em>property environment</em> or <em>PropEnv</em> of a term,
as the
+complete set of properties for that term, combining them from all the
+schema components that define the term. This can include element
+references, group references, local or global element declarations,
+global group definitions, and local and global type definitions.</p>
+</div>
+<div class="paragraph">
+<p>A key observation is that the PropEnv of a term is entirely defined by:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>local properties (including any dfdl:ref property) on the schema component
for the term.</p>
+<div class="ulist">
+<ul>
+<li>
+<p>E.g., for an element reference, the local properties expressed on the
element reference itself.</p>
+</li>
+</ul>
+</div>
+</li>
+<li>
+<p>default format object at top level of the schema where the term lexically
appears.</p>
+</li>
+<li>
+<p>definition object being referenced.</p>
+<div class="ulist">
+<ul>
+<li>
+<p>E.g., for an element reference, the global element declaration being
referenced.</p>
+</li>
+</ul>
+</div>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>So if two terms, say group references, have the same PropEnv, then we
+can share much about their definitions when implementing them.</p>
+</div>
+<div class="paragraph">
+<p>Hence, when constructing the Daffodil schema compiler’s representation
+for a term, we can maintain a cache for each global definition, with
+the key of the cache being the PropEnv.
+When a given term uses a global definition, if the point of use has
+the same PropEnv as another, both can share the part of the term that
+is context independent, that is, the shared region.</p>
+</div>
+<div class="paragraph">
+<p>The actual components of the PropEnv of a term are slighly more
+complicated than described above.
+The table below gives the components of the PropEnv for the various
+kinds of terms. Note that the "local properties" below includes any
+dfdl:ref property if it appears, but equality of any property which
+has as its value a QName, like dfdl:ref, the property value is based
+on a resolved QName, not the QName string.</p>
+</div>
+<table class="tableblock frame-all grid-all stretch">
+<caption class="title">Table 1. PropEnv Components</caption>
+<colgroup>
+<col style="width: 25%;">
+<col style="width: 75%;">
+</colgroup>
+<thead>
+<tr>
+<th class="tableblock halign-left valign-top">Term Definition (one PropEnv
subtype per)</th>
+<th class="tableblock halign-left valign-top">PropEnv Components</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Local
Element Decl with type reference</p></td>
+<td class="tableblock halign-left valign-top"><div class="content"><div
class="ulist">
+<ul>
+<li>
+<p>Local properties expressed on the Local Element Decl (Set of property name
+ value pairs)</p>
+</li>
+<li>
+<p>Lexically enclosing default format (object ref)</p>
+</li>
+<li>
+<p>Type definition (object ref) or primitive type (object ref)</p>
+</li>
+</ul>
+</div></div></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Local
Element Decl with anonymous Simple Type Definition</p></td>
+<td class="tableblock halign-left valign-top"><div class="content"><div
class="ulist">
+<ul>
+<li>
+<p>Combined set of local properties expressed on the Local Element Decl and
the anoymous Simple Type definition.</p>
+</li>
+<li>
+<p>Lexically enclosing default format (object ref)</p>
+</li>
+<li>
+<p>Base simple type definition (object ref), or primitive type (object ref)</p>
+</li>
+</ul>
+</div></div></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Local
Element Decl with anonymous complex type definition</p></td>
+<td class="tableblock halign-left valign-top"><div class="content"><div
class="ulist">
+<ul>
+<li>
+<p>Local properties expressed on the local element decl</p>
+</li>
+<li>
+<p>Lexically enclosing default format (object ref)</p>
+</li>
+<li>
+<p>Anonymous complex type (object ref)</p>
+</li>
+</ul>
+</div></div></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Element
Reference to Global Element Decl</p></td>
+<td class="tableblock halign-left valign-top"><div class="content"><div
class="ulist">
+<ul>
+<li>
+<p>Local properties expressed on the element reference</p>
+</li>
+<li>
+<p>Lexically enclosing default format (object ref)</p>
+</li>
+<li>
+<p>Global Element Decl (object ref)</p>
+</li>
+</ul>
+</div></div></td>
+</tr>
+</tbody>
+</table>
+<div class="paragraph">
+<p>Lookups by PropEnv compare sets by equality of members, and object
+references by pointer equality i.e, eq comparison.</p>
+</div>
+<div class="paragraph">
+<p>This definition of PropEnv can be improved by memoizing the
+construction of default format objects, so that equivalent default
+formats from different schema documents are represented by the same
+object.
+That is, so that multiple schema documents each containing:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre class="highlight"><code class="language-xml" data-lang="xml">
<dfdl:format ref="prefix:formatName"/></code></pre>
+</div>
+</div>
+<div class="paragraph">
+<p>are instantiated as the same object when the QName resolves to the same
format object.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="details-of-unique-and-shared-regions">Details of Unique and Shared
Regions</h4>
+<div class="paragraph">
+<p>The division of the reprsentation into the unique part which appears
+before the shared parts indicates where we can share implementation,
+and where we must have unique context-specific implementation for the
+term.</p>
+</div>
+<div class="paragraph">
+<p>To understand this better, the below breaks down the sub-regions of a term
further.
+First we look at the details of the unique-before region:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/diag-1313a13d91a6f8c668ade63f72bbb61d.png" alt="diag
1313a13d91a6f8c668ade63f72bbb61d" width="840" height="378">
+</div>
+</div>
+<div class="paragraph">
+<p>The sub-regions within the unique before region are:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>prefix infix sep’r MTA - mandatory text alignment for a separator in
infix or prefix position.
+When a separator is defined, this region is populated with up to 7 bits to
+obtain text alignment for the characters of the separator in the specified
charset encoding.
+Note that this encoding and that of the term’s initator/terminator are
not necessarily the same encoding.</p>
+</li>
+<li>
+<p>prefix infix sep’r - separator when defined and in prefix or infix
position.
+This consists of text characters.</p>
+</li>
+<li>
+<p>lSkip - leading skip region.
+This is fixed length (commonly 0)</p>
+</li>
+<li>
+<p>alignFill - alignment fill region.
+This dynamically sized region depends on the bit position where it starts.</p>
+</li>
+<li>
+<p>init’r MTA - mandatory text alignment for initiator.
+When an initiator is defined, this region is populated with up to 7 bits to
+obtain text alignment for the characters of the initiator in specified charset
encoding.</p>
+</li>
+<li>
+<p>init’r - initiator.
+This consists of text characters.</p>
+</li>
+<li>
+<p>prefix length - (element terms only) the prefix length.
+Sub-detail of this region is not shown here.
+The PrefixLengthQuasiElementDecl class is used to represent this subregion in
DSOM.</p>
+</li>
+<li>
+<p>value MTA - (element terms only, simple type only) mandatory text alignment
for the value.
+When the value has text representation this insures those characters start
+on the right bit-boundary for characters of the specified charset encoding.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>The sizes of the alignment regions (alignFill, and the MTA regions) in
+the above all depend on where the start of this term is.
+That depends on where the prior term ends, if there is a prior term.
+That is, the sizes of the alignment regions depend on the enclosing
+model groups, and what can appear before this term in that nest.</p>
+</div>
+<div class="paragraph">
+<p>Now we look at an expansion of the shared regions:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/diag-45bcd5e9c7484340ba070e865b4dd7ed.png" alt="diag
45bcd5e9c7484340ba070e865b4dd7ed" width="920" height="504">
+</div>
+</div>
+<div class="paragraph">
+<p>It is key that the shared region can assume the alignment
+fill and MTA regions have been sized per the requirements of this
+term.
+Hence, the shared region is context independent and its implementation - in
schema
+compiler objects and in runtime structure representation, need only be
represented once.</p>
+</div>
+<div class="admonitionblock important">
+<table>
+<tr>
+<td class="icon">
+<img src="/images/icons/important.png" alt="Important">
+</td>
+<td class="content">
+<div class="title">Corner Case: Interior Alignment</div>
+<div class="paragraph">
+<p>The semantics here aren’t identical to those of macro expansion, but
the difference is hard to
+detect.</p>
+</div>
+<div class="paragraph">
+<p>The <a
href="https://opendfdl.github.io/gwdrp-dfdl-v1.0.5_r11/gwdrp-dfdl-v1.0.5-r11-change-tracking.htm";>DFDL
Spec (recent draft)</a>
+includes a <a
href="https://opendfdl.github.io/gwdrp-dfdl-v1.0.5_r11/gwdrp-dfdl-v1.0.5-r11-change-tracking.htm#\_Toc27061169";>section
(23.6 in the linked draft)</a> on
+this
+<i>interior alignment problem</i>.</p>
+</div>
+<div class="paragraph">
+<p>The technique for approximating alignment info described here is slighly
less precise in its
+analysis because it resets its knowledge of the alignment at each known
alignment point.
+Whereas pure macro expansion could carry forward knowledge of alignment and
perhaps optimize out more
+alignment fill regions.</p>
+</div>
+<div class="paragraph">
+<p>Hence, it’s possible that the technique here will leave some
alignment fill regions in place and thereby some
+formats will experience circular deadlocks when unparsing due to this
variable-length interior-alignment.</p>
+</div>
+<div class="paragraph">
+<p>The Daffodil test suite includes examples of this interior alignment that
cause unparser circular deadlocks.
+(see testOutputValueCalcVariableLengthThenAlignmentDeadlock)</p>
+</div>
+</td>
+</tr>
+</table>
+</div>
+<div class="paragraph">
+<p>The shared region contains a text or simple binary value, a nil
representation, or inductively, a sequence of terms.
+When the term is an element of complex type, an element unused region can
follow (depending on the length kind),
+and if the term is a choice with dfdl:choiceLengthKind='explicit' then a
choice unused region can follow.</p>
+</div>
+<div class="paragraph">
+<p>Then, after the shared part, the shared-after region has these
subregions:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>term’r MTA - mandatory text alignment for the terminator.</p>
+</li>
+<li>
+<p>term’r - terminator</p>
+</li>
+<li>
+<p>tSkip - trailing skip region. This is fixed length (commonly 0)</p>
+</li>
+<li>
+<p>postfix sep’r MTA - - mandatory text alignment for a separator in
postfix position.
+When a separator is defined, this region is populated with up to 7 bits to
+obtain text alignment for the characters of the separator in the specified
charset encoding.
+Note that this encoding and that of the term’s initator/terminator are
not necessarily the same encoding.</p>
+</li>
+<li>
+<p>postfix sep’r - separator when defined and in postfix position.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>The terminator MTA region varies in size based on its starting
+alignment, and so it depends on the size of the shared region.</p>
+</div>
+<div class="paragraph">
+<p>The postfix separator MTA region similarly varies in size based on where the
+tSkip region ended, and so it depends on the size of the shared region, and
+the sizes of the terminator MTA region, terminator, and tSkip regions.</p>
+</div>
+<div class="admonitionblock note">
+<table>
+<tr>
+<td class="icon">
+<img src="/images/icons/note.png" alt="Note">
+</td>
+<td class="content">
+The shared-after region does not have any sub-regions the size
+of which depend on the context.
+However, we separate it from the central shared region in order to
+separate all concerns around alignment and fill into separate regions from the
+central shared region, which could perhaps be called the shared
+content region.
+The central shared region therefore does not deal with alignment at all.
+</td>
+</tr>
+</table>
+</div>
+<div class="paragraph">
+<p>The central idea here is that all the regions to the left (before) the
known alignment point are context dependent.
+That is, whether these alignment fill and mandatory text alignment regions can
be statically determined
+in size is dependent on where the prior term ended.</p>
+</div>
+<div class="paragraph">
+<p>In contrast, the known alignment point is a fresh start for alignment. The
alignment as required by the alignment properties
+will have been achieved at that point. Hence, from there and to the right
(after), everything can be assessed relative to
+this new fresh anchor.</p>
+</div>
+<div class="paragraph">
+<p>Inductively, even the context dependent regions are not dependent on terms
very far back.
+Let’s consider the regions in a straddle of two adjacent terms in a
sequence:</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/diag-e8cdc9d803fb5e074386a0e096177241.png" alt="diag
e8cdc9d803fb5e074386a0e096177241" width="1030" height="378">
+</div>
+</div>
+<div class="paragraph">
+<p>The Known Alignment Point 2 is only context dependent on the possible prior
terms back to Known Alignment Point 1.
+This is true certaintly when Term 1 is a scalar element or a model-group
itself.</p>
+</div>
+<div class="paragraph">
+<p>In reality the induction here is more complex because Term 1 might be an
optional element or an array, in which case the prior term before Term 1 may
also be involved
+in computing the region sizes for the Term 2 unique-before regions.
+But this never needs to consider anything further back than the prior scalar
term, or the start of the shared region of the enclosing model group, which need
+only be considered if Term 2 can be first within its enclosing model group.</p>
+</div>
+<div class="admonitionblock caution">
+<table>
+<tr>
+<td class="icon">
+<img src="/images/icons/caution.png" alt="Caution">
+</td>
+<td class="content">
+Separators may be present even if terms are not, based on
dfdl:separatorSuppressionPolicy and a few other properties.
+The diagrams show sequence before region as if it was part of the term, but
really it may appear even if the term is absent.
+</td>
+</tr>
+</table>
+</div>
+<div class="paragraph">
+<p>Given the above, for any term, we can compute its PropEnv, and create
+a distinct shared instance for that term for each unique PropEnv.</p>
+</div>
+<div class="paragraph">
+<p>Since the shared region contains all the children of any
+child-containing term, sharing this insures that the schema
+compilation space/speed is proportional to, roughly, the size of the
+schema without any non-constant multiplicative factor.
+This insures no combinatorial explosion of compilation time.</p>
+</div>
+</div>
+</div>
+<div class="sect2">
+<h3 id="optimizing-alignment-regions">Optimizing Alignment Regions</h3>
+<div class="paragraph">
+<p>A primary task of the schema compiler is to eliminate alignment fill
+(and mandatory text alignment aka MTA) regions that are unnecessary
+because the data can be proven to always be properly aligned.</p>
+</div>
+<div class="paragraph">
+<p>This is done by computing, for each region, a compile-time alignment
approximation.
+The AlignmentMultipleOf class represents this information.
+AlignmentMultipleOf objects form a mathematical lattice where perfect
+alignment is the bottom of the lattice, no alignment (or alignment
+multiple of 1 bit) is the top of the lattice, and various multiples
+(8, 32, etc.) live in between. For two lattice values a and b, we say
+a < b (a is 'weaker than' b) if a is a multiple of b.</p>
+</div>
+<div class="paragraph">
+<p>A key observation is this: Each time we create an alignment constraint
+for a shared region, this is not dependent on any other constraint.
+That is, the constraints on alignment do not propagate from term to term.
+A shared region can assume it starts at the alignment that is required
+for the term based on the dfdl:alignment property, the initiator MTA
+if there is an initiator, and the value MTA for text-representation
+simple values.</p>
+</div>
+<div class="paragraph">
+<p>Hence, terms that appear down-stream of any shared region do
<strong>not</strong>
+depend on any constraints propagating from before the shared region.
+This greatly reduces the complexity and the distance across the schema
+that constraint-changes propagate.
+It also insures there is no need for an iterative contraint solver, since
nothing
+about alignment propagates from one term to the next.
+Terms are separated by the fact that the shared region of a term starts from a
+known alignment (specified by its alignment and alignment units
+property).</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="other-context-dependent-computations">Other Context-Dependent
Computations</h3>
+<div class="paragraph">
+<p>By dividing up the representation of a term into the unique
context-dependent
+and shared context-indepenent parts explicitly we provide a home for the
various
+calculations the schema compiler performs in order to do other
optimizations.</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="/images/unshared-shared.png" alt="unshared shared" width="587"
height="237">
+</div>
+</div>
+<div class="paragraph">
+<p>In the above class diagram, members that must be computed on the unshared
term object are
+shown in the class.
+This set of things should always be minimized.
+That is, as few things should be computed on unshared term objects as posible.
+It is assumed that everything else is computed on the shared term object, but
members
+specifically about regions discussed in sections above are called out.</p>
+</div>
+<div class="paragraph">
+<p>The UnsharedTerm object is effectively state of the enclosing model group
DSOM object.
+It exists in 1 to 1 correspondence (aka it is "owned" by the enclosing model
group object.)</p>
+</div>
+<div class="paragraph">
+<p>In the Runtime 1 backend, a processor (parser/unparser) is generated for
the shared term object,
+and that processor is referenced from the processor generated for the unshared
term object.</p>
+</div>
+</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="transition-plan">Transition Plan</h2>
+<div class="sectionbody">
+<div class="admonitionblock important">
+<table>
+<tr>
+<td class="icon">
+<img src="/images/icons/important.png" alt="Important">
+</td>
+<td class="content">
+Once implemented this section can be deleted as it is describing transition
from an older Daffodil version to one that implements the shared objects.
+</td>
+</tr>
+</table>
+</div>
+<div class="paragraph">
+<p>We can examine several of the context-dependent calculations below.</p>
+</div>
+<div class="sect3">
+<h4
id="isalignmentfillneeded-isinitiatormtaneeded-isterminatormtaneeded-isprefixinfixseparatormtaneeded-ispostfixseparatormtaneeded">isAlignmentFillNeeded,
isInitiatorMTANeeded, isTerminatorMTANeeded, isPrefixInfixSeparatorMTANeeded,
isPostfixSeparatorMTANeeded</h4>
+<div class="paragraph">
+<p>These are the output of the analysis of alignments, and they turn on/off
the presence of these regions.
+These are of particular importance to a streaming unparser since they can
require suspension of unparsing until alignment can be determined at
runtime.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="ispotentiallytrailing">isPotentiallyTrailing</h4>
+<div class="paragraph">
+<p>This is context dependent because a given term definition can be reused in
multiple contexts, some of which where it is potentially trailing, and others
of which where it is not.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="needsbitorderchange">needsBitOrderChange</h4>
+<div class="paragraph">
+<p>This is used to optimize out suspensions in the unparser. It is context
dependent because whether the bit order is changing when the processor arrives
on a term depends on what preceded it in the processing.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="isscanningfordelimiter">isScanningForDelimiter</h4>
+<div class="paragraph">
+<p>This is used to determine whether a delimiter stack combinator is used as
part of the grammar. It is context dependent because a separator or terminator
can be defined on an enclosing model group.</p>
+</div>
+<div class="paragraph">
+<p>TBD: It’s possible we don’t need to wrap this combinator around
elements. Manipulating the delimiter stack should only be necessary when
delimiters go into and out of scope. So that would mean once per model group,
not per term within the model group.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="isscanningforterminator">isScanningForTerminator</h4>
+<div class="paragraph">
+<p>This determines whether the schema compiler will allow the entity "%ES;" as
a runtime value for the expression value of a dfdl:terminator property.
+Specifically if we are scanning for a terminator then the "%ES;" entity is
disallowed.</p>
+</div>
+<div class="paragraph">
+<p>This is context dependent because the terminator may be defined on an
enclosing model group, not on an element that is within that model group.
+The element may have a position in the model group such that it may be last,
and in that case if it has a length kind that does not use the terminator
(e.g., dfdl:lengthKind='pattern') then that terminator is not being scanned
for, and so it is allowable for it to be empty string at runtime.</p>
+</div>
+<div class="paragraph">
+<p>If this computation is done on the unshared term, it must inquire about
many characteristics of the enclosing model group.
+It may be that this computation is best done on the model-group object.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="initiatedcontent-check">initiatedContent Check</h4>
+<div class="paragraph">
+<p>When a model group has the dfdl:initiatedContent="yes" property, all
represented term children must have a defined initiator.</p>
+</div>
+<div class="paragraph">
+<p>This is context dependent because a global element declaration or a global
group definition may define an initiator, but the model group with initiated
content may refer to these via an element reference or group reference.
+That is, the relationship is not one of lexical enclosure.
+The check must be done as part of the unshared term, or on the enclosing model
group itself.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="refactoring-of-primary-dsom-classes">Refactoring of Primary DSOM
Classes</h3>
+<div class="paragraph">
+<p>Implementing this sharing requires splitting the DSOM Term clases each into
a unique and a shared part.</p>
+</div>
+<div class="paragraph">
+<p>This will be done by renaming each Term to SharedTerm, and introducing
UniqueTerm.</p>
+</div>
+<div class="paragraph">
+<p>For example ElementBase will be renamed to SharedElementBase, and new
abstract class UniqueElementBase will be created.</p>
+</div>
+<div class="paragraph">
+<p>This will occur uniformly across the DSOM Term subclasses/traits.</p>
+</div>
+<div class="paragraph">
+<p>Some of the mixins to these classes will go on only one or the other of the
shared or unique parts.</p>
+</div>
+<div class="paragraph">
+<p>Many of the mixins will have to split into a mixin for the shared part and
another mixin for the unique part.
+One example of this is the AlignedMixin. This currently deals with alignment
regions that are found in unique parts and shared parts, and those
+methods must go on different objects.</p>
+</div>
+<div class="paragraph">
+<p>The unique term classes will utilize a central memoizing factory to
allocate or find shared term instances based on having equivalent PropEnvs.</p>
+</div>
+</div>
+<div class="sect2">
+<h3 id="testing-notes">Testing Notes</h3>
+<div class="sect3">
+<h4 id="compiler-instrumentation">Compiler Instrumentation</h4>
+<div class="paragraph">
+<p>Some standard compiler instrumentation that is dumped out based on options
to daffodil compiler, to include:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>counters of number of distinct shared terms for each global def/decl.</p>
+</li>
+<li>
+<p>counters of number of inbound references to each shared term.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>The point is to have metrics that can be compared and which will make it
clear if under maintenance the sharing is somehow lost and we’re getting
bad compile-time behavior again.</p>
+</div>
+</div>
+</div>
+</div>
+</div>
+ </div>
+</div>
+
+
+ <footer>
+ <footer class="site-footer">
+ <div class="wrapper">
+ <div class="footer-col-wrapper" style="font-size: .85em;">
+ <hr>
+ <div class="container">
+ <div class="row">
+ <div class="col-xs-3" style="margin-top: 15px;">
+ <a href="https://incubator.apache.org";><img
src="/assets/themes/apache/img/incubator_feather_egg_logo.png"
+ alt="Apache
Incubator" style="width:100%;"/></a>
+ </div>
+ <div class="col-xs-9">
+ Apache Daffodil is an effort undergoing <a
href="https://incubator.apache.org/index.html";>Incubation</a>
+ at The Apache Software Foundation (ASF), sponsored by
the Incubator. Incubation is required of all newly
+ accepted projects until a further review indicates
that the infrastructure, communications, and decision
+ making process have stabilized in a manner consistent
with other successful ASF projects. While incubation
+ status is not necessarily a reflection of the
completeness or stability of the code, it does indicate that
+ the project has yet to be fully endorsed by the ASF.
+ </div>
+ </div>
+ </div>
+ <hr>
+ <div>
+ <div style="text-align: center;">
+ Copyright © 2020 <a href="https://www.apache.org";>The
Apache Software Foundation</a>.
+ Licensed under the <a
href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version
+ 2.0</a>.
+ <br>
+ Apache, the Apache Incubator project logo, Apache
Daffodil, Daffodil, and the
+ Apache Daffodil logo are trademarks of The Apache Software
Foundation.
+ </div>
+ </div>
+ </div>
+ </div>
+</footer>
+
+ </footer>
+ </div>
+
+ <script src="/assets/themes/apache/jquery/jquery-2.1.1.min.js"></script>
+
+ <script src="/assets/themes/apache/bootstrap/js/bootstrap.min.js"></script>
+
+
+ </body>
+</html>
+
diff --git a/content/images/diag-1313a13d91a6f8c668ade63f72bbb61d.png
b/content/images/diag-1313a13d91a6f8c668ade63f72bbb61d.png
new file mode 100644
index 0000000..2125a13
Binary files /dev/null and
b/content/images/diag-1313a13d91a6f8c668ade63f72bbb61d.png differ
diff --git a/content/images/diag-19f9d885d07ef92b8922b13dfaa30484.png
b/content/images/diag-19f9d885d07ef92b8922b13dfaa30484.png
new file mode 100644
index 0000000..67a4d02
Binary files /dev/null and
b/content/images/diag-19f9d885d07ef92b8922b13dfaa30484.png differ
diff --git a/content/images/diag-45bcd5e9c7484340ba070e865b4dd7ed.png
b/content/images/diag-45bcd5e9c7484340ba070e865b4dd7ed.png
new file mode 100644
index 0000000..f014c57
Binary files /dev/null and
b/content/images/diag-45bcd5e9c7484340ba070e865b4dd7ed.png differ
diff --git a/content/images/diag-51b412adf272104af315f2f6958aeb2d.png
b/content/images/diag-51b412adf272104af315f2f6958aeb2d.png
new file mode 100644
index 0000000..a63df17
Binary files /dev/null and
b/content/images/diag-51b412adf272104af315f2f6958aeb2d.png differ
diff --git a/content/images/diag-a233f3edc65e7c991774d9233c23f186.png
b/content/images/diag-a233f3edc65e7c991774d9233c23f186.png
new file mode 100644
index 0000000..4ce1b06
Binary files /dev/null and
b/content/images/diag-a233f3edc65e7c991774d9233c23f186.png differ
diff --git a/content/images/diag-e8cdc9d803fb5e074386a0e096177241.png
b/content/images/diag-e8cdc9d803fb5e074386a0e096177241.png
new file mode 100644
index 0000000..a90e69d
Binary files /dev/null and
b/content/images/diag-e8cdc9d803fb5e074386a0e096177241.png differ
diff --git a/content/images/test_packet_diagram_1.svg
b/content/images/test_packet_diagram_1.svg
deleted file mode 100644
index 408f441..0000000
--- a/content/images/test_packet_diagram_1.svg
+++ /dev/null
@@ -1,181 +0,0 @@
-<?xml version='1.0' encoding='UTF-8'?>
-<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN"
"http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd";>
-<svg viewBox="0 0 896 312" xmlns="http://www.w3.org/2000/svg";
xmlns:inkspace="http://www.inkscape.org/namespaces/inkscape";
xmlns:xlink="http://www.w3.org/1999/xlink";>
- <defs id="defs_block">
- <filter height="1.504" id="filter_blur" inkspace:collect="always"
width="1.1575" x="-0.07875" y="-0.252">
- <feGaussianBlur id="feGaussianBlur3780" inkspace:collect="always"
stdDeviation="4.2" />
- </filter>
- </defs>
- <title>blockdiag</title>
- <desc>packetdiag {
- colwidth = 32
- node_height = 24
- 0-15: Source Port
- 16-31: Destination Port
- 32-63: Sequence Number
- 64-95: Acknowledgment Number
- 96-99: Data Offset
- 100-105: Reserved
- 106: URG [rotate = 270]
- 107: ACK [rotate = 270]
- 108: PSH [rotate = 270]
- 109: RST [rotate = 270]
- 110: SYN [rotate = 270]
- 111: FIN [rotate = 270]
- 112-127: Window
- 128-143: Checksum
- 144-159: Urgent Pointer
- 160-191: (Options and Padding)
- 192-223: data [colheight = 2]
-}</desc>
- <path d="M 64 48 L 64 80" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="6"
x="64.0" y="42">0</text>
- <path d="M 88 64 L 88 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 112 64 L 112 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 136 64 L 136 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 160 64 L 160 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 184 64 L 184 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 208 64 L 208 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 232 64 L 232 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 256 48 L 256 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 280 64 L 280 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 304 64 L 304 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 328 64 L 328 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 352 64 L 352 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 376 64 L 376 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 400 64 L 400 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 424 64 L 424 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 48 L 448 80" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="12"
x="448.0" y="42">16</text>
- <path d="M 472 64 L 472 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 496 64 L 496 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 520 64 L 520 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 544 64 L 544 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 568 64 L 568 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 592 64 L 592 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 616 64 L 616 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 640 48 L 640 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 664 64 L 664 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 688 64 L 688 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 712 64 L 712 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 736 64 L 736 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 760 64 L 760 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 784 64 L 784 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 808 64 L 808 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 48 L 832 80" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="12"
x="832.0" y="42">32</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="384" x="64" y="80" />
- <path d="M 64 80 L 448 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 80 L 448 104" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 104 L 64 104" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 104 L 64 80" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="66"
x="256.0" y="98">Source Port</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="384" x="448" y="80" />
- <path d="M 448 80 L 832 80" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 80 L 832 104" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 104 L 448 104" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 104 L 448 80" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="96"
x="640.0" y="98">Destination Port</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="768" x="64" y="104" />
- <path d="M 64 104 L 832 104" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 104 L 832 128" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 128 L 64 128" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 128 L 64 104" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="90"
x="448.0" y="122">Sequence Number</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="768" x="64" y="128" />
- <path d="M 64 128 L 832 128" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 128 L 832 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 152 L 64 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 152 L 64 128" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="126"
x="448.0" y="146">Acknowledgment Number</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="96" x="64" y="152" />
- <path d="M 64 152 L 160 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 160 152 L 160 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 160 176 L 64 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 176 L 64 152" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="66"
x="112.0" y="170">Data Offset</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="144" x="160" y="152" />
- <path d="M 160 152 L 304 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 304 152 L 304 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 304 176 L 160 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 160 176 L 160 152" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="48"
x="232.0" y="170">Reserved</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="24" x="304" y="152" />
- <path d="M 304 152 L 328 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 328 152 L 328 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 328 176 L 304 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 304 176 L 304 152" fill="none" stroke="rgb(0,0,0)" />
- <g transform="rotate(270,304,176)">
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="18"
x="316.0" y="194">URG</text>
- </g>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="24" x="328" y="152" />
- <path d="M 328 152 L 352 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 352 152 L 352 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 352 176 L 328 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 328 176 L 328 152" fill="none" stroke="rgb(0,0,0)" />
- <g transform="rotate(270,328,176)">
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="18"
x="340.0" y="194">ACK</text>
- </g>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="24" x="352" y="152" />
- <path d="M 352 152 L 376 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 376 152 L 376 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 376 176 L 352 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 352 176 L 352 152" fill="none" stroke="rgb(0,0,0)" />
- <g transform="rotate(270,352,176)">
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="18"
x="364.0" y="194">PSH</text>
- </g>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="24" x="376" y="152" />
- <path d="M 376 152 L 400 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 400 152 L 400 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 400 176 L 376 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 376 176 L 376 152" fill="none" stroke="rgb(0,0,0)" />
- <g transform="rotate(270,376,176)">
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="18"
x="388.0" y="194">RST</text>
- </g>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="24" x="400" y="152" />
- <path d="M 400 152 L 424 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 424 152 L 424 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 424 176 L 400 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 400 176 L 400 152" fill="none" stroke="rgb(0,0,0)" />
- <g transform="rotate(270,400,176)">
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="18"
x="412.0" y="194">SYN</text>
- </g>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="24" x="424" y="152" />
- <path d="M 424 152 L 448 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 152 L 448 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 176 L 424 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 424 176 L 424 152" fill="none" stroke="rgb(0,0,0)" />
- <g transform="rotate(270,424,176)">
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="18"
x="436.0" y="194">FIN</text>
- </g>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="384" x="448" y="152" />
- <path d="M 448 152 L 832 152" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 152 L 832 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 176 L 448 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 176 L 448 152" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="36"
x="640.0" y="170">Window</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="384" x="64" y="176" />
- <path d="M 64 176 L 448 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 176 L 448 200" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 200 L 64 200" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 200 L 64 176" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="48"
x="256.0" y="194">Checksum</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="384" x="448" y="176" />
- <path d="M 448 176 L 832 176" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 176 L 832 200" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 200 L 448 200" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 448 200 L 448 176" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="84"
x="640.0" y="194">Urgent Pointer</text>
- <rect fill="rgb(255,255,255)" height="24" stroke="rgb(255,255,255)"
width="768" x="64" y="200" />
- <path d="M 64 200 L 832 200" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 200 L 832 224" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 224 L 64 224" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 224 L 64 200" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="126"
x="448.0" y="218">(Options and Padding)</text>
- <rect fill="rgb(255,255,255)" height="48" stroke="rgb(255,255,255)"
width="768" x="64" y="224" />
- <path d="M 64 224 L 832 224" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 224 L 832 272" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 832 272 L 64 272" fill="none" stroke="rgb(0,0,0)" />
- <path d="M 64 272 L 64 224" fill="none" stroke="rgb(0,0,0)" />
- <text fill="rgb(0,0,0)" font-family="sans-serif" font-size="11"
font-style="normal" font-weight="normal" text-anchor="middle" textLength="24"
x="448.0" y="254">data</text>
-</svg>
diff --git a/content/images/unshared-shared.png
b/content/images/unshared-shared.png
new file mode 100644
index 0000000..0f41f52
Binary files /dev/null and b/content/images/unshared-shared.png differ
