[jira] [Assigned] (DAFFODIL-1975) Incorrect separator behavior - missing separator for empty sequence (non-backward compatible)
[
https://issues.apache.org/jira/browse/DAFFODIL-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle reassigned DAFFODIL-1975:
--
Assignee: Michael Beckerle
> Incorrect separator behavior - missing separator for empty sequence
> (non-backward compatible)
> -
>
> Key: DAFFODIL-1975
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1975
> Project: Daffodil
> Issue Type: Bug
> Components: Back End, Compatibility
>Affects Versions: 2.1.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Minor
>
> Consider the following
> {{
>
>
>
>
> http://www.ogf.org/dfdl/;>
> {fn:true()}
>
>
>
>maxOccurs="unbounded" dfdl:textNumberPattern="#"
>dfdl:occursCountKind="implicit" />
>
>
> }}
> The sequence labeled INNER has no syntax, only an assertion. However, the
> OUTER sequence specifies infix separator.
> Per clarifying discussion on the DFDL workgroup mailing list, and tests with
> IBM DFDL, daffodil behavior is wrong if it suppresses the separator after the
> INNER sequence. Separator suppression applies only to "optional" children,
> not required children. All model group instances are considered required,
> even if empty.
> Hence, valid data to match the above schema looks like
> ,1,2,3
> The leading comma there is required.
> Daffodil must be doing something clever to detect that this initial sequence
> is empty and so is suppressing the separator for it. But this is incorrect.
> Test test_NumSeq_13a illustrates the bug. (Test added as part of sequence and
> separators refactoring 2018-08-01)
> Fixing this may cause many tests to break. This is a major
> non-backward-compatible change, and so should be fixed carefully. Schemas
> outside of daffodil for delimited formats may be impacted also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DAFFODIL-861) textStandardInfinityRep, textStandardNaNRep, and textStandardExponentRep with dfdl:ignoreCase="yes" is still case sensitive
[ https://issues.apache.org/jira/browse/DAFFODIL-861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661455#comment-16661455 ] Michael Beckerle commented on DAFFODIL-861: --- We should just document this limitation and then park this bug indefinitely. Unless ICU implements this, it will always be a limitation. > textStandardInfinityRep, textStandardNaNRep, and textStandardExponentRep with > dfdl:ignoreCase="yes" is still case sensitive > --- > > Key: DAFFODIL-861 > URL: https://issues.apache.org/jira/browse/DAFFODIL-861 > Project: Daffodil > Issue Type: Bug > Components: Back End, DFDL Language, ICU >Affects Versions: s11 >Reporter: Elizabeth Fahl >Priority: Minor > > textStandardInfinityRep, textStandardNaNRep, and textStandardExponentRep are > not ignoring case when dfdl:ignoreCase="yes". > See test "test_infnanCaseInsensitive" in > daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section13/text_number_props/TestTextNumberPropsDebug.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-861) textStandardInfinityRep, textStandardNaNRep, and textStandardExponentRep with dfdl:ignoreCase="yes" is still case sensitive
[ https://issues.apache.org/jira/browse/DAFFODIL-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-861: -- Priority: Minor (was: Major) > textStandardInfinityRep, textStandardNaNRep, and textStandardExponentRep with > dfdl:ignoreCase="yes" is still case sensitive > --- > > Key: DAFFODIL-861 > URL: https://issues.apache.org/jira/browse/DAFFODIL-861 > Project: Daffodil > Issue Type: Bug > Components: Back End, DFDL Language, ICU >Affects Versions: s11 >Reporter: Elizabeth Fahl >Priority: Minor > > textStandardInfinityRep, textStandardNaNRep, and textStandardExponentRep are > not ignoring case when dfdl:ignoreCase="yes". > See test "test_infnanCaseInsensitive" in > daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section13/text_number_props/TestTextNumberPropsDebug.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1097) Need SDE (and someday warning) from ambiguous path expression
[ https://issues.apache.org/jira/browse/DAFFODIL-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1097: --- Summary: Need SDE (and someday warning) from ambiguous path expression (was: Possible to create ambiguous path expression) > Need SDE (and someday warning) from ambiguous path expression > - > > Key: DAFFODIL-1097 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1097 > Project: Daffodil > Issue Type: Bug > Components: General >Reporter: Jessie Chab >Priority: Major > > I was able to create a schema where three elements on the same level had the > same name. When passing the ambiguous path into the fn:local-name() function, > the function believed it was being passed a multi-item sequence, since three > elements exist with this name. Since the elements are not optional and they > are not part of a choice, there should not be any ambiguity when parsing. > However, part of our reasoning for moving away from XPath was that we do not > need to traverse an entire tree to find all matching nodes for a path, > because there will be at most one match. That is not the case in this > scenario. > See test local_name_06 in the following files: > daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section23/dfdl_functions/Functions.tdml > daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section23/dfdl_expressions/TestDFDLExpressionsDebug.scala > daffodil-test/src/test/scala/edu/illinois/ncsa/daffodil/section23/dfdl_expressions/TestDFDLExpressions.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1056) DPath: Decreased precision - Compile Time vs Runtime SDE
[ https://issues.apache.org/jira/browse/DAFFODIL-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1056: --- Priority: Trivial (was: Major) > DPath: Decreased precision - Compile Time vs Runtime SDE > > > Key: DAFFODIL-1056 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1056 > Project: Daffodil > Issue Type: Bug > Components: Diagnostics, Usability >Affects Versions: s15 >Reporter: Jessie Chab >Priority: Trivial > > From Mike: Yeah, this is a decrease of precision, but the new DPath > implementation changes came with some loss of proper categorization. > Basically, some areas where the code was clearly compilation time or clearly > runtime have been replaced by common code that is used in both cases, and > that code issues an SDE. Nothing recaptures this and converts it into a > "Runtime" SDE currently. > So I'd say it's a bug that we've lost this precision, because I like having > the separation where SDEs that are detected to occur at runtime are > explicitly identified. But the DFDL spec doesn't say when anything has to be > detected. > So if we view the test as testing the DFDL spec requirement, then making the > test less precise is ok. If we view the test as verifying our desired behavor > for Daffodil, then the loss of precision is a bug. > I suppose really there are two tests then I would prefer not to fix this > particular issue right away however. > Changes in diagnostic behavior are probably the biggest observable places > where behavior will have changed > Associated test(s): > test_variableRefError -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1056) DPath: Decreased precision - Compile Time vs Runtime SDE
[ https://issues.apache.org/jira/browse/DAFFODIL-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661445#comment-16661445 ] Michael Beckerle commented on DAFFODIL-1056: To clarify: We used to get "Runtime Schema Definition Error", and now we get just "Schema Definition Error" eventhough it is detected at runtime. The term "Runtime" has been lost because code is being shared across compile and runtime, so the distinction is being lost. Very minor issue. > DPath: Decreased precision - Compile Time vs Runtime SDE > > > Key: DAFFODIL-1056 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1056 > Project: Daffodil > Issue Type: Bug > Components: Diagnostics, Usability >Affects Versions: s15 >Reporter: Jessie Chab >Priority: Major > > From Mike: Yeah, this is a decrease of precision, but the new DPath > implementation changes came with some loss of proper categorization. > Basically, some areas where the code was clearly compilation time or clearly > runtime have been replaced by common code that is used in both cases, and > that code issues an SDE. Nothing recaptures this and converts it into a > "Runtime" SDE currently. > So I'd say it's a bug that we've lost this precision, because I like having > the separation where SDEs that are detected to occur at runtime are > explicitly identified. But the DFDL spec doesn't say when anything has to be > detected. > So if we view the test as testing the DFDL spec requirement, then making the > test less precise is ok. If we view the test as verifying our desired behavor > for Daffodil, then the loss of precision is a bug. > I suppose really there are two tests then I would prefer not to fix this > particular issue right away however. > Changes in diagnostic behavior are probably the biggest observable places > where behavior will have changed > Associated test(s): > test_variableRefError -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1016) CLI: root/namespace option should recognize if only namespace is provided
[
https://issues.apache.org/jira/browse/DAFFODIL-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle updated DAFFODIL-1016:
---
Priority: Trivial (was: Major)
> CLI: root/namespace option should recognize if only namespace is provided
> -
>
> Key: DAFFODIL-1016
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1016
> Project: Daffodil
> Issue Type: Bug
> Components: CLI, Diagnostics
>Affects Versions: s14
>Reporter: Jessie Chab
>Priority: Trivial
>
> With the old -r and -n options, we had a test where you specify -n and a
> namespace without specifying a root. The CLI would complain.
> WIth the new functionality, if you provide a namespace and no root, the CLI
> tries to use the namespace as the root.
> eg:
> echo -ne "12" | ./daffodil-cli/target/start parse -s
> daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section06/entities/charClassEntities.dfdl.xsd
> -r "{http://www.example.org/example1/};
> I would expect this to recognize that the information enclosed in {} is the
> namespace, and no root is provided. However, we get the following error:
> [error] Compilation (ProcessorFactory) produced 1 errors/warnings.
> [error] Schema Definition Error: No root element found for
> {http://www.example.org/example1/} in any available namespace
> Schema context: SchemaSet. Location in
> daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section06/entities/charClassEntities.dfdl.xsd.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DAFFODIL-920) Abort error occurs with nested references
[ https://issues.apache.org/jira/browse/DAFFODIL-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661434#comment-16661434 ] Michael Beckerle commented on DAFFODIL-920: --- Dave, can you get the details off fogbugz that are needed to understand this issue and append them here as comments? I can't even seem to get fogbugz to respond (must be new name/address since I last used it.) > Abort error occurs with nested references > - > > Key: DAFFODIL-920 > URL: https://issues.apache.org/jira/browse/DAFFODIL-920 > Project: Daffodil > Issue Type: Bug > Components: General >Reporter: Elizabeth Fahl >Assignee: Dave Thompson >Priority: Major > > When using a hiddenGroupRef to reference an element that has nested > references an Abort exception was thrown. > See fogbugz-21038 for more information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-920) Abort error occurs with nested references
[ https://issues.apache.org/jira/browse/DAFFODIL-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reassigned DAFFODIL-920: - Assignee: Dave Thompson > Abort error occurs with nested references > - > > Key: DAFFODIL-920 > URL: https://issues.apache.org/jira/browse/DAFFODIL-920 > Project: Daffodil > Issue Type: Bug > Components: General >Reporter: Elizabeth Fahl >Assignee: Dave Thompson >Priority: Major > > When using a hiddenGroupRef to reference an element that has nested > references an Abort exception was thrown. > See fogbugz-21038 for more information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-866) Need clarification between canProceed vs isError
[ https://issues.apache.org/jira/browse/DAFFODIL-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661420#comment-16661420 ] Michael Beckerle commented on DAFFODIL-866: --- Let's just drop canProceed which is just !isError. Deprecate canProceed, and throw illegalStateException if you have not called onPath first. > Need clarification between canProceed vs isError > > > Key: DAFFODIL-866 > URL: https://issues.apache.org/jira/browse/DAFFODIL-866 > Project: Daffodil > Issue Type: Bug > Components: API >Reporter: Steve Lawrence >Priority: Major > Labels: beginner > > This two functions are part of the public API. This is confusing for API > users. Currently, there is not difference between the two functions. We > either need to remove one of the functions of clarify the distinctions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-866) Need clarification between canProceed vs isError
[ https://issues.apache.org/jira/browse/DAFFODIL-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-866: -- Labels: beginner (was: ) > Need clarification between canProceed vs isError > > > Key: DAFFODIL-866 > URL: https://issues.apache.org/jira/browse/DAFFODIL-866 > Project: Daffodil > Issue Type: Bug > Components: API >Reporter: Steve Lawrence >Priority: Major > Labels: beginner > > This two functions are part of the public API. This is confusing for API > users. Currently, there is not difference between the two functions. We > either need to remove one of the functions of clarify the distinctions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-598) No validation of Short Form or Element Form dfdl properties
[ https://issues.apache.org/jira/browse/DAFFODIL-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-598: -- Labels: beginner (was: ) > No validation of Short Form or Element Form dfdl properties > --- > > Key: DAFFODIL-598 > URL: https://issues.apache.org/jira/browse/DAFFODIL-598 > Project: Daffodil > Issue Type: Bug > Components: Front End, Usability >Affects Versions: s8 >Reporter: Michael Beckerle >Priority: Major > Labels: beginner > > We had to change the Schema for DFDL Schemas so that it would tolerate > non-native attributes (in order to fix DFDL-558). > This means the validator is no longer checking short form property bindings > for correctness. > So we need to implement our own code so that every annotated schema component > inspects any short-form properties on it, and checks that they are valid. > We can check some things centrally, i.e., is there a prefix bound to the DFDL > namespace at all? > Stubs for this are in SchemaComponent.scala in the definition of > AnnotatedSchemaComponent class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-80) Need to check for type, element, group recursion
[ https://issues.apache.org/jira/browse/DAFFODIL-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-80: - Priority: Minor (was: Major) > Need to check for type, element, group recursion > > > Key: DAFFODIL-80 > URL: https://issues.apache.org/jira/browse/DAFFODIL-80 > Project: Daffodil > Issue Type: Bug > Components: Front End, General >Reporter: Michael Beckerle >Priority: Minor > > DFDL v1.0 prohibits definition of recursive "types". Recursive schemas can > actually be created by way of elements with types, element references, or > group references. > A check needs to be in place to issue an SDE if recursive schema definition > is detected. > (As of creation of this bug, nothing is checking for this. Need to create > test cases that plumb recursion through all/many of the various ways it can > be created, and be sure the SDE is detected.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-852) Implement runtime checks for lastDeclaredRequiredElement
[ https://issues.apache.org/jira/browse/DAFFODIL-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reassigned DAFFODIL-852: - Assignee: Michael Beckerle > Implement runtime checks for lastDeclaredRequiredElement > > > Key: DAFFODIL-852 > URL: https://issues.apache.org/jira/browse/DAFFODIL-852 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Diagnostics >Affects Versions: s11 >Reporter: Josh Adams >Assignee: Michael Beckerle >Priority: Major > > This is primarily an issue for cases where an element is > maxOccurs="unbounded" occursCountKind="implicit", as this is only allowed on > the last required element of a sequence (see DFDL-732) > So far I believe that we have the compile time checks completed. However, > there are cases where it cannot be determined whether or not an element is > required at compile time. For example: > - unbounded/implicit element followed by a choice where a required element > is selected at runtime (see > edu.illinois.ncsa.daffodil.section14.occursCountKind.TestOCKImplicitNew > test_ockImplicit12) > - unbounded/implicit element followed by an element where lengthKind=pattern > and that pattern can match an empty string. (see > edu.illinois.ncsa.daffodil.section14.occursCountKind.TestOCKImplicitNew > test_ockImplicit14) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1717) Use DPath expressions on final result infoset from parse
[ https://issues.apache.org/jira/browse/DAFFODIL-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1717: --- Priority: Minor (was: Major) > Use DPath expressions on final result infoset from parse > > > Key: DAFFODIL-1717 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1717 > Project: Daffodil > Issue Type: New Feature > Components: API, Back End >Reporter: Michael Beckerle >Priority: Minor > > Users are converting the DFDL infoset to JDOM or w3c. DOM trees just so they > can issue simple XPath expressions against the infoset. > In many cases a DPath expression on the original Daffodil infoset would work > fine, and would save a complete traversal and copy of the infoset. > So we should support DPath expressions on the result infoset via our Java and > Scala APIs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1742) TIFF - need DFDL schema and approach to TIFF
[ https://issues.apache.org/jira/browse/DAFFODIL-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661408#comment-16661408 ] Michael Beckerle commented on DAFFODIL-1742: This is a major feature requirement for DFDL v2.0 - TIFF is base+offset oriented. The format can't be parsed end to end. Each record gives an offset from the end of a header, to the next record, so they can, literally, be backwards in the file - one must "seek" to parse this. It is important that any approach to this is (a) declarative (b) invertible - unparsing should also work - resulting in each record following the previous one, but these offsets from the base have to get computed automatically much the way stored lengths do (via outputValueCalc). > TIFF - need DFDL schema and approach to TIFF > > > Key: DAFFODIL-1742 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1742 > Project: Daffodil > Issue Type: New Feature > Components: DFDL Language, DFDL Schemas >Reporter: Michael Beckerle >Priority: Minor > > TIFF image format is not something DFDL v1.0 can describe. > Nevertheless we need an approach that illustrates how to deal with it. > I *think* this means a preprocessing program, and then a DFDL schema to turn > the result of the preprocessor into an Infoset. For unparsing, a > post-processor also. > The right approach remains to be seen, but we need to show how to deal with > TIFF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1742) TIFF - need DFDL schema and approach to TIFF
[ https://issues.apache.org/jira/browse/DAFFODIL-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1742: --- Priority: Minor (was: Major) > TIFF - need DFDL schema and approach to TIFF > > > Key: DAFFODIL-1742 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1742 > Project: Daffodil > Issue Type: New Feature > Components: DFDL Language, DFDL Schemas >Reporter: Michael Beckerle >Priority: Minor > > TIFF image format is not something DFDL v1.0 can describe. > Nevertheless we need an approach that illustrates how to deal with it. > I *think* this means a preprocessing program, and then a DFDL schema to turn > the result of the preprocessor into an Infoset. For unparsing, a > post-processor also. > The right approach remains to be seen, but we need to show how to deal with > TIFF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1742) TIFF - need DFDL schema and approach to TIFF
[ https://issues.apache.org/jira/browse/DAFFODIL-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1742: --- Component/s: DFDL Language > TIFF - need DFDL schema and approach to TIFF > > > Key: DAFFODIL-1742 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1742 > Project: Daffodil > Issue Type: New Feature > Components: DFDL Language, DFDL Schemas >Reporter: Michael Beckerle >Priority: Major > > TIFF image format is not something DFDL v1.0 can describe. > Nevertheless we need an approach that illustrates how to deal with it. > I *think* this means a preprocessing program, and then a DFDL schema to turn > the result of the preprocessor into an Infoset. For unparsing, a > post-processor also. > The right approach remains to be seen, but we need to show how to deal with > TIFF. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1949) Ability to add XML Comments or Processing Instructions to any element
[ https://issues.apache.org/jira/browse/DAFFODIL-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661401#comment-16661401 ] Michael Beckerle commented on DAFFODIL-1949: Do we know who requested this and the specifics of their use case? Hate to over-generalize it just to be sure their use case is covered. > Ability to add XML Comments or Processing Instructions to any element > - > > Key: DAFFODIL-1949 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1949 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > Fix For: 2.4.0 > > > User community requests ability to add XML comments to elements based on > schema annotations. > Also processing instructions similarly. This allows ascribing validation or > other errors to parts of a specification such as an Inspection/Sanitization > Guide for data. > Note that a feature is present to add Processing instructions that contain a > dump of what is called the node-cache of computed property values - this is > for debugging purposes but does show how to add PI to a node so it > illustrates part of what is needed for this new feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1949) Ability to add XML Comments or Processing Instructions to any element
[ https://issues.apache.org/jira/browse/DAFFODIL-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661397#comment-16661397 ] Michael Beckerle commented on DAFFODIL-1949: There is a DFDL Language issue here: what should the annotations look like that indicate one wants a particular comment appended to the infoset, and where (after an element, as a child of an element, as a particular location in a sequence, etc. ?) Ditto for Processing Instruction. Suggest for JSON the only viable representation is to just provide a label to be used to label comments "Eg., use 'comment' as the key", and similarly for processing instruction 'use "processingInstruction" as the key." > Ability to add XML Comments or Processing Instructions to any element > - > > Key: DAFFODIL-1949 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1949 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > Fix For: 2.4.0 > > > User community requests ability to add XML comments to elements based on > schema annotations. > Also processing instructions similarly. This allows ascribing validation or > other errors to parts of a specification such as an Inspection/Sanitization > Guide for data. > Note that a feature is present to add Processing instructions that contain a > dump of what is called the node-cache of computed property values - this is > for debugging purposes but does show how to add PI to a node so it > illustrates part of what is needed for this new feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1949) Ability to add XML Comments or Processing Instructions to any element
[ https://issues.apache.org/jira/browse/DAFFODIL-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1949: --- Component/s: DFDL Language > Ability to add XML Comments or Processing Instructions to any element > - > > Key: DAFFODIL-1949 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1949 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > Fix For: 2.4.0 > > > User community requests ability to add XML comments to elements based on > schema annotations. > Also processing instructions similarly. This allows ascribing validation or > other errors to parts of a specification such as an Inspection/Sanitization > Guide for data. > Note that a feature is present to add Processing instructions that contain a > dump of what is called the node-cache of computed property values - this is > for debugging purposes but does show how to add PI to a node so it > illustrates part of what is needed for this new feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DAFFODIL-640) choiceLengthKind='explicit'
[ https://issues.apache.org/jira/browse/DAFFODIL-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660985#comment-16660985 ] Michael Beckerle edited comment on DAFFODIL-640 at 10/23/18 5:11 PM: - Note: Not used in any of the published schemas on DFDLSchemas (as of 2018-10-23). The workaround is to pad branches of the choice to the required length. was (Author: mbeckerle): Note: Not used in any of the published schemas on DFDLSchemas (as of 2018-10-23). > choiceLengthKind='explicit' > --- > > Key: DAFFODIL-640 > URL: https://issues.apache.org/jira/browse/DAFFODIL-640 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-640) choiceLengthKind='explicit'
[ https://issues.apache.org/jira/browse/DAFFODIL-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-640: -- Labels: (was: ForInteroperabilityTest) > choiceLengthKind='explicit' > --- > > Key: DAFFODIL-640 > URL: https://issues.apache.org/jira/browse/DAFFODIL-640 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-640) choiceLengthKind='explicit'
[ https://issues.apache.org/jira/browse/DAFFODIL-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660985#comment-16660985 ] Michael Beckerle commented on DAFFODIL-640: --- Note: Not used in any of the published schemas on DFDLSchemas (as of 2018-10-23). > choiceLengthKind='explicit' > --- > > Key: DAFFODIL-640 > URL: https://issues.apache.org/jira/browse/DAFFODIL-640 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-840) text numbers: textStandardBase
[ https://issues.apache.org/jira/browse/DAFFODIL-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-840: -- Labels: (was: ForInteroperabilityTest) > text numbers: textStandardBase > -- > > Key: DAFFODIL-840 > URL: https://issues.apache.org/jira/browse/DAFFODIL-840 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Compatibility, DFDL Language, Middle > End >Reporter: Steve Lawrence >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-840) text numbers: textStandardBase
[ https://issues.apache.org/jira/browse/DAFFODIL-840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-840: -- Priority: Minor (was: Major) > text numbers: textStandardBase > -- > > Key: DAFFODIL-840 > URL: https://issues.apache.org/jira/browse/DAFFODIL-840 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Compatibility, DFDL Language, Middle > End >Reporter: Steve Lawrence >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-230) documentFinalTerminatorCanBeMissing not working
[ https://issues.apache.org/jira/browse/DAFFODIL-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-230: -- Labels: (was: ForInteroperabilityTest) > documentFinalTerminatorCanBeMissing not working > --- > > Key: DAFFODIL-230 > URL: https://issues.apache.org/jira/browse/DAFFODIL-230 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Paul Chu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-230) documentFinalTerminatorCanBeMissing not working
[ https://issues.apache.org/jira/browse/DAFFODIL-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-230: -- Priority: Minor (was: Major) > documentFinalTerminatorCanBeMissing not working > --- > > Key: DAFFODIL-230 > URL: https://issues.apache.org/jira/browse/DAFFODIL-230 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Paul Chu >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-639) unicodeByteOrderMark feature
[ https://issues.apache.org/jira/browse/DAFFODIL-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660965#comment-16660965 ] Michael Beckerle commented on DAFFODIL-639: --- DFDL Workgroup is discussing (Oct 2018) whether all this BOM stuff should be optional functionality. If so then we're unlikely to implement this at all. > unicodeByteOrderMark feature > > > Key: DAFFODIL-639 > URL: https://issues.apache.org/jira/browse/DAFFODIL-639 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Minor > > This is not a property. The unicodeByteOrderMark is a member of the Infoset > Document Item. (aka the root element). > It depends on the dfdl:encoding property, which can be a runtime expression; > hence, this must be computed in an Evaluatable which in turn evaluates the > encodingEv. > Likely an Evaluatable[Option[ByteOrder]] is the type. > If no encoding property is defined this should be a constant None. > If the encoding property is defined and known to NOT be one of UTF-8, UTF-16, > or UTF-32, then this should be a constant None. > When unparsing, the value will either have been set from parsing, or can be > set from an API call. (New API method on Infoset needed.) > The API call is allowed, but the value ignored/unused by the unparser unless > the encoding is UTF-8, UTF-16, or UTF-32. > When the encoding evaluates to UTF-8, then the unicodeByteOrderMark will be > determined by the first 3 bytes being: > * 0xEF 0xBF 0xBE - ByteOrder.LittleEndian - 3 bytes are consumed (note: > strictly speaking, this shouldn't occur, but will if a naive utf-8 encoder > encodes a little-endian BOM into a 3-byte UTF-8 sequence. To insure such data > will round trip between UTF-8 and UTF-16 (LE - via BOM), we match this > sequence, and choose LittleEndian byte order) > * 0xEF 0xBB 0xBF - ByteOrder-BigEndian - 3 bytes are consumed > * anything else - no bytes are consumed, and the unicodeByteOrderMark is not > set (has no value) > when unparsing, if unicodeByteOrderMark is not set, then no byte order mark > is output. > For UTF-16, > * 0xFE 0xFF - byteOrder.BigEndian - 2 bytes are consumed > * 0xFF 0xFE - byteOrder.LittleEndian - 2 bytes are consumed > * anything else - parse error. > When unparsing, if encoding is UTF-16, and unicodeByteOrderMark is not set - > unparse error. > UTF-32 works like utf-16, except the byte patterns are 00 00 FE FF for > bigEndian, and FF FE 00 00 for littleEndian. > Recommended: package this code for reuse, assuming it needs to be used as a > library for reading/decoding strings generally. It's not impossible that the > above runtime errors when the byte order is not known, will be augmented in > the future by a mode where each individual text string at fine granularity is > examined for a byte order mark at the start. There also may be a need for > utf-16 heuristic byte-order determination - that is by looking at the bytes > for the characters and determining if they make more sense as big-endian or > little endian. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-640) choiceLengthKind='explicit'
[ https://issues.apache.org/jira/browse/DAFFODIL-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-640: -- Labels: ForInteroperabilityTest (was: ) > choiceLengthKind='explicit' > --- > > Key: DAFFODIL-640 > URL: https://issues.apache.org/jira/browse/DAFFODIL-640 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Minor > Labels: ForInteroperabilityTest > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-640) choiceLengthKind='explicit'
[ https://issues.apache.org/jira/browse/DAFFODIL-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-640: -- Priority: Major (was: Minor) > choiceLengthKind='explicit' > --- > > Key: DAFFODIL-640 > URL: https://issues.apache.org/jira/browse/DAFFODIL-640 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-640) choiceLengthKind='explicit'
[ https://issues.apache.org/jira/browse/DAFFODIL-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660967#comment-16660967 ] Michael Beckerle commented on DAFFODIL-640: --- Used in COBOL data extensively. > choiceLengthKind='explicit' > --- > > Key: DAFFODIL-640 > URL: https://issues.apache.org/jira/browse/DAFFODIL-640 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language >Reporter: Michael Beckerle >Priority: Minor > Labels: ForInteroperabilityTest > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-230) documentFinalTerminatorCanBeMissing not working
[ https://issues.apache.org/jira/browse/DAFFODIL-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-230: -- Priority: Major (was: Minor) > documentFinalTerminatorCanBeMissing not working > --- > > Key: DAFFODIL-230 > URL: https://issues.apache.org/jira/browse/DAFFODIL-230 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Paul Chu >Priority: Major > Labels: ForInteroperabilityTest > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-230) documentFinalTerminatorCanBeMissing not working
[ https://issues.apache.org/jira/browse/DAFFODIL-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-230: -- Labels: ForInteroperabilityTest (was: ) > documentFinalTerminatorCanBeMissing not working > --- > > Key: DAFFODIL-230 > URL: https://issues.apache.org/jira/browse/DAFFODIL-230 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Paul Chu >Priority: Minor > Labels: ForInteroperabilityTest > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1968) Support --stream option for CLI performance command
[ https://issues.apache.org/jira/browse/DAFFODIL-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1968: --- Priority: Minor (was: Major) > Support --stream option for CLI performance command > --- > > Key: DAFFODIL-1968 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1968 > Project: Daffodil > Issue Type: New Feature > Components: CLI >Affects Versions: 2.2.0 >Reporter: Steve Lawrence >Priority: Minor > > The performance subcommand currently works by creating a ByteBuffer and just > repeatedly calling parse on that. In order to test streaming performance we > would need to create an InputStream and continuously provide data to it, > perhaps via a PipeInput/OutputStream pair or something similar. This only > applies when testing parse performance (i.e. having both --stream and > --unparse options should be an error). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-935) DFDL encodingErrorPolicy='error' feature
[ https://issues.apache.org/jira/browse/DAFFODIL-935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660961#comment-16660961 ] Michael Beckerle commented on DAFFODIL-935: --- Note: This is one of the big interoperability barriers with IBM's DFDL, as they only implement encodingErrorPolicy='error', so technically until we implement this, no schema that then runs into a decoding error can be portable. > DFDL encodingErrorPolicy='error' feature > > > Key: DAFFODIL-935 > URL: https://issues.apache.org/jira/browse/DAFFODIL-935 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Compatibility, DFDL Language >Affects Versions: s13 >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > 0.13.0 has encodingErrorPolicy="replace". (DFDL-893). This is to implement > the much more challenging precise-error feature. This creates performance > challenges because one cannot buffer-ahead character conversion from bytes in > the obvious ways that say, java.io.BufferedReader does. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-935) DFDL encodingErrorPolicy='error' feature
[ https://issues.apache.org/jira/browse/DAFFODIL-935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-935: -- Labels: ForInteroperabilityTest (was: ) > DFDL encodingErrorPolicy='error' feature > > > Key: DAFFODIL-935 > URL: https://issues.apache.org/jira/browse/DAFFODIL-935 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Compatibility, DFDL Language >Affects Versions: s13 >Reporter: Michael Beckerle >Priority: Minor > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > 0.13.0 has encodingErrorPolicy="replace". (DFDL-893). This is to implement > the much more challenging precise-error feature. This creates performance > challenges because one cannot buffer-ahead character conversion from bytes in > the obvious ways that say, java.io.BufferedReader does. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-935) DFDL encodingErrorPolicy='error' feature
[ https://issues.apache.org/jira/browse/DAFFODIL-935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-935: -- Priority: Major (was: Minor) > DFDL encodingErrorPolicy='error' feature > > > Key: DAFFODIL-935 > URL: https://issues.apache.org/jira/browse/DAFFODIL-935 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Compatibility, DFDL Language >Affects Versions: s13 >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > 0.13.0 has encodingErrorPolicy="replace". (DFDL-893). This is to implement > the much more challenging precise-error feature. This creates performance > challenges because one cannot buffer-ahead character conversion from bytes in > the obvious ways that say, java.io.BufferedReader does. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-935) DFDL encodingErrorPolicy='error' feature
[ https://issues.apache.org/jira/browse/DAFFODIL-935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-935: -- Fix Version/s: 2.3.0 > DFDL encodingErrorPolicy='error' feature > > > Key: DAFFODIL-935 > URL: https://issues.apache.org/jira/browse/DAFFODIL-935 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Compatibility, DFDL Language >Affects Versions: s13 >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > 0.13.0 has encodingErrorPolicy="replace". (DFDL-893). This is to implement > the much more challenging precise-error feature. This creates performance > challenges because one cannot buffer-ahead character conversion from bytes in > the obvious ways that say, java.io.BufferedReader does. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-341) variables - newVariableInstance
[ https://issues.apache.org/jira/browse/DAFFODIL-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-341: -- Summary: variables - newVariableInstance (was: variables (internal) - newVariableInstance) > variables - newVariableInstance > --- > > Key: DAFFODIL-341 > URL: https://issues.apache.org/jira/browse/DAFFODIL-341 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Michael Beckerle >Priority: Major > > Refactored variables tasks into more relevant chunks with different > priorities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-335) defaultValue of newVariableInstance does not override global default value
[ https://issues.apache.org/jira/browse/DAFFODIL-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-335. - Resolution: Duplicate Duplicate of DAFFODIL-341 newVariableInstance is not yet implemented at all, so this ticket about some aspect of its functionality doesn't make sense, and is probably a misinterpretation of the way the test failed. In any case DAFFODIL-341 is the ticket for implementing newVariableInstance in its entirety. > defaultValue of newVariableInstance does not override global default value > -- > > Key: DAFFODIL-335 > URL: https://issues.apache.org/jira/browse/DAFFODIL-335 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Paul Chu >Priority: Minor > > The test, varInstance, in > daffodil-test\src\test\scala-debug\daffodil\section07\variables\TestVariablesDebug.scala > fails with error message: > java.lang.Exception: Comparison failed. > Expected http://www.ibm.com/xmlns/dfdl/testData; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; > xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/; > xmlns:xs="http://www.w3.org/2001/XMLSchema; > xmlns:xsd="http://www.w3.org/2001/XMLSchema; > xmlns:ct="http://w3.ibm.com/xmlns/dfdl/ctInfoset; > xmlns:ex="http://example.com; > xmlns="http://www.example.org/example1/;>7 > Actual http://example.com;>6 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1951) JAVA API and CLI access to XML Diff algorithm
[ https://issues.apache.org/jira/browse/DAFFODIL-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1951: --- Priority: Minor (was: Major) > JAVA API and CLI access to XML Diff algorithm > - > > Key: DAFFODIL-1951 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1951 > Project: Daffodil > Issue Type: New Feature > Components: API >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > > Our XML Diff algorithm is of utility outside of the rest of Daffodil, and > user community has requested API access and documentation of it (from Java), > and CLI, for its use. > Note however, there are other tickets calling into question whether we > actually should be finding someone else's more comprehensive XML diff > solution so as to not have to maintain our own such infrastructure. > See DAFFODIL-1891, DAFFODIL-1887, DAFFODIL-288 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1950) Documentation/Example of how to use JAPI in "no extra data allowed" mode
[ https://issues.apache.org/jira/browse/DAFFODIL-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1950: --- Priority: Minor (was: Major) > Documentation/Example of how to use JAPI in "no extra data allowed" mode > > > Key: DAFFODIL-1950 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1950 > Project: Daffodil > Issue Type: New Feature > Components: Documentation >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > Fix For: 2.4.0 > > > This code may exist in our JAPI unit tests already. > A common usage pattern for our user community is to parse, inspect, sanitize, > and reassemble (unparse) data to provide guaranteed spec-compliant clean data > to applications. > Left over data past the normal end marker or length of a file is a common > thing people want to eliminate. > However, whether left over data is a problem or not depends on the > application need. For some applications, e.g, message streaming, the "left > over" data is really just the first part of the next message, and must be > where we pick up to keep parsing after finishing a first message. > Since both (and perhaps other) modes of operation are reasonable and > expected, we need examples of how to implement these patterns, and they need > to be implementable from our Java API. > There may already be unit tests illustrating this in the JAPI unit test set. > Our TDML runner certaily does that, but it is written in Scala. If so then > this issue is really just requesting a Wiki documentation page highlighting > this code and the usage pattern of interest. > This issue may be subsumed by the Message streaming API issue, which is > adding a new mode of operation where left-over data is expected, and doesn't > indicate any problem with the data stream. (DAFFODIL-1565, DAFFODIL-1065) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1949) Ability to add XML Comments or Processing Instructions to any element
[ https://issues.apache.org/jira/browse/DAFFODIL-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1949: --- Priority: Minor (was: Major) > Ability to add XML Comments or Processing Instructions to any element > - > > Key: DAFFODIL-1949 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1949 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Front End >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > Fix For: 2.4.0 > > > User community requests ability to add XML comments to elements based on > schema annotations. > Also processing instructions similarly. This allows ascribing validation or > other errors to parts of a specification such as an Inspection/Sanitization > Guide for data. > Note that a feature is present to add Processing instructions that contain a > dump of what is called the node-cache of computed property values - this is > for debugging purposes but does show how to add PI to a node so it > illustrates part of what is needed for this new feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1917) Support for new character classes
[ https://issues.apache.org/jira/browse/DAFFODIL-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1917: --- Priority: Minor (was: Major) > Support for new character classes > - > > Key: DAFFODIL-1917 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1917 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Front End, Middle End >Reporter: Steve Lawrence >Priority: Minor > > Daffodil v2.0 wishlist includes new character class entities LSP, LSP* and > LSP+, which are the same as WSP except they do not match line endings. This > is useful as currently it is impossible to match an empty line followed by a > newline via something like %WSP*;%NL; due to the greediness of WSP* and the > inability for our delimiter scanner to backtrack of forward lookahead. > Related, it might be useful to also add NL+ and NL* to match an unbounded > number of newlines. > Additionally, it might be a good idea to cause an SDE if a character class > that contains * or > + is immediately followed by a character inside that character class. The > reason being that such sequences will always fail without lookahead or > backtracking, which probalby isn't worth implementing. > For security reasons, we should also implement a tunable to limit the number > or repetitions of these character entities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1754) DFDLSchema example for Asterisk
[
https://issues.apache.org/jira/browse/DAFFODIL-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle closed DAFFODIL-1754.
--
Resolution: Not A Problem
DFDL Schemas that are desirable are separate topic/effort from apache daffodil.
Not tracked with Daffodil JIRA tickets.
> DFDLSchema example for Asterisk
> ---
>
> Key: DAFFODIL-1754
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1754
> Project: Daffodil
> Issue Type: New Feature
> Components: DFDL Schemas
>Reporter: Michael Beckerle
>Priority: Major
>
> Asterix format has come up 3 times now.
> This is a publicly available standard data format.
> We can economize effort in support for people on Asterix by just creating the
> starting point schema for it and publishing it to DFDLSchemas.
> We should create one the way we think it should be done for maintainability
> and performance (ex: using occursCountKind="expression" and occursCount="{
> ../hdr/flag }" vs. discriminators)
> There is an excellent looking tool called www.asterixtk.com which has a free
> version and it includes some sample data. Also has ability to output XML. It
> would be interesting to see to what degree DFDL can produce that XML or
> something close to it. I don't know if there is an XML Schema for this XML.
> Creating a comprehensive DFDL schema for Asterix is a big job. I think a
> schema for just one message type (e.g., cat062), that will parse one example
> piece of data is all that is required really. If we're lucky people who
> really want to do Asterix will undertake the added QA/test, and growing the
> schema's coverage of the spec.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1710) Apache Tika integration
[ https://issues.apache.org/jira/browse/DAFFODIL-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1710. -- Resolution: Not A Problem Not a Daffodil bug, improvement or feature. JIRA tickets are not for "good idea" keeping track. > Apache Tika integration > --- > > Key: DAFFODIL-1710 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1710 > Project: Daffodil > Issue Type: New Feature > Components: API, Integrations >Reporter: Michael Beckerle >Priority: Major > > Daffodil's parser could be encapsulated with the Apache Tika APIs allowing > any DFDL-described format to be mined for text content in the Tika way. > Probably this would want to be schema-aware in that Tika events would not > want to be reported for numeric content, but only text content. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1159) Reimplement sequenceKind unordered feature
[ https://issues.apache.org/jira/browse/DAFFODIL-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1159: --- Labels: ForInteroperabilityTest (was: ) > Reimplement sequenceKind unordered feature > -- > > Key: DAFFODIL-1159 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1159 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Affects Versions: 1.0.0 >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > As part of DPath work, unordered sequences were disabled and no longer work. > This ticket is to reimplement them at least to the same level of function we > had before replacing Saxon with DPath. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1238) onPathParser and onPathUnparser
[ https://issues.apache.org/jira/browse/DAFFODIL-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1238: --- Priority: Minor (was: Major) > onPathParser and onPathUnparser > --- > > Key: DAFFODIL-1238 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1238 > Project: Daffodil > Issue Type: New Feature > Components: API >Reporter: Michael Beckerle >Priority: Minor > > To reduce overhead during development, and for applications that only need > one or the other of Parser and Unparser, we could split these and let the API > handle them entirely separately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-1989) lengthKind 'explicit' with expression should not use minLength nor textOutputMinLength
[ https://issues.apache.org/jira/browse/DAFFODIL-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Adams reassigned DAFFODIL-1989: Assignee: Josh Adams > lengthKind 'explicit' with expression should not use minLength nor > textOutputMinLength > -- > > Key: DAFFODIL-1989 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1989 > Project: Daffodil > Issue Type: Bug > Components: Back End, Middle End >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Assignee: Josh Adams >Priority: Major > Fix For: 2.3.0 > > > Per DFDL Erratum 5.18, if the dfdl:lengthKind is explicit, the facet > minLength is NOT used to determine the target length. The target length is > the value of the length expression (or constant) directly. The minLength > facet is only used for validation (if enabled). > Code in Daffodil e.g., UnparseTargetLengthInCharactersEv, MinLengthInBitsEv, > and UnparseTargetLengthInBitsEv, all take minLength into account even for > lengthKind explicit, and do not consider whether the expression is a constant > or not. (That check occurs elsewhere, so need not be taken into account, but > minLength needs to be disregarded as far as the target length is concerned, > but still kept around for validation purposes.) > Similarly, for types other than string/hexBinary that are text, > textOutputMinLength should be ignored if dfdl:lengthKind is explicit, though > it is an error if the length is a constant and is out of range for > textOutputMinLength. (Similarly for minLength/maxLength, except that is an > SDW) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1011) Debugger should be schema aware
[ https://issues.apache.org/jira/browse/DAFFODIL-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1011: --- Issue Type: Improvement (was: New Feature) > Debugger should be schema aware > --- > > Key: DAFFODIL-1011 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1011 > Project: Daffodil > Issue Type: Improvement > Components: Debugger, Usability >Reporter: Steve Lawrence >Priority: Major > > It would be nice if you could load in a schema into the debugger and have it > associate the schema with the parsers. This would allow one to view the > properties associated with all parsers, even if the parsers themselves do not > store the requested properties. This might also allow for features like > easier setting of breakpoints and things like autocompletion of paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1073) Add new compile() method that supports strings/xml nodes
[ https://issues.apache.org/jira/browse/DAFFODIL-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1073: --- Priority: Minor (was: Major) > Add new compile() method that supports strings/xml nodes > > > Key: DAFFODIL-1073 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1073 > Project: Daffodil > Issue Type: New Feature > Components: API >Reporter: Steve Lawrence >Priority: Minor > > Not all system/API will want/be able to read schemas from the harddrive. It > might be helpful to have an alternate compile method that can read in all > schema information from memory. This could certainly be helpful if we ever > had a two pass system that autogenerated schemas. It would be nice not to > have to temporarily write them to disc. > One thing that makes this tricky is out to handle import/includes, since > those use schemaLocation to define where to find a schema. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-628) Add support for JAF logging mechanism
[ https://issues.apache.org/jira/browse/DAFFODIL-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-628: -- Priority: Minor (was: Major) > Add support for JAF logging mechanism > - > > Key: DAFFODIL-628 > URL: https://issues.apache.org/jira/browse/DAFFODIL-628 > Project: Daffodil > Issue Type: New Feature > Components: API, Back End >Reporter: Steve Lawrence >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-451) Implement a Check for the Distinctness of Delimiters
[ https://issues.apache.org/jira/browse/DAFFODIL-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-451: -- Priority: Minor (was: Major) > Implement a Check for the Distinctness of Delimiters > > > Key: DAFFODIL-451 > URL: https://issues.apache.org/jira/browse/DAFFODIL-451 > Project: Daffodil > Issue Type: New Feature > Components: Middle End, Usability >Reporter: Jessie Chab >Priority: Minor > Time Spent: 49m > > Taylor: Do you know if the padChar is allowed to be the same as an escapeChar? > Mike: bad idea. I think we added an errata just a week or so ago that says it > is an SDE if any of the things are not distinct. E.g., the terminator cannot > be the same as the pad character either, nor any other delimiter, nor the > textStandardGroupingSeparator nor \ > Mike: delimiters can be shared prefixes of each other, and an enclosing group > could have the same delimiters as something it encloses (e.g., enclosing > gorup has comma separator, enclosed inner group does also) We do need an > explicit check for the distinctness of these things. So jessie's test should > throw an SDE. > So... that test should result in an SDE. We'll need to create a ticket/bug to > add that. > Conversation above first referenced in DFDL-259 (Implement Trim/Pad/Fill) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1959) EXIficient Inputter and outputter for XML EXI representation
[ https://issues.apache.org/jira/browse/DAFFODIL-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1959: --- Priority: Minor (was: Major) > EXIficient Inputter and outputter for XML EXI representation > > > Key: DAFFODIL-1959 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1959 > Project: Daffodil > Issue Type: New Feature > Components: Back End >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Minor > Labels: beginner > > Create EXI (dense binary XML) representation using an EXI-specific > InfosetInputter and InfosetOutputter. > These would be very similar to the XML InfosetInputter and Outputter - as EXI > libraries such as EXIfficient https://github.com/EXIficient/exificient > already have SAX/StAX, APIs, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-565) tolerate attributeFormDefault on xs:schema
[ https://issues.apache.org/jira/browse/DAFFODIL-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-565: -- Labels: ForInteroperabilityTest (was: ) > tolerate attributeFormDefault on xs:schema > -- > > Key: DAFFODIL-565 > URL: https://issues.apache.org/jira/browse/DAFFODIL-565 > Project: Daffodil > Issue Type: New Feature > Components: DFDL Language, Front End >Affects Versions: s7 >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > > We recognize this option, but issue a not-yet-implemented SDE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-110) Implement separator suppression policies
[ https://issues.apache.org/jira/browse/DAFFODIL-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-110: -- Labels: ForInteroperabilityTest (was: ) > Implement separator suppression policies > > > Key: DAFFODIL-110 > URL: https://issues.apache.org/jira/browse/DAFFODIL-110 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-110) Implement separator suppression policies
[ https://issues.apache.org/jira/browse/DAFFODIL-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-110: -- Component/s: DFDL Language > Implement separator suppression policies > > > Key: DAFFODIL-110 > URL: https://issues.apache.org/jira/browse/DAFFODIL-110 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-110) Implement separator suppression policies
[ https://issues.apache.org/jira/browse/DAFFODIL-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reassigned DAFFODIL-110: - Assignee: Michael Beckerle > Implement separator suppression policies > > > Key: DAFFODIL-110 > URL: https://issues.apache.org/jira/browse/DAFFODIL-110 > Project: Daffodil > Issue Type: New Feature > Components: Back End, DFDL Language, Front End, Middle > End >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-828) IBM Compatibility - Test out HL7 schemas
[ https://issues.apache.org/jira/browse/DAFFODIL-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reassigned DAFFODIL-828: - Assignee: Michael Beckerle > IBM Compatibility - Test out HL7 schemas > > > Key: DAFFODIL-828 > URL: https://issues.apache.org/jira/browse/DAFFODIL-828 > Project: Daffodil > Issue Type: New Feature > Components: Compatibility, DFDL Schemas, Infrastructure, QA >Reporter: Jessie Chab >Assignee: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > IBM has been creating public DFDL schemas for various data types. When these > are released, we try to parse data against the schema using Daffodil for > compatibility's sake. A set of DFDL schemas that model HL7 v2.7 messages are > now available in a DFDLSchemas GitHub repsitory at > http://dfdlschemas.github.io/HL7-v2.7/ > ([View on github|https://github.com/DFDLSchemas/HL7-v2.7]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-828) IBM Compatibility - Test out HL7 schemas
[ https://issues.apache.org/jira/browse/DAFFODIL-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660931#comment-16660931 ] Michael Beckerle commented on DAFFODIL-828: --- This is a subject of discussion (Oct 2018) in the DFDL Workgroup. Heretofore, the assumption was that model groups are "always required" and hence, would always have a separator in the enclosing sequence, if that sequence has separators specified. There are exceptions to this for (a) computed elements (input Value calc), and (b) hidden groups. But there is a question about whether a group that has no syntax, and exists only to express a statement such as a dfdl:assert or a dfdl:setVariable, should get a separator or not. Further discussion is whether model groups should in fact EVER have separators, or if the presence of separators is a function of whether there is anything to separate (something with syntax), and the separatorSuppressionPolicy. Ultimately this is closely related to issues surrounding trailing separator suppression, so assigning to myself as I am dealing with those currently. > IBM Compatibility - Test out HL7 schemas > > > Key: DAFFODIL-828 > URL: https://issues.apache.org/jira/browse/DAFFODIL-828 > Project: Daffodil > Issue Type: New Feature > Components: Compatibility, DFDL Schemas, Infrastructure, QA >Reporter: Jessie Chab >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > IBM has been creating public DFDL schemas for various data types. When these > are released, we try to parse data against the schema using Daffodil for > compatibility's sake. A set of DFDL schemas that model HL7 v2.7 messages are > now available in a DFDLSchemas GitHub repsitory at > http://dfdlschemas.github.io/HL7-v2.7/ > ([View on github|https://github.com/DFDLSchemas/HL7-v2.7]) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-999) IBM's ISO8583 format not currently compatible with Daffodil
[ https://issues.apache.org/jira/browse/DAFFODIL-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Lawrence updated DAFFODIL-999: Fix Version/s: 2.3.0 > IBM's ISO8583 format not currently compatible with Daffodil > --- > > Key: DAFFODIL-999 > URL: https://issues.apache.org/jira/browse/DAFFODIL-999 > Project: Daffodil > Issue Type: New Feature > Components: Compatibility, DFDL Schemas >Reporter: Jessie Chab >Assignee: Steve Lawrence >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > IBM's ISO8583 format currently uses "textStandardExponentCharacter" instead > of "textStandardExponentRep", and also uses lengthKind "prefixed" which is > not yet implemented in Daffodil. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1559) Add option to disable CRLF to LF XML canonicalization
[ https://issues.apache.org/jira/browse/DAFFODIL-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1559: --- Priority: Minor (was: Major) > Add option to disable CRLF to LF XML canonicalization > - > > Key: DAFFODIL-1559 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1559 > Project: Daffodil > Issue Type: New Feature > Components: API >Reporter: Steve Lawrence >Priority: Minor > > See the review or more details. The short of it is that when converting parse > results to XML, we convert CR to LF, and we convert CRLF to LF. This means > that we lose the information that the data used to contain CRLF. This is > similar to how we lose that information with delimiters if someone uses NL, > but it's slightly different since it is actual data. However, it's most user > friendly and consistent with other XML technologies to have this behavior. > Perhaps we need an option to convert CRLF to somewhere in PUA so that this > information can be maintained if someone needs it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-999) IBM's ISO8583 format not currently compatible with Daffodil
[ https://issues.apache.org/jira/browse/DAFFODIL-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reassigned DAFFODIL-999: - Assignee: Steve Lawrence > IBM's ISO8583 format not currently compatible with Daffodil > --- > > Key: DAFFODIL-999 > URL: https://issues.apache.org/jira/browse/DAFFODIL-999 > Project: Daffodil > Issue Type: New Feature > Components: Compatibility, DFDL Schemas >Reporter: Jessie Chab >Assignee: Steve Lawrence >Priority: Major > Labels: ForInteroperabilityTest > > IBM's ISO8583 format currently uses "textStandardExponentCharacter" instead > of "textStandardExponentRep", and also uses lengthKind "prefixed" which is > not yet implemented in Daffodil. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1481) HIPAA-5010 schema - see if it works on daffodil
[ https://issues.apache.org/jira/browse/DAFFODIL-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1481. -- Resolution: Not A Problem Schema not publicly available. Can't test. Our interop testing will focus on schemas that IBM has published. > HIPAA-5010 schema - see if it works on daffodil > --- > > Key: DAFFODIL-1481 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1481 > Project: Daffodil > Issue Type: New Feature > Components: Compatibility, DFDL Schemas, QA >Affects Versions: 2.0.0 >Reporter: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > Fix For: 2.3.0 > > > IBM announced another DFDL schema available. > https://github.com/DFDLSchemas/HIPAA-5010 > Like several others, these are not publicly available, they must be > purchased/licensed from IBM; however, we may get permission to evaluate for > purposes of seeing if we're compatible. > There are other JIRA tickets about IBM compatibility that should probably > take priority over this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-1981) Support for textStandardExponentRep="" missing
[ https://issues.apache.org/jira/browse/DAFFODIL-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Adams reassigned DAFFODIL-1981: Assignee: (was: Josh Adams) > Support for textStandardExponentRep="" missing > -- > > Key: DAFFODIL-1981 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1981 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.2.0 >Reporter: Steve Lawrence >Priority: Major > > The DFDL specification states if testStandardExponentRep is the empty string, > then no exponent character separates the mantissa from the exponent, allowing > numbers like "123+2" to be valid. However, the version of ICU used by > Daffodil does not handle the empty string expontent rep correctly, and so > this feature is missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1981) Support for textStandardExponentRep="" missing
[ https://issues.apache.org/jira/browse/DAFFODIL-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660722#comment-16660722 ] Josh Adams commented on DAFFODIL-1981: -- This is something that worked in ICU4J v58.2, but was broken in v59.1 during a major rewrite of the number handling. However, a parseNoExponent setting has been added to DecimalFormat that supposedly will enable handling of empty string exponents, but it is currently marked as deprecated due to its experimental status. It is supposed to be moved out of experimental status in the near future.  http://bugs.icu-project.org/trac/ticket/11725 > Support for textStandardExponentRep="" missing > -- > > Key: DAFFODIL-1981 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1981 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.2.0 >Reporter: Steve Lawrence >Assignee: Josh Adams >Priority: Major > > The DFDL specification states if testStandardExponentRep is the empty string, > then no exponent character separates the mantissa from the exponent, allowing > numbers like "123+2" to be valid. However, the version of ICU used by > Daffodil does not handle the empty string expontent rep correctly, and so > this feature is missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-845) textNumberProps: Issues with textNumberCheckPolicy="lax"
[ https://issues.apache.org/jira/browse/DAFFODIL-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-845: -- Issue Type: Bug (was: Wish) > textNumberProps: Issues with textNumberCheckPolicy="lax" > > > Key: DAFFODIL-845 > URL: https://issues.apache.org/jira/browse/DAFFODIL-845 > Project: Daffodil > Issue Type: Bug > Components: ICU >Affects Versions: s11 >Reporter: Jessie Chab >Priority: Major > > According to errata item 2.21: > 2.21. Section 13.6. Change meaning of textNumberCheckPolicy enum 'lax' to > align with ICU.: "If ���lax' and dfdl:textNumberRep is 'standard' then > grouping separators are ignored, leading and trailing whitespace is ignored, > leading zeros are ignored, quoted characters may be omitted." > I wrote two tests for the last part of that requirement, and they result in > Parse Errors. > see tests test_textNumberCheckPolicy_lax01, test_textNumberCheckPolicy_lax05 > in files: > daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section13/text_number_props/TextNumberProps.tdml > daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section13/text_number_props/TestTextNumberPropsDebug.scala > daffodil-test/src/test/scala/edu/illinois/ncsa/daffodil/section13/text_number_props/TestTextNumberProps.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-878) TextNumberProps: digits after E should give minimum exponent digit count
[ https://issues.apache.org/jira/browse/DAFFODIL-878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660688#comment-16660688 ] Michael Beckerle commented on DAFFODIL-878: --- See also DAFFODIL-845 on number parsing not being strict enough. > TextNumberProps: digits after E should give minimum exponent digit count > > > Key: DAFFODIL-878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-878 > Project: Daffodil > Issue Type: Wish > Components: ICU >Reporter: Jessie Chab >Priority: Minor > > According to the spec, the number of digits after the E in scientific > notation gives the minimum exponent digit count. I would expect that if > textNumberCheckPolicy is strict, then data that provides a number of digits > less than this minimum should fail. However the data parses without issue. > see test > textNumberPattern_scientificNotation02 in the following files: > daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section13/text_number_props/TextNumberProps.tdml > daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section13/text_number_props/TestTextNumberPropsDebug.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-878) TextNumberProps: digits after E should give minimum exponent digit count
[ https://issues.apache.org/jira/browse/DAFFODIL-878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-878: -- Issue Type: Bug (was: Wish) > TextNumberProps: digits after E should give minimum exponent digit count > > > Key: DAFFODIL-878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-878 > Project: Daffodil > Issue Type: Bug > Components: ICU >Reporter: Jessie Chab >Priority: Minor > > According to the spec, the number of digits after the E in scientific > notation gives the minimum exponent digit count. I would expect that if > textNumberCheckPolicy is strict, then data that provides a number of digits > less than this minimum should fail. However the data parses without issue. > see test > textNumberPattern_scientificNotation02 in the following files: > daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section13/text_number_props/TextNumberProps.tdml > daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section13/text_number_props/TestTextNumberPropsDebug.scala -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1165) sbt universal:packageZipTarball fails on Windows w/ Cygwin
[
https://issues.apache.org/jira/browse/DAFFODIL-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle updated DAFFODIL-1165:
---
Issue Type: Bug (was: Wish)
> sbt universal:packageZipTarball fails on Windows w/ Cygwin
> --
>
> Key: DAFFODIL-1165
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1165
> Project: Daffodil
> Issue Type: Bug
> Components: Infrastructure, Windows
>Affects Versions: s15
>Reporter: Jessie Chab
>Priority: Minor
>
> I verified that tar was installed on Cygwin:
> jchab@WIN-EEA4S63ULQH ~/daffodil
> $ tar
> tar: You must specify one of the `-Acdtrux' or `--test-label' options
> Try `tar --help' or `tar --usage' for more information.
> but I get the following error when running sbt universal:packageBin
> {code}
> tar: Cannot connect to C: resolve failed
> java.lang.RuntimeException: Error tarballing
> C:\cygwin\home\jchab\daffodil\daffodil-cli\target\universal\daffodil-0.15.0-SNAPSHOT.tgz.
> Exit code: 128
> at scala.sys.package$.error(package.scala:27)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$makeTarball$1.apply(Archives.scala:134)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$makeTarball$1.apply(Archives.scala:118)
> at sbt.IO$.withTemporaryDirectory(IO.scala:285)
> at
> com.typesafe.sbt.packager.universal.Archives$.makeTarball(Archives.scala:118)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$10.apply(Archives.scala:111)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$10.apply(Archives.scala:111)
> at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:35)
> at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:34)
> at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
> at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
> at sbt.std.Transform$$anon$4.work(System.scala:64)
> at
> sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
> at
> sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
> at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
> at sbt.Execute.work(Execute.scala:244)
> at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
> at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
> at
> sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
> at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> [error] (daffodil-cli/universal:packageZipTarball) Error tarballing
> C:\cygwin\home\jchab\daffodil\daffodil-cli\target\universal\daffodil-0.15.0-SNAPSHOT.tgz.
> Exit code: 128
> [error] Total time: 22 s, completed Nov 25, 2014 11:01:38 AM
> {code}
> sbt universal:packageBin works without issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1602) Add Fuzz testing
[ https://issues.apache.org/jira/browse/DAFFODIL-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1602: --- Priority: Minor (was: Major) > Add Fuzz testing > > > Key: DAFFODIL-1602 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1602 > Project: Daffodil > Issue Type: Improvement > Components: Infrastructure, QA, TDML Runner >Reporter: Michael Beckerle >Priority: Minor > > We should be doing fuzz testing, both for schemas, and for data files being > parsed and unparsed. (fuzzed valid XML infoset input???) > There are tools for this like : http://lcamtuf.coredump.cx/afl/ > See also wikipedia on fuzz testing: https://en.wikipedia.org/wiki/Fuzz_testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1168) sbt universal:packageZipTarball fails on Windows (CMD)
[
https://issues.apache.org/jira/browse/DAFFODIL-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle updated DAFFODIL-1168:
---
Issue Type: Bug (was: Wish)
> sbt universal:packageZipTarball fails on Windows (CMD)
> --
>
> Key: DAFFODIL-1168
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1168
> Project: Daffodil
> Issue Type: Bug
> Components: Infrastructure, Windows
>Affects Versions: s15
>Reporter: Jessie Chab
>Priority: Minor
>
> tar is not natively on Windows, so I had to install [GNU
> Tar|http://gnuwin32.sourceforge.net/packages/gtar.htm]. I set up my Path
> correctly and verified that I could run the command. However, when I try to
> run sbt universal:packageZipTarball, I get the following error:
> {code}
> [info] Loading project definition from C:\Users\jchab\NGF\daffodil\project
> [info] Set current project to daffodil (in build
> file:/C:/Users/jchab/NGF/daffodil/)
> [info] Updating {file:/C:/Users/jchab/NGF/daffodil/}daffodil-cli...
> [info] Wrote
> C:\Users\jchab\NGF\daffodil\daffodil-cli\target\scala-2.10\daffodil-cli_2.10-0.15.0-SNAPSHOT.pom
> [info] Resolving org.fusesource.jansi#jansi;1.4 ...4 .5 ...PSHOT
> [info] Done updating.
> [info] Wrote
> C:\Users\jchab\NGF\daffodil\daffodil-lib\target\scala-2.10\daffodil-lib_2.10-0.15.0-SNAPSHOT.pom
> [info] Wrote
> C:\Users\jchab\NGF\daffodil\daffodil-io\target\scala-2.10\daffodil-io_2.10-0.15.0-SNAPSHOT.pom
> [info] Wrote
> C:\Users\jchab\NGF\daffodil\daffodil-core\target\scala-2.10\daffodil-core_2.10-0.15.0-SNAPSHOT.pom
> [info] Wrote
> C:\Users\jchab\NGF\daffodil\daffodil-tdml\target\scala-2.10\daffodil-tdml_2.10-0.15.0-SNAPSHOT.pom
> tar: Cannot open
> C:\Users\jchab\AppData\Local\Temp\sbt_5185c472\daffodil-0.15.0-SNAPSHOT.tar:
> Function not implemented
> tar: Error is not recoverable: exiting now
> java.lang.RuntimeException: Error tarballing
> C:\Users\jchab\NGF\daffodil\daffodil-cli\target\universal\daffodil-0.15.0-SNAPSHOT.tgz.
> Exit code: 2
> at scala.sys.package$.error(package.scala:27)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$makeTarball$1.apply(Archives.scala:134)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$makeTarball$1.apply(Archives.scala:118)
> at sbt.IO$.withTemporaryDirectory(IO.scala:285)
> at
> com.typesafe.sbt.packager.universal.Archives$.makeTarball(Archives.scala:118)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$10.apply(Archives.scala:111)
> at
> com.typesafe.sbt.packager.universal.Archives$$anonfun$10.apply(Archives.scala:111)
> at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:35)
> at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:34)
> at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
> at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
> at sbt.std.Transform$$anon$4.work(System.scala:64)
> at
> sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
> at
> sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
> at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
> at sbt.Execute.work(Execute.scala:244)
> at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
> at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
> at
> sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
> at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> [error] (daffodil-cli/universal:packageZipTarball) Error tarballing
> C:\Users\jchab\NGF\daffodil\daffodil-cli\target\universal\daffodil-0.15.0-SNAPSHOT.tgz.
> Exit code: 2
> [error] Total time: 19 s, completed Dec 1, 2014 11:11:44 AM
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1523) fn:tokenize - propose for DFDL inclusion
[ https://issues.apache.org/jira/browse/DAFFODIL-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1523. -- Resolution: Not A Problem DFDL Spec/Workgroup issues not to be tracked on Apache Daffodil trackers. > fn:tokenize - propose for DFDL inclusion > > > Key: DAFFODIL-1523 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1523 > Project: Daffodil > Issue Type: Wish > Components: DFDL Language >Reporter: Michael Beckerle >Priority: Major > > That is very clean looking. It does still re-tokenize the thing 3 extra times. > See pcap.dfdl.xsd with the unparsing changes. > We can propose adding fn:tokenize to DFDL. The problem of sharing > sub-expressions usefully remains. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1554) DFDL Spec - consider arrays of zero-length as legal in hidden groups since they're defaultable in some sense.
[ https://issues.apache.org/jira/browse/DAFFODIL-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1554. -- Resolution: Not A Problem DFDL Workgroup activity not to be tracked on Apache Daffodil JIRA tickets. Track elsewhere. > DFDL Spec - consider arrays of zero-length as legal in hidden groups since > they're defaultable in some sense. > - > > Key: DAFFODIL-1554 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1554 > Project: Daffodil > Issue Type: Task > Components: DFDL Language, Unparsing >Reporter: Michael Beckerle >Priority: Major > > If the array can have zero elements, then that should satisfy the need, as a > zero-occurrance array is "defaultable" in some sense. > The other "trick" which is to make it an array of complex elements which > themselves contain defaultable/OVC elements, may or may not work, and it's a > hack. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1863) DFDL Workgroup Proposal for hexBinary lengthUnits 'bits'
[ https://issues.apache.org/jira/browse/DAFFODIL-1863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1863. -- Resolution: Not A Problem DFDL spec-related work not to be tracked on Apache Daffodil JIRA tickets. > DFDL Workgroup Proposal for hexBinary lengthUnits 'bits' > > > Key: DAFFODIL-1863 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1863 > Project: Daffodil > Issue Type: Task > Components: Documentation >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Minor > Fix For: 2.3.0 > > > We have an implementation of lengthUnits 'bits' for hexBinary. > A write up of this in the form of an erratum for the DFDL v1.0 spec is needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1173) Test: Update testlink cases for CLI tests that have been automated
[ https://issues.apache.org/jira/browse/DAFFODIL-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1173. -- Resolution: Won't Fix NO longer maintaining testlink - was a Tresys internal system for requirements tracking. > Test: Update testlink cases for CLI tests that have been automated > -- > > Key: DAFFODIL-1173 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1173 > Project: Daffodil > Issue Type: Task > Components: QA >Affects Versions: s15 >Reporter: Elizabeth Fahl >Assignee: Dave Thompson >Priority: Minor > > Many CLI tests have been updated to automated tests but the testlink cases > still have the manual steps listed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-798) verify Errata Test Coverage
[ https://issues.apache.org/jira/browse/DAFFODIL-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-798: -- Issue Type: Improvement (was: Task) > verify Errata Test Coverage > --- > > Key: DAFFODIL-798 > URL: https://issues.apache.org/jira/browse/DAFFODIL-798 > Project: Daffodil > Issue Type: Improvement > Components: QA >Affects Versions: s11 >Reporter: Andrew Seyler >Assignee: Dave Thompson >Priority: Major > > review Wiki Errata Implemented/Not Implemented page: > https://opensource.ncsa.illinois.edu/confluence/pages/viewpage.action?pageId=22052868 > synopsis: > Every errata that has a "No" status, and is not about unparsing, we need a > JIRA task, and a requirement (or a few perhaps for the larger errata) to > track them. Well, I guess we need requirements for all of them - I know Tham > did an analysis of this not too far back. > For these errata numbers, I was not sure of the answers. I guessed, but > someone from test really needs to see if I am right here. In some cases I may > have been optimistic, others pessimistic versus reality. > 3.22 - code is there. Do we have test coverage? > 3.29 - I vaguely recall discussion of this. Did tests get created? > 2.66 - tbd: what is the behavior and do we test this? > 2.68 - is this done? > 2.103 - do we have tests about alignment of individual instances of an array > vs. the whole array? > 2.116 - do we check this for defineVariable even? > 2.117 - do we have test coverage of this. I recall the design discussions > with Taylor, but not sure if this is tested. > 2.120 - feels to me like we might have partial coverage of this. > 2.121 - done? tested? > 2.127 - done? tested? > 2.129 - do we check for choice with no branches? > 2.134 - done? tested? > comparing to Eratta at http://redmine.ogf.org/dmsf/dfdl-wg?folder_id=5485 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-798) verify Errata Test Coverage
[ https://issues.apache.org/jira/browse/DAFFODIL-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-798: -- Issue Type: Task (was: Improvement) > verify Errata Test Coverage > --- > > Key: DAFFODIL-798 > URL: https://issues.apache.org/jira/browse/DAFFODIL-798 > Project: Daffodil > Issue Type: Task > Components: QA >Affects Versions: s11 >Reporter: Andrew Seyler >Assignee: Dave Thompson >Priority: Major > > review Wiki Errata Implemented/Not Implemented page: > https://opensource.ncsa.illinois.edu/confluence/pages/viewpage.action?pageId=22052868 > synopsis: > Every errata that has a "No" status, and is not about unparsing, we need a > JIRA task, and a requirement (or a few perhaps for the larger errata) to > track them. Well, I guess we need requirements for all of them - I know Tham > did an analysis of this not too far back. > For these errata numbers, I was not sure of the answers. I guessed, but > someone from test really needs to see if I am right here. In some cases I may > have been optimistic, others pessimistic versus reality. > 3.22 - code is there. Do we have test coverage? > 3.29 - I vaguely recall discussion of this. Did tests get created? > 2.66 - tbd: what is the behavior and do we test this? > 2.68 - is this done? > 2.103 - do we have tests about alignment of individual instances of an array > vs. the whole array? > 2.116 - do we check this for defineVariable even? > 2.117 - do we have test coverage of this. I recall the design discussions > with Taylor, but not sure if this is tested. > 2.120 - feels to me like we might have partial coverage of this. > 2.121 - done? tested? > 2.127 - done? tested? > 2.129 - do we check for choice with no branches? > 2.134 - done? tested? > comparing to Eratta at http://redmine.ogf.org/dmsf/dfdl-wg?folder_id=5485 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-798) verify Errata Test Coverage
[ https://issues.apache.org/jira/browse/DAFFODIL-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-798: -- Issue Type: Improvement (was: Task) > verify Errata Test Coverage > --- > > Key: DAFFODIL-798 > URL: https://issues.apache.org/jira/browse/DAFFODIL-798 > Project: Daffodil > Issue Type: Improvement > Components: QA >Affects Versions: s11 >Reporter: Andrew Seyler >Assignee: Dave Thompson >Priority: Major > > review Wiki Errata Implemented/Not Implemented page: > https://opensource.ncsa.illinois.edu/confluence/pages/viewpage.action?pageId=22052868 > synopsis: > Every errata that has a "No" status, and is not about unparsing, we need a > JIRA task, and a requirement (or a few perhaps for the larger errata) to > track them. Well, I guess we need requirements for all of them - I know Tham > did an analysis of this not too far back. > For these errata numbers, I was not sure of the answers. I guessed, but > someone from test really needs to see if I am right here. In some cases I may > have been optimistic, others pessimistic versus reality. > 3.22 - code is there. Do we have test coverage? > 3.29 - I vaguely recall discussion of this. Did tests get created? > 2.66 - tbd: what is the behavior and do we test this? > 2.68 - is this done? > 2.103 - do we have tests about alignment of individual instances of an array > vs. the whole array? > 2.116 - do we check this for defineVariable even? > 2.117 - do we have test coverage of this. I recall the design discussions > with Taylor, but not sure if this is tested. > 2.120 - feels to me like we might have partial coverage of this. > 2.121 - done? tested? > 2.127 - done? tested? > 2.129 - do we check for choice with no branches? > 2.134 - done? tested? > comparing to Eratta at http://redmine.ogf.org/dmsf/dfdl-wg?folder_id=5485 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1629) Add Calabash Extension to regression suite
[ https://issues.apache.org/jira/browse/DAFFODIL-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1629: --- Issue Type: Improvement (was: Task) > Add Calabash Extension to regression suite > -- > > Key: DAFFODIL-1629 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1629 > Project: Daffodil > Issue Type: Improvement > Components: Integrations, QA >Reporter: Steve Lawrence >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-947) Warn when space separated fields start or end with a space
[ https://issues.apache.org/jira/browse/DAFFODIL-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-947: -- Issue Type: Improvement (was: Task) > Warn when space separated fields start or end with a space > -- > > Key: DAFFODIL-947 > URL: https://issues.apache.org/jira/browse/DAFFODIL-947 > Project: Daffodil > Issue Type: Improvement > Components: Diagnostics, Usability >Reporter: Steve Lawrence >Priority: Minor > > If a user specifies something like dfdl:separator=" ", they probably intended > to have dfdl:separator="%SP;". However, since dfdl:separator is a space > separated list of separators, this is treated as if there is no separator. We > should warn the user when they have spaces at the beginning or end of for > dfdl properties that are space separated lists. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-902) Fix tests that fail due to dfdl schema validation.
[ https://issues.apache.org/jira/browse/DAFFODIL-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-902: -- Component/s: Infrastructure > Fix tests that fail due to dfdl schema validation. > -- > > Key: DAFFODIL-902 > URL: https://issues.apache.org/jira/browse/DAFFODIL-902 > Project: Daffodil > Issue Type: Task > Components: Infrastructure, QA >Reporter: Taylor Wise >Priority: Major > > Perhaps the right thing is to use a flag like this for now, but just add a > jira task to find and fix all of the tests that have this. > Locate tests that have the validateDFDLSchemas flag set to false. Correct > the tests that fail when setting the flag to true. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-902) Fix tests that fail due to dfdl schema validation.
[ https://issues.apache.org/jira/browse/DAFFODIL-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-902: -- Issue Type: Improvement (was: Task) > Fix tests that fail due to dfdl schema validation. > -- > > Key: DAFFODIL-902 > URL: https://issues.apache.org/jira/browse/DAFFODIL-902 > Project: Daffodil > Issue Type: Improvement > Components: Infrastructure, QA >Reporter: Taylor Wise >Priority: Major > > Perhaps the right thing is to use a flag like this for now, but just add a > jira task to find and fix all of the tests that have this. > Locate tests that have the validateDFDLSchemas flag set to false. Correct > the tests that fail when setting the flag to true. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1298) Duplicate test cases in Functions.tdml
[ https://issues.apache.org/jira/browse/DAFFODIL-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1298: --- Issue Type: Bug (was: Task) > Duplicate test cases in Functions.tdml > -- > > Key: DAFFODIL-1298 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1298 > Project: Daffodil > Issue Type: Bug > Components: Clean Ups, QA >Affects Versions: 1.0.0 >Reporter: Michael Beckerle >Priority: Major > > I tweeked the TDML runner to reject duplicate test names in the same TDML > file, and I found that the file Functions.tdml has quite a few duplicates in > it. These tests are passing because the first one found is used. The later > test with duplicate name is not used. > However, really we should not be depending on this. All tests in a TDML file > should have unique names. The schema for TDML files is supposed to have a > uniqueness constraint that assures this, but it seems either that uniqueness > constraint is not properly expressed, or we're invoking the validation of > TDML files without the options that tell it to validate the key/uniqueness > constraints (which can be disabled because they are so expensive to execute > by the Xerces XML validator). I believe the TDML runner should turn on "full > schema checking" or something like that to get the uniqueness constraints > checked. (TDML runner defect is DFDL-1299) > Here's the list of duplicates from Functions.tdml > More than one test case for name 'hexBinary_constructor_03'. > More than one test case for name 'nonNeg_constructor_03'. > More than one test case for name 'byte_constructor_03'. > More than one test case for name 'uint_constructor_01'. > More than one test case for name 'hexBinary_constructor_04'. > More than one test case for name 'uint_constructor_04'. > More than one test case for name 'byte_constructor_04'. > More than one test case for name 'ubyte_constructor_01'. > More than one test case for name 'ubyte_constructor_02'. > More than one test case for name 'ubyte_constructor_03'. > More than one test case for name 'hexBinary_constructor_01'. > More than one test case for name 'nonNeg_constructor_01'. > More than one test case for name 'byte_constructor_01'. > More than one test case for name 'uint_constructor_03'. > More than one test case for name 'date_constructor_01'. > More than one test case for name 'hexBinary_constructor_02'. > More than one test case for name 'uint_constructor_02'. > More than one test case for name 'nonNeg_constructor_04'. > More than one test case for name 'byte_constructor_02'. > More than one test case for name 'ubyte_constructor_04'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1157) Ensure all DFDL schemas have a .dfdl.xsd extension
[ https://issues.apache.org/jira/browse/DAFFODIL-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1157. -- Resolution: Fixed Fix Version/s: 2.2.0 2.3.0 built-in-formats.xsd replaced by DFDLGeneralFormat.dfdl.xsd. Others should be listed as separate bugs if found. It's not a bug (for Daffodil) if some schema doesn't follow this naming convention. IBMers don't actually use this convention. > Ensure all DFDL schemas have a .dfdl.xsd extension > -- > > Key: DAFFODIL-1157 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1157 > Project: Daffodil > Issue Type: Task > Components: DFDL Schemas >Affects Versions: s15 >Reporter: Elizabeth Fahl >Priority: Minor > Fix For: 2.3.0, 2.2.0 > > > All DFDL schemas should be checked and updated to end in ".dfdl.xsd". > built-in-formats.xsd > (daffodil-lib/src/main/resources/xsd/built-in-formats.xsd) is an example of a > file that needs to be renamed but there might be more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1300) Fix Test Scala files to not create the test suite repeatedly
[
https://issues.apache.org/jira/browse/DAFFODIL-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle updated DAFFODIL-1300:
---
Issue Type: Improvement (was: Task)
> Fix Test Scala files to not create the test suite repeatedly
>
>
> Key: DAFFODIL-1300
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1300
> Project: Daffodil
> Issue Type: Improvement
> Components: QA
>Reporter: Michael Beckerle
>Priority: Major
>
> Running tests seemed to be awfully slow.
> So I learned today that for every single test case line in our test scala
> files (e.g, TestFacets.scala), it reloads the entire test suite TDML from
> scratch. This makes the tests much slower of course, particularly so for the
> large TDML files (Functions.tdml for example).
> The fix to this is to look at how TestFacets.scala is now (just pushed to
> 1.1.0 branch).
> This is the recipe for loading up the test suite exactly once, but also
> releasing it at the end of the job so that these things don't build up
> endlessly in memory. Basically, all those variables that hold the test runner
> objects, all those go in the associated peer object, not in the test class.
> E.g., from TestFacets.scala:
> object TestFacets {
> val testDir = "/edu/illinois/ncsa/daffodil/section05/facets/"
> val aa = testDir + "Facets.tdml"
> var runner = new DFDLTestSuite(Misc.getRequiredResource(aa),
> validateTDMLFile =
> false, validateDFDLSchemas = false)
> /**
>* Avoid memory leak of adding more and more test suites to static objects
> as we run more and more test suites.
>*/
> @AfterClass def tearDown() { runner = null }
> }
> class TestFacets {
> import TestFacets._ // Imports all of the definitions in the object so we
> don't have to edit the test lines.
> @Test def test_minMaxInExdateTime01() {
> runner.runOneTest("minMaxInExdateTime01") }
> ...
> }
> Tests run much faster once this is done.
> For now I have retained noisy messages which illustrate how often we are
> reloading the TDML files to construct the test suite objects.
> There are also warnings to stderr for duplicate test cases in the same tdml
> file -(see JIRA Ticket DFDL-1298 and DFDL-1299. )
> We should make a pass and fix all the TDML running scala test files to use
> this technique.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DAFFODIL-20) obtain and integrate/run IBM test cases (even if they all break)
[ https://issues.apache.org/jira/browse/DAFFODIL-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-20: - Issue Type: Improvement (was: Task) > obtain and integrate/run IBM test cases (even if they all break) > > > Key: DAFFODIL-20 > URL: https://issues.apache.org/jira/browse/DAFFODIL-20 > Project: Daffodil > Issue Type: Improvement > Components: Clean Ups, Compatibility, QA >Reporter: Joe Futrelle >Assignee: Michael Beckerle >Priority: Major > Labels: ForInteroperabilityTest > > Revisit when advanced text number properties are implemented -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1458) pull-style unparser
[
https://issues.apache.org/jira/browse/DAFFODIL-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Beckerle closed DAFFODIL-1458.
--
Resolution: Won't Fix
pointless wish list item.
Many varieties of APIs for Daffodil are desirable. Unless there's a specific
need for this one, we don't want to list it. It's cool that this could be done,
but not worth a tracker/ticket because that requires people to sift it
periodically.
> pull-style unparser
> ---
>
> Key: DAFFODIL-1458
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1458
> Project: Daffodil
> Issue Type: Wish
> Components: API, Back End, Unparsing
>Reporter: Michael Beckerle
>Priority: Major
>
> Using java.io.PipedOutputStream, java.io.PipedInputStream, and InvertControl,
> we should be able to implement a pull-style unparser where normal reading
> from some inputStream in effect drives the unparsing to an outputStream that
> feeds into the input stream.
> {code}
> val inputStream: InputStream = DataProcessor.pullUnparser(infosetCursor)
> {code}
> Reading bytes from the inputStream will force pulling of buffers of data from
> an internal output stream and co-routines with the unparser that is
> populating the outputStream.
> A throw of an UnparseError must occur on the main co-routine if the
> pullUnparser encounters such. This will occur when data being pulled contains
> the representation of the element whose unparsing creates the error, not
> before.
> Ex: if there's 100 unparsed bytes in the buffer, but what's next should be
> the representation of an outputValueCalc element whose expression caused an
> UnparseError. Well the 100 bytes should be delivered all the way to the
> inputStream and from it to the application reading from it. Only on the next
> read call on the inputStream should that cause the UnparseError to be issued.
> However, the error thrown may aggregate multiple UnparseErrors &
> RuntimeSchemaDefinitionErrors, as not only the first one whose element
> creates the error should be reported, but potentially there are others from
> unparsing of later elements (into buffers) that are relevant to understanding
> what went wrong while unparsing.
> A buffered output stream "block size" can avoid excessive context switching
> by delaying the resume of the consuming co-routine (main, which is reading
> from the inputStream) until a block of data has been produced by the
> unparser. (Note that the minimum block size would be 1 byte, even though
> unparsing may occur at the level of individual bits.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DAFFODIL-894) utf16Width property - unpaired surrogate behavior
[ https://issues.apache.org/jira/browse/DAFFODIL-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660630#comment-16660630 ] Michael Beckerle commented on DAFFODIL-894: --- Current sentiment in the DFDL Workgroup is to make utf16Width='variable' optional functionality, because it's not really feasible to implement in the JVM/Java world. In that case this would be a feature we would consciously choose not to implement in Daffodil, at least as it is working today (JVM-language, etc.) > utf16Width property - unpaired surrogate behavior > - > > Key: DAFFODIL-894 > URL: https://issues.apache.org/jira/browse/DAFFODIL-894 > Project: Daffodil > Issue Type: Wish > Components: Back End >Affects Versions: s11 >Reporter: Michael Beckerle >Priority: Major > > Part of DFDL-893 but worth carving off as a lower-priority aspect. > This was a comment on the original January 2011 version 1.0.3 spec of DFDL. > The Sept 2013 1.0.4 spec says: > When utf16Width is 'variable', then on parsing an un-paired surrogate > codepoint causes a decode error, which can be controlled via > dfdl:encodingErrorPolicy. > Is this the actual behavior, i.e., does ICU let us even do this? (If not then > this will be an issue with the DFDL spec.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1386) single utf-8 4-byte character becomes surrogate character pairs in scala/java string
[ https://issues.apache.org/jira/browse/DAFFODIL-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1386. -- Resolution: Won't Fix Pointless wish list item. Nobody asking for this. It's not really feasible to fix, closing. > single utf-8 4-byte character becomes surrogate character pairs in scala/java > string > > > Key: DAFFODIL-1386 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1386 > Project: Daffodil > Issue Type: Wish > Components: Back End >Reporter: Michael Beckerle >Priority: Major > > Recent changes in 1.2.0 to the data input layers removed a feature which is > the ability to treat surrogate pair characters as single characters. > See test_encodingNoError. > This test has a TDML representation where a single character in utf-8 that > has a 4-byte encoding has to become a surrogate-pair (two codepoints) in a > java/scala string, but the data input stream's char iterator on a call to > next() returns only 1 codepoint. There is no accomodation in the data input > stream layers for the possibility of a single character needing 2 codepoints. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1282) Abort: Invariant broken when parser is fed random junk
[ https://issues.apache.org/jira/browse/DAFFODIL-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660624#comment-16660624 ] Michael Beckerle commented on DAFFODIL-1282: See also DAFFODIL-1602 about adding fuzz testing > Abort: Invariant broken when parser is fed random junk > --- > > Key: DAFFODIL-1282 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1282 > Project: Daffodil > Issue Type: Bug > Components: General >Reporter: Jessie Chab >Priority: Major > > I attempted to parse a file that I had accidentally overwritten with a saved > parser. I got the following error: > edu.illinois.ncsa.daffodil.exceptions.Abort: Invariant broken. Unsupported > conversion to Long. (null,null) of type Tuple2 > edu.illinois.ncsa.daffodil.exceptions.Assert$.abort(Assert.scala:107) > edu.illinois.ncsa.daffodil.exceptions.Assert$.invariantFailed(Assert.scala:148) > edu.illinois.ncsa.daffodil.dpath.AsIntConverters$.asLong(AsIntConverters.scala:90) > edu.illinois.ncsa.daffodil.processors.SpecifiedLengthExplicitBytesParser.getLength(SpecifiedLength.scala:314) > at edu.illinois.ncsa.daffodil.exceptions.Assert$.abort(Assert.scala:107) > at > edu.illinois.ncsa.daffodil.exceptions.Assert$.invariantFailed(Assert.scala:148) > at > edu.illinois.ncsa.daffodil.dpath.AsIntConverters$.asLong(AsIntConverters.scala:90) > at > edu.illinois.ncsa.daffodil.processors.SpecifiedLengthExplicitBytesParser.getLength(SpecifiedLength.scala:314) > at > edu.illinois.ncsa.daffodil.processors.SpecifiedLengthExplicitBytesParser$$anonfun$parse$8.apply(SpecifiedLength.scala:322) > at > edu.illinois.ncsa.daffodil.processors.SpecifiedLengthExplicitBytesParser$$anonfun$parse$8.apply(SpecifiedLength.scala:319) > at > edu.illinois.ncsa.daffodil.processors.WithParseErrorThrowing$class.withParseErrorThrowing(ParseErrors.scala:217) > at > edu.illinois.ncsa.daffodil.processors.SpecifiedLengthParserBase.withParseErrorThrowing(SpecifiedLength.scala:166) > at > edu.illinois.ncsa.daffodil.processors.SpecifiedLengthExplicitBytesParser.parse(SpecifiedLength.scala:319) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$9.apply(ElementCombinator1.scala:131) > at > edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$9.apply(ElementCombinator1.scala:130) > at scala.Option.map(Option.scala:145) > at > edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:130) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.parsers.Rep$.loopExactlyTotalN(RepParsers.scala:104) > at > edu.illinois.ncsa.daffodil.processors.parsers.RepExactlyTotalOccursCountParser.parseAllRepeats(RepParsers.scala:311) > at > edu.illinois.ncsa.daffodil.processors.parsers.RepParser.parse(RepParsers.scala:76) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$2.apply(Parser.scala:171) > at > edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$2.apply(Parser.scala:169) > at scala.collection.immutable.List.foreach(List.scala:318) > at > edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:169) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.parsers.ArrayCombinatorParser.parse(ElementKindParsers.scala:85) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$2.apply(Parser.scala:171) > at > edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$2.apply(Parser.scala:169) > at scala.collection.immutable.List.foreach(List.scala:318) > at > edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:169) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:69) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:55) > at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:90) > at > edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$9.apply(ElementCombinator1.scala:131) > at >
[jira] [Updated] (DAFFODIL-1940) Automatically publish SNAPSHOTs to repository.apache.org
[ https://issues.apache.org/jira/browse/DAFFODIL-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1940: --- Issue Type: Improvement (was: Bug) > Automatically publish SNAPSHOTs to repository.apache.org > > > Key: DAFFODIL-1940 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1940 > Project: Daffodil > Issue Type: Improvement > Components: Infrastructure >Reporter: Steve Lawrence >Priority: Major > > Some users want to use the latest and greatest version of Daffodil. > Additionally, it would be nice if the schemas on DFDLSchemas github were > tested with the latest version of Daffodil when new Daffodil commits are made. > To support this, we need to set up an automated Jenkins build that is > triggered on each commit of Daffodil and automatically builds and publishes a > SNAPSHOT to repository.apache.org. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
