[jira] [Created] (DAFFODIL-2861) Assertion thrown when encoding error with dfdl:encodingErrorPolicy="error"

[Steve Lawrence (Jira)]

Steve Lawrence created DAFFODIL-2861:
----------------------------------------

             Summary: Assertion thrown when encoding error with 
dfdl:encodingErrorPolicy="error"
                 Key: DAFFODIL-2861
                 URL: https://issues.apache.org/jira/browse/DAFFODIL-2861
             Project: Daffodil
          Issue Type: Bug
          Components: Back End
    Affects Versions: 3.0.0
            Reporter: Steve Lawrence


If a schema sets {{dfdl:encodingErrorPolicy="error"}}, then Daffodil outputs 
this warning:

> [warning] Schema Definition Warning: dfdl:encodingErrorPolicy="error" is not 
> yet implemented. The 'replace' value will be used.

The idea is we want to be portable with schemas written for IBM DFDL (which 
only supports "error" and not "replace"), so we support the "error" property 
value but use the our only supported "replace" behavior. In the common case 
where there are no encoding errors, the behavior will be exactly the same as 
IBM DFDL.

But if there is an encoding error, then Daffodil throws this assertion:
{code}
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!                 Not Yet Implemented                  !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 You are using a feature that is not yet implemented:

 Some(Not yet implemented: dfdl:encodingErrorPolicy="error"
org.apache.daffodil.lib.exceptions.Assert$.nyi(Assert.scala:151)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decodeOneHandleMalformed(BitsCharsetDecoder.scala:122)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decode(BitsCharsetDecoder.scala:84)
org.apache.daffodil.io.InputSourceDataInputStreamCharIterator.fetch(InputSourceDataInputStream.scala:906))
{code}

So even though the warning says we are going to use "replace", we still clearly 
use "error", which results in an assertion failure that we don't support it.

I confirmed this occurs at least as far back as Daffodil 3.0.0.

Here is a schema that reproduces the issue:

{code:xml}
  <include 
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormatPortable.dfdl.xsd" />

  <annotation>
    <appinfo source="http://www.ogf.org/dfdl/";>
      <dfdl:format ref="ex:GeneralFormat"/>
    </appinfo>
  </annotation>

  <element name="test" type="xs:string"
    dfdl:lengthKind="delimited" dfdl:terminator="%NL;"
    dfdl:encoding="ASCII" />
{code}

Any file that non-ASCII characters followed by a new line should trigger this.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)