[
https://issues.apache.org/jira/browse/DAFFODIL-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Lawrence updated DAFFODIL-2861:
-------------------------------------
Description:
If a schema sets {{dfdl:encodingErrorPolicy="error"}}, then Daffodil outputs
this warning:
{quote}
[warning] Schema Definition Warning: dfdl:encodingErrorPolicy="error" is not
yet implemented. The 'replace' value will be used.
{quote}
The idea is we want to be portable with schemas written for IBM DFDL (which
only supports "error" and not "replace"), so we support the "error" property
value but use the our only supported "replace" behavior. In the common case
where there are no encoding errors, the behavior will be exactly the same as
IBM DFDL.
But if there is an encoding error, then Daffodil throws this assertion:
{code}
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! Not Yet Implemented !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
You are using a feature that is not yet implemented:
Some(Not yet implemented: dfdl:encodingErrorPolicy="error"
org.apache.daffodil.lib.exceptions.Assert$.nyi(Assert.scala:151)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decodeOneHandleMalformed(BitsCharsetDecoder.scala:122)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decode(BitsCharsetDecoder.scala:84)
org.apache.daffodil.io.InputSourceDataInputStreamCharIterator.fetch(InputSourceDataInputStream.scala:906))
{code}
So even though the warning says we are going to use "replace", we still clearly
use "error", which results in an assertion failure that we don't support it.
I confirmed this occurs at least as far back as Daffodil 3.0.0.
Here is a schema that reproduces the issue:
{code:xml}
<include
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormatPortable.dfdl.xsd" />
<annotation>
<appinfo source="http://www.ogf.org/dfdl/";>
<dfdl:format ref="ex:GeneralFormat"/>
</appinfo>
</annotation>
<element name="test" type="xs:string"
dfdl:lengthKind="delimited" dfdl:terminator="%NL;"
dfdl:encoding="ASCII" />
{code}
Any file that non-ASCII characters followed by a new line should trigger this.
was:
If a schema sets {{dfdl:encodingErrorPolicy="error"}}, then Daffodil outputs
this warning:
> [warning] Schema Definition Warning: dfdl:encodingErrorPolicy="error" is not
> yet implemented. The 'replace' value will be used.
The idea is we want to be portable with schemas written for IBM DFDL (which
only supports "error" and not "replace"), so we support the "error" property
value but use the our only supported "replace" behavior. In the common case
where there are no encoding errors, the behavior will be exactly the same as
IBM DFDL.
But if there is an encoding error, then Daffodil throws this assertion:
{code}
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! Not Yet Implemented !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
You are using a feature that is not yet implemented:
Some(Not yet implemented: dfdl:encodingErrorPolicy="error"
org.apache.daffodil.lib.exceptions.Assert$.nyi(Assert.scala:151)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decodeOneHandleMalformed(BitsCharsetDecoder.scala:122)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decode(BitsCharsetDecoder.scala:84)
org.apache.daffodil.io.InputSourceDataInputStreamCharIterator.fetch(InputSourceDataInputStream.scala:906))
{code}
So even though the warning says we are going to use "replace", we still clearly
use "error", which results in an assertion failure that we don't support it.
I confirmed this occurs at least as far back as Daffodil 3.0.0.
Here is a schema that reproduces the issue:
{code:xml}
<include
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormatPortable.dfdl.xsd" />
<annotation>
<appinfo source="http://www.ogf.org/dfdl/";>
<dfdl:format ref="ex:GeneralFormat"/>
</appinfo>
</annotation>
<element name="test" type="xs:string"
dfdl:lengthKind="delimited" dfdl:terminator="%NL;"
dfdl:encoding="ASCII" />
{code}
Any file that non-ASCII characters followed by a new line should trigger this.
> Assertion thrown when encoding error with dfdl:encodingErrorPolicy="error"
> --------------------------------------------------------------------------
>
> Key: DAFFODIL-2861
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2861
> Project: Daffodil
> Issue Type: Bug
> Components: Back End
> Affects Versions: 3.0.0
> Reporter: Steve Lawrence
> Priority: Major
>
> If a schema sets {{dfdl:encodingErrorPolicy="error"}}, then Daffodil outputs
> this warning:
> {quote}
> [warning] Schema Definition Warning: dfdl:encodingErrorPolicy="error" is not
> yet implemented. The 'replace' value will be used.
> {quote}
> The idea is we want to be portable with schemas written for IBM DFDL (which
> only supports "error" and not "replace"), so we support the "error" property
> value but use the our only supported "replace" behavior. In the common case
> where there are no encoding errors, the behavior will be exactly the same as
> IBM DFDL.
> But if there is an encoding error, then Daffodil throws this assertion:
> {code}
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> !! Not Yet Implemented !!
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> You are using a feature that is not yet implemented:
> Some(Not yet implemented: dfdl:encodingErrorPolicy="error"
> org.apache.daffodil.lib.exceptions.Assert$.nyi(Assert.scala:151)
> org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decodeOneHandleMalformed(BitsCharsetDecoder.scala:122)
> org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decode(BitsCharsetDecoder.scala:84)
> org.apache.daffodil.io.InputSourceDataInputStreamCharIterator.fetch(InputSourceDataInputStream.scala:906))
> {code}
> So even though the warning says we are going to use "replace", we still
> clearly use "error", which results in an assertion failure that we don't
> support it.
> I confirmed this occurs at least as far back as Daffodil 3.0.0.
> Here is a schema that reproduces the issue:
> {code:xml}
> <include
> schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormatPortable.dfdl.xsd"
> />
> <annotation>
> <appinfo source="http://www.ogf.org/dfdl/";>
> <dfdl:format ref="ex:GeneralFormat"/>
> </appinfo>
> </annotation>
> <element name="test" type="xs:string"
> dfdl:lengthKind="delimited" dfdl:terminator="%NL;"
> dfdl:encoding="ASCII" />
> {code}
> Any file that non-ASCII characters followed by a new line should trigger this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
