[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #194: Fix poor diagnostic.

[GitBox]

mbeckerle commented on a change in pull request #194: Fix poor diagnostic.
URL: https://github.com/apache/incubator-daffodil/pull/194#discussion_r263957341
 
 

 ##
 File path: 
daffodil-lib/src/main/scala/org/apache/daffodil/util/SchemaUtils.scala
 ##
 @@ -97,7 +94,7 @@ object SchemaUtils {
 scope = XMLUtils.combineScopes("ex", targetNamespace, scope)
 
 val schemaNode =
-  
+  
 
 Review comment:
   This is the DAFFODIL-2084 fix.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: Travis auto build/test failed - but I can't figure out why. Was: Fw: Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)

[Steve Lawrence]

Yeah, it's really hard to find errors in our SBT output, especially on
SBT output. I'd like to change this so test only output failures at
least when run on Travis, but there's a bug in sbt that prevents
changing the log level for tests [1]. I was able to find the two tests
that fail:

org.apache.daffodil.section12.delimiter_properties.TestDelimiterProperties.test_DelimProp_10_02
org.apache.daffodil.section12.delimiter_properties.TestDelimiterProperties.test_DelimProp_10_03

Note that TravisCI only shows the first 1 lines of the output, to
see more you need to download the raw log.

- Steve


[1] https://github.com/sbt/sbt/issues/4480

On 3/8/19 5:42 PM, Beckerle, Mike wrote:
> So I pushed a change and created a PR. 
> https://github.com/apache/incubator-daffodil/pull/194
> 
> 
> I got back a TravisCI failure. But when I review the log (which is HUGE), I 
> can't see why it is failing.
> 
> sbt test, sbt it:test, and ratCheck all pass.
> 
> 
> The "[error]" tagged lines all look to be expected from negative it:test that 
> are specifically creating errors. I see those when I run sbt it:test on my 
> local machine.
> 
> 
> So I can't see where the TravisCI failure is coming from.
> 
> 
> The change set for this PR is pretty darn small, and is about diagnostics so 
> should have zero performance impact.
> 
> 
> Any insights appreciated.
> 
> 
> ...mike beckerle
> 
> Tresys
> 
> 
> 
> From: Travis CI 
> Sent: Friday, March 8, 2019 5:08 PM
> To: Beckerle, Mike
> Subject: Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)
> 
> 
> 
> mbeckerle
> 
> /
> 
> incubator-daffodil
> 
> 
> [branch 
> icon]daffodil-2092
> 
> 
> [build has failed]
> Build #101 
> failed
> [arrow to build time]
> [clock icon]1 hr, 27 mins, and 44 secs
> 
> 
> [Michael Beckerle avatar]Michael Beckerle
> 
> 0972ac9 CHANGESET 
> →
> 
> 
> Fix poor diagnostic. Fix spurious attributeFormDefault.
> 
> DAFFODIL-2092, DAFFODIL-2084
> 
> 
> 
> 
> Want to know about upcoming build environment updates?
> 
> Would you like to stay up-to-date with the upcoming Travis CI build 
> environment updates? We set up a mailing list for you!
> 
> SIGN UP HERE
> 
> [book icon]
> 
> Documentation about Travis CI
> 
> 
> 
> Have any questions? We're here to help.
> Unsubscribe
>  from build emails from the mbeckerle/incubator-daffodil repository.
> To unsubscribe from all build emails, please update your 
> settings.
> [black and white travis ci logo]
> 
> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF: Konstantin Haase, 
> Fritz Thielemann | Contact: 
> cont...@travis-ci.com | Amtsgericht 
> Charlottenburg, Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 a 
> Umsatzsteuergesetz: DE282002648
> 
> 

Re: Travis auto build/test failed - but I can't figure out why. Was: Fw: Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)

[Beckerle, Mike]

Apologies. Nevermind. As usual, pilot error.



From: Beckerle, Mike
Sent: Friday, March 8, 2019 5:42:43 PM
To: dev@daffodil.apache.org
Subject: Travis auto build/test failed - but I can't figure out why. Was: Fw: 
Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)


So I pushed a change and created a PR. 
https://github.com/apache/incubator-daffodil/pull/194


I got back a TravisCI failure. But when I review the log (which is HUGE), I 
can't see why it is failing.

sbt test, sbt it:test, and ratCheck all pass.


The "[error]" tagged lines all look to be expected from negative it:test that 
are specifically creating errors. I see those when I run sbt it:test on my 
local machine.


So I can't see where the TravisCI failure is coming from.


The change set for this PR is pretty darn small, and is about diagnostics so 
should have zero performance impact.


Any insights appreciated.


...mike beckerle

Tresys



From: Travis CI 
Sent: Friday, March 8, 2019 5:08 PM
To: Beckerle, Mike
Subject: Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)



mbeckerle

/

incubator-daffodil


[branch 
icon]daffodil-2092


[build has failed]
Build #101 
failed
[arrow to build time]
[clock icon]1 hr, 27 mins, and 44 secs


[Michael Beckerle avatar]Michael Beckerle

0972ac9 CHANGESET 
→


Fix poor diagnostic. Fix spurious attributeFormDefault.

DAFFODIL-2092, DAFFODIL-2084




Want to know about upcoming build environment updates?

Would you like to stay up-to-date with the upcoming Travis CI build environment 
updates? We set up a mailing list for you!

SIGN UP HERE

[book icon]

Documentation about Travis CI



Have any questions? We're here to help.
Unsubscribe
 from build emails from the mbeckerle/incubator-daffodil repository.
To unsubscribe from all build emails, please update your 
settings.
[black and white travis ci logo]

Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF: Konstantin Haase, 
Fritz Thielemann | Contact: cont...@travis-ci.com 
| Amtsgericht Charlottenburg, Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 
a Umsatzsteuergesetz: DE282002648

Travis auto build/test failed - but I can't figure out why. Was: Fw: Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)

[Beckerle, Mike]

So I pushed a change and created a PR. 
https://github.com/apache/incubator-daffodil/pull/194


I got back a TravisCI failure. But when I review the log (which is HUGE), I 
can't see why it is failing.

sbt test, sbt it:test, and ratCheck all pass.


The "[error]" tagged lines all look to be expected from negative it:test that 
are specifically creating errors. I see those when I run sbt it:test on my 
local machine.


So I can't see where the TravisCI failure is coming from.


The change set for this PR is pretty darn small, and is about diagnostics so 
should have zero performance impact.


Any insights appreciated.


...mike beckerle

Tresys



From: Travis CI 
Sent: Friday, March 8, 2019 5:08 PM
To: Beckerle, Mike
Subject: Failed: mbeckerle/incubator-daffodil#101 (daffodil-2092 - 0972ac9)



mbeckerle

/

incubator-daffodil


[branch 
icon]daffodil-2092


[build has failed]
Build #101 
failed
[arrow to build time]
[clock icon]1 hr, 27 mins, and 44 secs


[Michael Beckerle avatar]Michael Beckerle

0972ac9 CHANGESET 
→


Fix poor diagnostic. Fix spurious attributeFormDefault.

DAFFODIL-2092, DAFFODIL-2084




Want to know about upcoming build environment updates?

Would you like to stay up-to-date with the upcoming Travis CI build environment 
updates? We set up a mailing list for you!

SIGN UP HERE

[book icon]

Documentation about Travis CI



Have any questions? We're here to help.
Unsubscribe
 from build emails from the mbeckerle/incubator-daffodil repository.
To unsubscribe from all build emails, please update your 
settings.
[black and white travis ci logo]

Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF: Konstantin Haase, 
Fritz Thielemann | Contact: cont...@travis-ci.com 
| Amtsgericht Charlottenburg, Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 
a Umsatzsteuergesetz: DE282002648

[jira] [Commented] (DAFFODIL-2084) attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - generates spurious warning

[Michael Beckerle (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788309#comment-16788309
 ] 

Michael Beckerle commented on DAFFODIL-2084:


Sorry, just saw this.

 

The fix is one-line change. I included it in this Pull Request:

 

[https://github.com/apache/incubator-daffodil/pull/194]

 

 

> attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - 
> generates spurious warning
> ---
>
> Key: DAFFODIL-2084
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2084
> Project: Daffodil
>  Issue Type: Bug
>  Components: TDML Runner
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Major
>  Labels: beginner
> Fix For: 2.4.0
>
>
> attributeFormDefault="unqualified" appears in SchemaUtils.scala, and is added 
> to some of the schemas it assembles. 
> I don't know why this would be there, this utility is synthesizing DFDL 
> schemas, which don't have attributes.
> This causes a schema definition warning about attributeFormDefault.
> This should simply not be there.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DAFFODIL-2092) Bad diagnostic when using dynamic separator the value of which is NUL (ascii 0)

[Michael Beckerle (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788307#comment-16788307
 ] 

Michael Beckerle commented on DAFFODIL-2092:


Fix in review: [https://github.com/apache/incubator-daffodil/pull/194]

 

> Bad diagnostic when using dynamic separator the value of which is NUL (ascii 
> 0)
> ---
>
> Key: DAFFODIL-2092
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2092
> Project: Daffodil
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Major
> Attachments: my.dfdl.xsd, my2.csv
>
>
> This test actually shows up two bugs.
> One bug, which appears in both 2.2.0 and 2.3.0 is that if you have
> {code:java}
> dfdl:separator="{ ./sep }"{code}
> and the value of the sep element is a string containing a single NUL 
> character, then on 2.2.0 it complains the separator cannot be an empty 
> string. Clearly this string is not empty.
> On 2.3.0, the diagnostic message also complains about what is in the string 
> somewhat differently however. The message is:
> {code:java}
> [error] Schema Definition Error: The property 'null' cannot start or end with 
> the string " ", did you mean to use '%SP;' instead?
> Schema context: sequence[2] Location line 80 column 10 in 
> file:/home/mbeckerle-unencrypted/DFDLSchemas/CSV/src/test/resources/com/tresys/my.dfdl.xsd{code}
> Note that there are two problems here. First is "The property 'null' cannot 
> " that's broken because the property name isn't being provided. Instead 
> it somehow has 'null' as the property name.
> Second it somehow thinks there is whitespace/space in the delimiter.
> The DFDL schema and test data file are attached to this bug.  If placed in 
> the same directory, the test can be run from the CLI using:
> {code:java}
> daffodil -v parse  -s my.dfdl.xsd  my2.csv{code}
> The data consists of a header row, then a set of data row lines. Each data 
> row begins with a single character, which goes into the Sep element, and that 
> element's value is to be used as the separator for the remaining elements of 
> the row.
> When this value of Sep is an ordinary character such as "," or "|", things 
> work fine.
> When this value of Sep is an Ascii NUL, these errors occur.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #194: Fix poor diagnostic.

[GitBox]

mbeckerle commented on a change in pull request #194: Fix poor diagnostic.
URL: https://github.com/apache/incubator-daffodil/pull/194#discussion_r263927525
 
 

 ##
 File path: 
daffodil-lib/src/main/scala/org/apache/daffodil/cookers/EntityReplacer.scala
 ##
 @@ -47,8 +47,7 @@ final class EntityReplacer {
   val dfdlCharClassEntityName = "NL|WSP|WSP\\*|WSP\\+|ES"
 
   val entityCharacterUnicode: List[(String, String, Matcher)] =
-List(
-  ("NUL", "\u", Pattern.compile("%" + "NUL" + ";", 
Pattern.MULTILINE).matcher("")),
+List(("NUL", "\u", Pattern.compile("%" + "NUL" + ";", 
Pattern.MULTILINE).matcher("")),
 
 Review comment:
   Does not seem to be any way to avoid, other than turning off all 
auto-formatting.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #194: Fix poor diagnostic.

[GitBox]

mbeckerle commented on a change in pull request #194: Fix poor diagnostic.
URL: https://github.com/apache/incubator-daffodil/pull/194#discussion_r263926097
 
 

 ##
 File path: 
daffodil-lib/src/main/scala/org/apache/daffodil/cookers/EntityReplacer.scala
 ##
 @@ -47,8 +47,7 @@ final class EntityReplacer {
   val dfdlCharClassEntityName = "NL|WSP|WSP\\*|WSP\\+|ES"
 
   val entityCharacterUnicode: List[(String, String, Matcher)] =
-List(
-  ("NUL", "\u", Pattern.compile("%" + "NUL" + ";", 
Pattern.MULTILINE).matcher("")),
+List(("NUL", "\u", Pattern.compile("%" + "NUL" + ";", 
Pattern.MULTILINE).matcher("")),
 
 Review comment:
   Just eclipse changing the whitespace I will see if this can be avoided.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #194: Fix poor diagnostic.

[GitBox]

mbeckerle commented on a change in pull request #194: Fix poor diagnostic.
URL: https://github.com/apache/incubator-daffodil/pull/194#discussion_r263925869
 
 

 ##
 File path: 
daffodil-japi/src/test/java/org/apache/daffodil/example/TestJavaAPI.java
 ##
 @@ -98,7 +98,7 @@ private DataProcessor reserializeDataProcessor(DataProcessor 
dp) throws IOExcept
  * For more information, see https://github.com/sbt/sbt/issues/163
  */
 @Override
-protected Class resolveClass(java.io.ObjectStreamClass desc) 
throws IOException, ClassNotFoundException {
+protected Class resolveClass(java.io.ObjectStreamClass desc) 
throws IOException, ClassNotFoundException {
 
 Review comment:
   This was creating a warning in Eclipse for me. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (DAFFODIL-2084) attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - generates spurious warning

[Mark C. Otto (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788278#comment-16788278
 ] 

Mark C. Otto commented on DAFFODIL-2084:


Yup. DAFFODIL-2084.

Mark Otto
Managing Principal, Services
P: 410.290.1411 x161
mo...@tresys.com | tresys.com



> attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - 
> generates spurious warning
> ---
>
> Key: DAFFODIL-2084
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2084
> Project: Daffodil
>  Issue Type: Bug
>  Components: TDML Runner
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Major
>  Labels: beginner
> Fix For: 2.4.0
>
>
> attributeFormDefault="unqualified" appears in SchemaUtils.scala, and is added 
> to some of the schemas it assembles. 
> I don't know why this would be there, this utility is synthesizing DFDL 
> schemas, which don't have attributes.
> This causes a schema definition warning about attributeFormDefault.
> This should simply not be there.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DAFFODIL-2084) attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - generates spurious warning

[Steve Lawrence (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788276#comment-16788276
 ] 

Steve Lawrence commented on DAFFODIL-2084:
--

FYI, Mark Otto mentioned working on a bug but an problem with JIRA
didn't allowi him to assign it to himself. I think this was the bug, but
I'm not positive. Mark, can you confirm?



> attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - 
> generates spurious warning
> ---
>
> Key: DAFFODIL-2084
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2084
> Project: Daffodil
>  Issue Type: Bug
>  Components: TDML Runner
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Major
>  Labels: beginner
> Fix For: 2.4.0
>
>
> attributeFormDefault="unqualified" appears in SchemaUtils.scala, and is added 
> to some of the schemas it assembles. 
> I don't know why this would be there, this utility is synthesizing DFDL 
> schemas, which don't have attributes.
> This causes a schema definition warning about attributeFormDefault.
> This should simply not be there.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (DAFFODIL-2084) attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - generates spurious warning

[Michael Beckerle (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DAFFODIL-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Beckerle reassigned DAFFODIL-2084:
--

Assignee: Michael Beckerle

> attributeFormDefault="unqualified" appears in SchemaUtils used by TDML - 
> generates spurious warning
> ---
>
> Key: DAFFODIL-2084
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2084
> Project: Daffodil
>  Issue Type: Bug
>  Components: TDML Runner
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Major
>  Labels: beginner
> Fix For: 2.4.0
>
>
> attributeFormDefault="unqualified" appears in SchemaUtils.scala, and is added 
> to some of the schemas it assembles. 
> I don't know why this would be there, this utility is synthesizing DFDL 
> schemas, which don't have attributes.
> This causes a schema definition warning about attributeFormDefault.
> This should simply not be there.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (DAFFODIL-2092) Bad diagnostic when using dynamic separator the value of which is NUL (ascii 0)

[Michael Beckerle (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DAFFODIL-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Beckerle reassigned DAFFODIL-2092:
--

Assignee: Michael Beckerle

> Bad diagnostic when using dynamic separator the value of which is NUL (ascii 
> 0)
> ---
>
> Key: DAFFODIL-2092
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2092
> Project: Daffodil
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Assignee: Michael Beckerle
>Priority: Major
> Attachments: my.dfdl.xsd, my2.csv
>
>
> This test actually shows up two bugs.
> One bug, which appears in both 2.2.0 and 2.3.0 is that if you have
> {code:java}
> dfdl:separator="{ ./sep }"{code}
> and the value of the sep element is a string containing a single NUL 
> character, then on 2.2.0 it complains the separator cannot be an empty 
> string. Clearly this string is not empty.
> On 2.3.0, the diagnostic message also complains about what is in the string 
> somewhat differently however. The message is:
> {code:java}
> [error] Schema Definition Error: The property 'null' cannot start or end with 
> the string " ", did you mean to use '%SP;' instead?
> Schema context: sequence[2] Location line 80 column 10 in 
> file:/home/mbeckerle-unencrypted/DFDLSchemas/CSV/src/test/resources/com/tresys/my.dfdl.xsd{code}
> Note that there are two problems here. First is "The property 'null' cannot 
> " that's broken because the property name isn't being provided. Instead 
> it somehow has 'null' as the property name.
> Second it somehow thinks there is whitespace/space in the delimiter.
> The DFDL schema and test data file are attached to this bug.  If placed in 
> the same directory, the test can be run from the CLI using:
> {code:java}
> daffodil -v parse  -s my.dfdl.xsd  my2.csv{code}
> The data consists of a header row, then a set of data row lines. Each data 
> row begins with a single character, which goes into the Sep element, and that 
> element's value is to be used as the separator for the remaining elements of 
> the row.
> When this value of Sep is an ordinary character such as "," or "|", things 
> work fine.
> When this value of Sep is an Ascii NUL, these errors occur.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DAFFODIL-2092) Bad diagnostic when using dynamic separator the value of which is NUL (ascii 0)

[Michael Beckerle (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788234#comment-16788234
 ] 

Michael Beckerle commented on DAFFODIL-2092:


This problem happens because the data file my2.csv has an extra LF at the end.

This LF is parsed as the SEP for another record. Then it tries to convert this 
to a separator, but separator is a whitespace separated list, so you get a 
SchemaDefinitionError because a whitespace separated list cannot begin with a 
whitespace character.

So this is not a bug.

The diagnostic message is still erroneous. This should state that for the 
separator property, it is a whitespace-separated list of delimiters, but 
contains only whitespace characters, and should display those characters using 
the utility routine that replaces whitespace characters with the little glyphy 
versions of them. (Or alternatively, provide a hex dump of them., or both.)

 

 

 

 

> Bad diagnostic when using dynamic separator the value of which is NUL (ascii 
> 0)
> ---
>
> Key: DAFFODIL-2092
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2092
> Project: Daffodil
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Priority: Major
> Attachments: my.dfdl.xsd, my2.csv
>
>
> This test actually shows up two bugs.
> One bug, which appears in both 2.2.0 and 2.3.0 is that if you have
> {code:java}
> dfdl:separator="{ ./sep }"{code}
> and the value of the sep element is a string containing a single NUL 
> character, then on 2.2.0 it complains the separator cannot be an empty 
> string. Clearly this string is not empty.
> On 2.3.0, the diagnostic message also complains about what is in the string 
> somewhat differently however. The message is:
> {code:java}
> [error] Schema Definition Error: The property 'null' cannot start or end with 
> the string " ", did you mean to use '%SP;' instead?
> Schema context: sequence[2] Location line 80 column 10 in 
> file:/home/mbeckerle-unencrypted/DFDLSchemas/CSV/src/test/resources/com/tresys/my.dfdl.xsd{code}
> Note that there are two problems here. First is "The property 'null' cannot 
> " that's broken because the property name isn't being provided. Instead 
> it somehow has 'null' as the property name.
> Second it somehow thinks there is whitespace/space in the delimiter.
> The DFDL schema and test data file are attached to this bug.  If placed in 
> the same directory, the test can be run from the CLI using:
> {code:java}
> daffodil -v parse  -s my.dfdl.xsd  my2.csv{code}
> The data consists of a header row, then a set of data row lines. Each data 
> row begins with a single character, which goes into the Sep element, and that 
> element's value is to be used as the separator for the remaining elements of 
> the row.
> When this value of Sep is an ordinary character such as "," or "|", things 
> work fine.
> When this value of Sep is an Ascii NUL, these errors occur.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (DAFFODIL-2092) Bad diagnostic when using dynamic separator the value of which is NUL (ascii 0)

[Michael Beckerle (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788234#comment-16788234
 ] 

Michael Beckerle edited comment on DAFFODIL-2092 at 3/8/19 7:34 PM:


This problem happens because the data file my2.csv has an extra LF at the end.

This LF is parsed as the SEP for another record. Then it tries to convert this 
to a separator, but separator is a whitespace separated list, so you get a 
SchemaDefinitionError because a whitespace separated list cannot begin with a 
whitespace character.

So this is not a bug.

The diagnostic message is still erroneous. This should state that for the 
separator property, it is a whitespace-separated list of delimiters, but 
contains only whitespace characters, and should display those characters using 
the utility routine that replaces whitespace characters with the little glyphy 
versions of them. (Or alternatively, provide a hex dump of them., or both.)


was (Author: mbeckerle):
This problem happens because the data file my2.csv has an extra LF at the end.

This LF is parsed as the SEP for another record. Then it tries to convert this 
to a separator, but separator is a whitespace separated list, so you get a 
SchemaDefinitionError because a whitespace separated list cannot begin with a 
whitespace character.

So this is not a bug.

The diagnostic message is still erroneous. This should state that for the 
separator property, it is a whitespace-separated list of delimiters, but 
contains only whitespace characters, and should display those characters using 
the utility routine that replaces whitespace characters with the little glyphy 
versions of them. (Or alternatively, provide a hex dump of them., or both.)

 

 

 

 

> Bad diagnostic when using dynamic separator the value of which is NUL (ascii 
> 0)
> ---
>
> Key: DAFFODIL-2092
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2092
> Project: Daffodil
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Priority: Major
> Attachments: my.dfdl.xsd, my2.csv
>
>
> This test actually shows up two bugs.
> One bug, which appears in both 2.2.0 and 2.3.0 is that if you have
> {code:java}
> dfdl:separator="{ ./sep }"{code}
> and the value of the sep element is a string containing a single NUL 
> character, then on 2.2.0 it complains the separator cannot be an empty 
> string. Clearly this string is not empty.
> On 2.3.0, the diagnostic message also complains about what is in the string 
> somewhat differently however. The message is:
> {code:java}
> [error] Schema Definition Error: The property 'null' cannot start or end with 
> the string " ", did you mean to use '%SP;' instead?
> Schema context: sequence[2] Location line 80 column 10 in 
> file:/home/mbeckerle-unencrypted/DFDLSchemas/CSV/src/test/resources/com/tresys/my.dfdl.xsd{code}
> Note that there are two problems here. First is "The property 'null' cannot 
> " that's broken because the property name isn't being provided. Instead 
> it somehow has 'null' as the property name.
> Second it somehow thinks there is whitespace/space in the delimiter.
> The DFDL schema and test data file are attached to this bug.  If placed in 
> the same directory, the test can be run from the CLI using:
> {code:java}
> daffodil -v parse  -s my.dfdl.xsd  my2.csv{code}
> The data consists of a header row, then a set of data row lines. Each data 
> row begins with a single character, which goes into the Sep element, and that 
> element's value is to be used as the separator for the remaining elements of 
> the row.
> When this value of Sep is an ordinary character such as "," or "|", things 
> work fine.
> When this value of Sep is an Ascii NUL, these errors occur.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DAFFODIL-2092) Bad diagnostic when using dynamic separator the value of which is NUL (ascii 0)

[Michael Beckerle (JIRA)]

[ 
https://issues.apache.org/jira/browse/DAFFODIL-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788111#comment-16788111
 ] 

Michael Beckerle commented on DAFFODIL-2092:


On 2.2.0, the error you get is:
{code:java}
[error] Schema Definition Error: Property separator cannot be empty 
string.{code}
which is not correct because the string isn't empty, it should contain a single 
NUL character/byte.

> Bad diagnostic when using dynamic separator the value of which is NUL (ascii 
> 0)
> ---
>
> Key: DAFFODIL-2092
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2092
> Project: Daffodil
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
>Priority: Major
> Attachments: my.dfdl.xsd, my2.csv
>
>
> This test actually shows up two bugs.
> One bug, which appears in both 2.2.0 and 2.3.0 is that if you have
> {code:java}
> dfdl:separator="{ ./sep }"{code}
> and the value of the sep element is a string containing a single NUL 
> character, then on 2.2.0 it complains the separator cannot be an empty 
> string. Clearly this string is not empty.
> On 2.3.0, the diagnostic message also complains about what is in the string 
> somewhat differently however. The message is:
> {code:java}
> [error] Schema Definition Error: The property 'null' cannot start or end with 
> the string " ", did you mean to use '%SP;' instead?
> Schema context: sequence[2] Location line 80 column 10 in 
> file:/home/mbeckerle-unencrypted/DFDLSchemas/CSV/src/test/resources/com/tresys/my.dfdl.xsd{code}
> Note that there are two problems here. First is "The property 'null' cannot 
> " that's broken because the property name isn't being provided. Instead 
> it somehow has 'null' as the property name.
> Second it somehow thinks there is whitespace/space in the delimiter.
> The DFDL schema and test data file are attached to this bug.  If placed in 
> the same directory, the test can be run from the CLI using:
> {code:java}
> daffodil -v parse  -s my.dfdl.xsd  my2.csv{code}
> The data consists of a header row, then a set of data row lines. Each data 
> row begins with a single character, which goes into the Sep element, and that 
> element's value is to be used as the separator for the remaining elements of 
> the row.
> When this value of Sep is an ordinary character such as "," or "|", things 
> work fine.
> When this value of Sep is an Ascii NUL, these errors occur.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DAFFODIL-2092) Bad diagnostic when using dynamic separator the value of which is NUL (ascii 0)

[Michael Beckerle (JIRA)]

Michael Beckerle created DAFFODIL-2092:
--

 Summary: Bad diagnostic when using dynamic separator the value of 
which is NUL (ascii 0)
 Key: DAFFODIL-2092
 URL: https://issues.apache.org/jira/browse/DAFFODIL-2092
 Project: Daffodil
  Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Michael Beckerle
 Attachments: my.dfdl.xsd, my2.csv

This test actually shows up two bugs.

One bug, which appears in both 2.2.0 and 2.3.0 is that if you have
{code:java}
dfdl:separator="{ ./sep }"{code}
and the value of the sep element is a string containing a single NUL character, 
then on 2.2.0 it complains the separator cannot be an empty string. Clearly 
this string is not empty.

On 2.3.0, the diagnostic message also complains about what is in the string 
somewhat differently however. The message is:
{code:java}
[error] Schema Definition Error: The property 'null' cannot start or end with 
the string " ", did you mean to use '%SP;' instead?
Schema context: sequence[2] Location line 80 column 10 in 
file:/home/mbeckerle-unencrypted/DFDLSchemas/CSV/src/test/resources/com/tresys/my.dfdl.xsd{code}
Note that there are two problems here. First is "The property 'null' cannot 
" that's broken because the property name isn't being provided. Instead it 
somehow has 'null' as the property name.

Second it somehow thinks there is whitespace/space in the delimiter.

The DFDL schema and test data file are attached to this bug.  If placed in the 
same directory, the test can be run from the CLI using:
{code:java}
daffodil -v parse  -s my.dfdl.xsd  my2.csv{code}
The data consists of a header row, then a set of data row lines. Each data row 
begins with a single character, which goes into the Sep element, and that 
element's value is to be used as the separator for the remaining elements of 
the row.

When this value of Sep is an ordinary character such as "," or "|", things work 
fine.

When this value of Sep is an Ascii NUL, these errors occur.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (DAFFODIL-2078) Unexpected exception with dfdl:contentLength

[Steve Lawrence (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DAFFODIL-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Lawrence resolved DAFFODIL-2078.
--
   Resolution: Fixed
Fix Version/s: 2.4.0

Fixed in commit a6dc540fa13a45e119bbfda77ea4e514ccd1a88f

> Unexpected exception with dfdl:contentLength
> 
>
> Key: DAFFODIL-2078
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2078
> Project: Daffodil
>  Issue Type: Bug
>  Components: Back End
>Affects Versions: 2.2.0
>Reporter: Brandon Sloane
>Assignee: Steve Lawrence
>Priority: Major
> Fix For: 2.4.0
>
> Attachments: test.dfdl.xsd
>
>
> DFDL schema which uses the dfdl:contentLength on a parent node fails to 
> compile, even when the length is explicit.
> I am unsure if the attached schema should compile. In general it is not 
> possible to use dfdl:contentLength on a parent node, as said node has not yet 
> finished parsing. However, when the lengthKind is explicitly the value is 
> knowable. Regardless, if this is not allowed, a better error message is 
> needed.
>  
> To reproduce, run ``` daffodil save-parser -s test.dfdl.xsd test.dfdl.xsd.bin 
> ``` on the attached schema. Relevent portion reproduced below:
> ```
> 
>  
>  
>   dfdl:occursCount='\{ if(dfdl:contentLength(.., "bits") eq 0) then 1 else 2 }' 
> dfdl:inputValueCalc="\{ 1 }" />
>  
>  
>  
> ```
>  
> Full error is:
>  
> ```
> !!
> !! An unexpected exception occurred. This is a bug! !!
> !!
> Please report this bug and help us fix it:
> https://daffodil.apache.org/community/#issue-tracker
> Please include the following exception, the command you
>  ran, and any input, schema, or tdml files used that led
>  to this bug.
> java.lang.NullPointerException
>  at 
> org.apache.daffodil.infoset.InfosetLengthUnknownException.(InfosetImpl.scala:192)
>  at 
> org.apache.daffodil.infoset.InfosetContentLengthUnknownException.(InfosetImpl.scala:201)
>  at 
> org.apache.daffodil.infoset.ContentLengthState.throwUnknown(InfosetImpl.scala:558)
>  at 
> org.apache.daffodil.infoset.LengthState.lengthInBits(InfosetImpl.scala:488)
>  at 
> org.apache.daffodil.dpath.DFDLLengthFunctionBase.$anonfun$getLength$1(DFDLFunctions2.scala:35)
>  at 
> org.apache.daffodil.dpath.DFDLLengthFunctionBase.$anonfun$getLength$1$adapted(DFDLFunctions2.scala:34)
>  at org.apache.daffodil.dpath.DState$.withRetryIfBlocking(DState.scala:364)
>  at 
> org.apache.daffodil.dpath.DFDLLengthFunctionBase.getLength(DFDLFunctions2.scala:34)
>  at 
> org.apache.daffodil.dpath.DFDLLengthFunctionBase.computeValue(DFDLFunctions2.scala:70)
>  at org.apache.daffodil.dpath.FNTwoArgsNodeAndValue.run(FNBases.scala:163)
>  at org.apache.daffodil.dpath.CompiledDPath.run(DPathRuntime.scala:127)
>  at org.apache.daffodil.dpath.CompareOperator.run(DPathRuntime.scala:230)
>  at org.apache.daffodil.dpath.CompiledDPath.run(DPathRuntime.scala:127)
>  at org.apache.daffodil.dpath.IF.run(DPathRuntime.scala:193)
>  at org.apache.daffodil.dpath.CompiledDPath.run(DPathRuntime.scala:127)
>  at 
> org.apache.daffodil.dpath.CompiledDPath.runExpressionForConstant(DPathRuntime.scala:81)
>  at 
> org.apache.daffodil.dpath.DFDLPathExpressionParser.compile(DFDLExpressionParser.scala:55)
>  at 
> org.apache.daffodil.dsom.ExpressionCompiler.compileExpression1(CompiledExpression.scala:208)
>  at 
> org.apache.daffodil.dsom.ExpressionCompiler.compileExpression(CompiledExpression.scala:50)
>  at 
> org.apache.daffodil.dsom.ExpressionCompiler.compileProperty(CompiledExpression.scala:73)
>  at 
> org.apache.daffodil.dsom.ElementRuntimeValuedPropertiesMixin.$anonfun$occursCountExpr$1(RuntimePropertyMixins.scala:453)
>  at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
>  at 
> org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
>  at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
>  at 
> org.apache.daffodil.dsom.ElementRuntimeValuedPropertiesMixin.org$apache$daffodil$dsom$ElementRuntimeValuedPropertiesMixin$$occursCountExpr(RuntimePropertyMixins.scala:450)
>  at 
> org.apache.daffodil.dsom.ElementRuntimeValuedPropertiesMixin.org$apache$daffodil$dsom$ElementRuntimeValuedPropertiesMixin$$occursCountExpr$(RuntimePropertyMixins.scala:450)
>  at 
> org.apache.daffodil.dsom.LocalElementDecl.org$apache$daffodil$dsom$ElementRuntimeValuedPropertiesMixin$$occursCountExpr$lzycompute(LocalElementDecl.scala:22)
>  at 
> org.apache.daffodil.dsom.LocalElementDecl.org$apache$daffodil$dsom$ElementRuntimeValuedPropertiesMixin$$occursCountExpr(LocalElementDecl.scala:22)
>  at 
> org.apache.daffodil.dsom.ElementRuntimeValuedPropertiesMixin.occursCountEv(RuntimePropertyMixins.scala:457)
>  

[GitHub] [incubator-daffodil] stevedlawrence merged pull request #193: Ensure content/valueLength expressions are not considered constants

[GitBox]

stevedlawrence merged pull request #193: Ensure content/valueLength expressions 
are not considered constants
URL: https://github.com/apache/incubator-daffodil/pull/193
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services