[jira] [Reopened] (DAFFODIL-1884) Regression in bitOrder changing
[ https://issues.apache.org/jira/browse/DAFFODIL-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reopened DAFFODIL-1884: > Regression in bitOrder changing > --- > > Key: DAFFODIL-1884 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1884 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.1.0 >Reporter: Steve Lawrence >Assignee: Dave Thompson >Priority: Major > Fix For: 2.1.0 > > > test_bitOrderOVC1 in > ./daffodil-test/src/test/scala-new/edu/illinois/ncsa/daffodil/section00/general/TestUnparserGeneral2.scala > currently fails. It appears to be a regression with recent changes to how > bitOrder works, but was not discovered because the test is in scala-new. This > test should be fixed and moved into the scala directory since the scala-new > directory is no longer used and ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (DAFFODIL-1884) Regression in bitOrder changing
[ https://issues.apache.org/jira/browse/DAFFODIL-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle closed DAFFODIL-1884. -- Resolution: Fixed Assignee: Dave Thompson (was: Michael Beckerle) > Regression in bitOrder changing > --- > > Key: DAFFODIL-1884 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1884 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.1.0 >Reporter: Steve Lawrence >Assignee: Dave Thompson >Priority: Major > Fix For: 2.1.0 > > > test_bitOrderOVC1 in > ./daffodil-test/src/test/scala-new/edu/illinois/ncsa/daffodil/section00/general/TestUnparserGeneral2.scala > currently fails. It appears to be a regression with recent changes to how > bitOrder works, but was not discovered because the test is in scala-new. This > test should be fixed and moved into the scala directory since the scala-new > directory is no longer used and ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1884) Regression in bitOrder changing
[ https://issues.apache.org/jira/browse/DAFFODIL-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356272#comment-16356272 ] Michael Beckerle commented on DAFFODIL-1884: Fixed in commit 945a1c1ab363485003ebf14fef3463055d4fab37 > Regression in bitOrder changing > --- > > Key: DAFFODIL-1884 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1884 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.1.0 >Reporter: Steve Lawrence >Assignee: Michael Beckerle >Priority: Major > Fix For: 2.1.0 > > > test_bitOrderOVC1 in > ./daffodil-test/src/test/scala-new/edu/illinois/ncsa/daffodil/section00/general/TestUnparserGeneral2.scala > currently fails. It appears to be a regression with recent changes to how > bitOrder works, but was not discovered because the test is in scala-new. This > test should be fixed and moved into the scala directory since the scala-new > directory is no longer used and ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (DAFFODIL-1884) Regression in bitOrder changing
[ https://issues.apache.org/jira/browse/DAFFODIL-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle resolved DAFFODIL-1884. Resolution: Fixed > Regression in bitOrder changing > --- > > Key: DAFFODIL-1884 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1884 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.1.0 >Reporter: Steve Lawrence >Assignee: Dave Thompson >Priority: Major > Fix For: 2.1.0 > > > test_bitOrderOVC1 in > ./daffodil-test/src/test/scala-new/edu/illinois/ncsa/daffodil/section00/general/TestUnparserGeneral2.scala > currently fails. It appears to be a regression with recent changes to how > bitOrder works, but was not discovered because the test is in scala-new. This > test should be fixed and moved into the scala directory since the scala-new > directory is no longer used and ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] mbeckerle closed pull request #34: Daffodil 1884 bit order
mbeckerle closed pull request #34: Daffodil 1884 bit order
URL: https://github.com/apache/incubator-daffodil/pull/34
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
index 44784cbf1..33f9c3eee 100644
---
a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
+++
b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
@@ -247,15 +247,6 @@ trait DataOutputStreamImplMixin extends
DataStreamCommonState
private var fragmentLastByteLimit_ : Int = 0
def fragmentLastByteLimit = fragmentLastByteLimit_
- private var maybeFragmentBitOrder: Maybe[BitOrder] = Nope
- def fragmentBitOrder = {
-Assert.usage(maybeFragmentBitOrder.isDefined)
-this.maybeFragmentBitOrder.value
- }
- def setFragmentBitOrder(bo: BitOrder) {
-maybeFragmentBitOrder = One(bo)
- }
-
def setFragmentLastByte(newFragmentByte: Int, nBitsInUse: Int) {
Assert.usage(nBitsInUse >= 0 && nBitsInUse <= 7)
Assert.usage(newFragmentByte >= 0 && newFragmentByte <= 255) // no bits
above first byte are in use.
diff --git
a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
index 7a9cfd27d..9ebbb51d0 100644
---
a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
+++
b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
@@ -179,7 +179,7 @@ final class DirectOrBufferedDataOutputStream private[io]
(var splitFrom: DirectO
*/
private var _javaOutputStream: java.io.OutputStream = bufferingJOS
- private[io] final def isBuffering: Boolean = {
+ final def isBuffering: Boolean = {
val res = getJavaOutputStream() _eq_ bufferingJOS
res
}
diff --git
a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
index d46485215..e69f8b201 100644
---
a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
+++
b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
@@ -71,9 +71,6 @@ import
edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import edu.illinois.ncsa.daffodil.processors.charset.BitsCharsetDecoder
import edu.illinois.ncsa.daffodil.processors.charset.BitsCharsetEncoder
import edu.illinois.ncsa.daffodil.processors.unparsers.UState
-import edu.illinois.ncsa.daffodil.processors.parsers.ParserBitOrderChecks
-import edu.illinois.ncsa.daffodil.processors.parsers.PState
-import edu.illinois.ncsa.daffodil.processors.unparsers.UnparserBitOrderChecks
/**
* Trait mixed into the PState.Mark object class and the ParseOrUnparseState
@@ -200,6 +197,23 @@ abstract class ParseOrUnparseState protected (
private def runtimeData = processor.context
private def termRuntimeData = runtimeData.asInstanceOf[TermRuntimeData]
+ /**
+ * Checks if the bit order change is legal.
+ *
+ * For parsing we know the bitPos, so we can determine if we're at a byte
boundary.
+ *
+ * For unparsing we may not know the absolute bitPos, so we cannot
necessarily
+ * determine if the boundary is legal or not.
+ *
+ * If we know the absoluteBitPos we do the check (as for parsing).
+ *
+ * If we do not know the absoluteBitPos, then in that case, we split the
+ * DataOutputStream into original and buffered. The "check" then occurs
+ * when these DataOutputStreams are collapsed back together.
+ *
+ * So this "check" call, can have an important side effect when unparsing
that
+ * queues up the check to be done in the future.
+ */
protected def checkBitOrder(): Unit
/**
@@ -306,7 +320,7 @@ abstract class ParseOrUnparseState protected (
if (this.processor.isPrimitive)
if (decoderCacheEntry_.encodingMandatoryAlignmentInBitsArg != 1)
if (this.bitPos1b % 8 != 1)
-ParserBitOrderChecks.checkParseBitOrder(this.asInstanceOf[PState])
+checkBitOrder()
}
decoderCacheEntry_
}
@@ -318,7 +332,7 @@ abstract class ParseOrUnparseState protected (
if (this.processor.isPrimitive)
if (encoderCacheEntry_.encodingMandatoryAlignmentInBitsArg != 1)
if (this.bitPos1b % 8 != 1)
-
UnparserBitOrderChecks.checkUnparseBitOrder(this.asInstanceOf[UState])
+
[GitHub] mbeckerle commented on a change in pull request #34: Daffodil 1884 bit order
mbeckerle commented on a change in pull request #34: Daffodil 1884 bit order
URL: https://github.com/apache/incubator-daffodil/pull/34#discussion_r166789367
##
File path:
daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/Unparser.scala
##
@@ -64,6 +64,10 @@ sealed trait Unparser
if (ustate.dataProc.isDefined) ustate.dataProc.get.before(ustate, this)
try {
unparse(ustate)
+ this.context match {
+case trd: TermRuntimeData =>
ustate.dataOutputStream.setPriorBitOrder(ustate.bitOrder)
+case _ => //ok
+ }
Review comment:
Interesting that I can simplify this and just call ustate.bitOrder here. The
setPriorBitOrder is a red-herring.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mbeckerle commented on a change in pull request #34: Daffodil 1884 bit order
mbeckerle commented on a change in pull request #34: Daffodil 1884 bit order
URL: https://github.com/apache/incubator-daffodil/pull/34#discussion_r166787539
##
File path:
daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/Unparser.scala
##
@@ -64,6 +64,10 @@ sealed trait Unparser
if (ustate.dataProc.isDefined) ustate.dataProc.get.before(ustate, this)
try {
unparse(ustate)
+ this.context match {
+case trd: TermRuntimeData =>
ustate.dataOutputStream.setPriorBitOrder(ustate.bitOrder)
+case _ => //ok
+ }
Review comment:
They are different. Parsing proceeds in order (ignoring backtracking) so
every primitive unparser that could leave one at a frag byte must get its bit
order, and it is checked, and checking sets the prior if it is changing.
We did have code at one time like this match-case in unparser in the parser
code, but I convinced myself by the above argument that we don't need it.
Unparsing some unparsers are suspendable, so there are a bunch of cases
here. Clearly calling this setter here is overkilling the situation. In theory
some places in the unparser code dealing with splitting/suspending are missing
proper management of the priorBitOrder. Finding those has been problematic. So
this is a temporary fix, until I can rationalize the setting of priorBitOrder
fully in the unparser.
I will insert comments here accordingly explaining that this is a fix, but
inefficient, and should be unnecessary.
No question that a design note is needed here explaining the invariants
here, and how it is supposed to work, all of which would be nice to capture
when writing the code the first time, but somehow never happens.
I am adding more comments to the code here and in other places explaining
what I can about how this works, and hopefully I'll find the issue as a part of
that process. If not, well at least all the tests we have are passing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (DAFFODIL-1890) Update user related links from the wiki to daffodil.apache.org
[ https://issues.apache.org/jira/browse/DAFFODIL-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Lawrence resolved DAFFODIL-1890. -- Resolution: Fixed Assignee: Dave Thompson (was: Steve Lawrence) Fixed in cdb5ba23d26a4455f1f05e50260ac36c294666ab > Update user related links from the wiki to daffodil.apache.org > -- > > Key: DAFFODIL-1890 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1890 > Project: Daffodil > Issue Type: Bug >Reporter: Steve Lawrence >Assignee: Dave Thompson >Priority: Major > Fix For: 2.1.0 > > > User related documentation has been moved from the wiki to > daffodil.apache.org. Those pages should be removed from the wiki, so links > should be updated to reference the daffodil.apache.org pages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] stevedlawrence commented on a change in pull request #34: Daffodil 1884 bit order
stevedlawrence commented on a change in pull request #34: Daffodil 1884 bit
order
URL: https://github.com/apache/incubator-daffodil/pull/34#discussion_r166741490
##
File path:
daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/Unparser.scala
##
@@ -64,6 +64,10 @@ sealed trait Unparser
if (ustate.dataProc.isDefined) ustate.dataProc.get.before(ustate, this)
try {
unparse(ustate)
+ this.context match {
+case trd: TermRuntimeData =>
ustate.dataOutputStream.setPriorBitOrder(ustate.bitOrder)
+case _ => //ok
+ }
Review comment:
As I recall, we had something similar ``this.context match {`` for the parse
side of things, but I don't see it now. Maybe I'm misremembering or maybe it
was replaced after a code review? Regardless, this doesn't exist on the parse
side. Does parsing handle this differently or is this an inherent difference in
parse vs. unparse so it is required here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (DAFFODIL-1884) Regression in bitOrder changing
[ https://issues.apache.org/jira/browse/DAFFODIL-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1884: --- Affects Version/s: 2.1.0 Component/s: Back End > Regression in bitOrder changing > --- > > Key: DAFFODIL-1884 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1884 > Project: Daffodil > Issue Type: Bug > Components: Back End >Affects Versions: 2.1.0 >Reporter: Steve Lawrence >Assignee: Michael Beckerle >Priority: Major > Fix For: 2.1.0 > > > test_bitOrderOVC1 in > ./daffodil-test/src/test/scala-new/edu/illinois/ncsa/daffodil/section00/general/TestUnparserGeneral2.scala > currently fails. It appears to be a regression with recent changes to how > bitOrder works, but was not discovered because the test is in scala-new. This > test should be fixed and moved into the scala directory since the scala-new > directory is no longer used and ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1884) Regression in bitOrder changing
[ https://issues.apache.org/jira/browse/DAFFODIL-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355973#comment-16355973 ] Michael Beckerle commented on DAFFODIL-1884: Fix is in this PR https://github.com/apache/incubator-daffodil/pull/34 > Regression in bitOrder changing > --- > > Key: DAFFODIL-1884 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1884 > Project: Daffodil > Issue Type: Bug >Reporter: Steve Lawrence >Assignee: Michael Beckerle >Priority: Major > Fix For: 2.1.0 > > > test_bitOrderOVC1 in > ./daffodil-test/src/test/scala-new/edu/illinois/ncsa/daffodil/section00/general/TestUnparserGeneral2.scala > currently fails. It appears to be a regression with recent changes to how > bitOrder works, but was not discovered because the test is in scala-new. This > test should be fixed and moved into the scala directory since the scala-new > directory is no longer used and ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] mbeckerle commented on a change in pull request #34: Daffodil 1884 bit order
mbeckerle commented on a change in pull request #34: Daffodil 1884 bit order
URL: https://github.com/apache/incubator-daffodil/pull/34#discussion_r166736419
##
File path:
daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/PState.scala
##
@@ -284,8 +284,53 @@ final class PState private (
dataInputStream.setDebugging(flag)
}
+ /**
+ * Checks for bit order change. If the bit order is changing, checks if we're
+ * on a proper byte boundary.
+ */
final override protected def checkBitOrder(): Unit = {
-ParserBitOrderChecks.checkParseBitOrder(this)
Review comment:
All I did here is get rid of the objects ParserBitOrderChecks and
UnparserBitOrderChecks and made those methods of PState and UState respectively.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (DAFFODIL-1894) NoSuchElementException when getting namedTypes in union restrictions
[
https://issues.apache.org/jira/browse/DAFFODIL-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Lawrence updated DAFFODIL-1894:
-
Affects Version/s: 2.1.0
> NoSuchElementException when getting namedTypes in union restrictions
>
>
> Key: DAFFODIL-1894
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1894
> Project: Daffodil
> Issue Type: Bug
> Components: Front End
>Affects Versions: 2.1.0
>Reporter: Steve Lawrence
>Priority: Major
> Fix For: 2.2.0
>
>
> Accidentally trying to compile the iCalendar.xsd schema (note, not the
> iCalendar.dfdl.xsd schema) results in the following exception. Note that this
> schema does not have any DFDL properties, so its a little surprising
> compilation got so far. Unclear if this error is related to not have DFDL
> properties or something else.
> {code:java}
> java.util.NoSuchElementException: None.get
> at scala.None$.get(Option.scala:347)
> at scala.None$.get(Option.scala:345)
> at
> edu.illinois.ncsa.daffodil.dsom.Union$$anonfun$namedTypes$1.apply(RestrictionUnion.scala:229)
> at
> edu.illinois.ncsa.daffodil.dsom.Union$$anonfun$namedTypes$1.apply(RestrictionUnion.scala:229)
> at scala.collection.immutable.List.map(List.scala:273)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.namedTypes$lzycompute(RestrictionUnion.scala:228)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.namedTypes(RestrictionUnion.scala:228)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.directMemberTypes$lzycompute(RestrictionUnion.scala:231)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.directMemberTypes(RestrictionUnion.scala:231)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.unionMemberTypes$lzycompute(RestrictionUnion.scala:234)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.unionMemberTypes(RestrictionUnion.scala:233)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.primType$lzycompute(RestrictionUnion.scala:196)
> at
> edu.illinois.ncsa.daffodil.dsom.Union.primType(RestrictionUnion.scala:195)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2$$anonfun$apply$1.apply(SimpleTypes.scala:61)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2$$anonfun$apply$1.apply(SimpleTypes.scala:61)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2.apply(SimpleTypes.scala:61)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2.apply(SimpleTypes.scala:61)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$class.primType(SimpleTypes.scala:61)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase.primType(SimpleTypes.scala:135)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$simpleTypeRuntimeData$7.apply(SimpleTypes.scala:193)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$simpleTypeRuntimeData$7.apply(SimpleTypes.scala:193)
> at
> edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.primType$lzycompute(RuntimeData.scala:270)
> at
> edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.primType(RuntimeData.scala:270)
> at
> edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.preSerialization(RuntimeData.scala:286)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply$mcV$sp(SimpleTypes.scala:142)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply(SimpleTypes.scala:142)
> at
> edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply(SimpleTypes.scala:142)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:600)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:598)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:596)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.valueAsAny(OOLAG.scala:594)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply$mcV$sp(OOLAG.scala:302)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:302)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:302)
> at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:75)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$class.checkErrors(OOLAG.scala:302)
> at
> edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$class.isError(OOLAG.scala:361)
> at
>
[jira] [Updated] (DAFFODIL-1894) NoSuchElementException when getting namedTypes in union restrictions
[
https://issues.apache.org/jira/browse/DAFFODIL-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Lawrence updated DAFFODIL-1894:
-
Description:
Accidentally trying to compile the iCalendar.xsd schema (note, not the
iCalendar.dfdl.xsd schema) results in the following exception. Note that this
schema does not have any DFDL properties, so its a little surprising
compilation got so far. Unclear if this error is related to not have DFDL
properties or something else.
{code:java}
java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:347)
at scala.None$.get(Option.scala:345)
at
edu.illinois.ncsa.daffodil.dsom.Union$$anonfun$namedTypes$1.apply(RestrictionUnion.scala:229)
at
edu.illinois.ncsa.daffodil.dsom.Union$$anonfun$namedTypes$1.apply(RestrictionUnion.scala:229)
at scala.collection.immutable.List.map(List.scala:273)
at
edu.illinois.ncsa.daffodil.dsom.Union.namedTypes$lzycompute(RestrictionUnion.scala:228)
at
edu.illinois.ncsa.daffodil.dsom.Union.namedTypes(RestrictionUnion.scala:228)
at
edu.illinois.ncsa.daffodil.dsom.Union.directMemberTypes$lzycompute(RestrictionUnion.scala:231)
at
edu.illinois.ncsa.daffodil.dsom.Union.directMemberTypes(RestrictionUnion.scala:231)
at
edu.illinois.ncsa.daffodil.dsom.Union.unionMemberTypes$lzycompute(RestrictionUnion.scala:234)
at
edu.illinois.ncsa.daffodil.dsom.Union.unionMemberTypes(RestrictionUnion.scala:233)
at
edu.illinois.ncsa.daffodil.dsom.Union.primType$lzycompute(RestrictionUnion.scala:196)
at
edu.illinois.ncsa.daffodil.dsom.Union.primType(RestrictionUnion.scala:195)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2$$anonfun$apply$1.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2$$anonfun$apply$1.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$class.primType(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase.primType(SimpleTypes.scala:135)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$simpleTypeRuntimeData$7.apply(SimpleTypes.scala:193)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$simpleTypeRuntimeData$7.apply(SimpleTypes.scala:193)
at
edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.primType$lzycompute(RuntimeData.scala:270)
at
edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.primType(RuntimeData.scala:270)
at
edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.preSerialization(RuntimeData.scala:286)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply$mcV$sp(SimpleTypes.scala:142)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply(SimpleTypes.scala:142)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply(SimpleTypes.scala:142)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:600)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:598)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:596)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.valueAsAny(OOLAG.scala:594)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply$mcV$sp(OOLAG.scala:302)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:302)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:302)
at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:75)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$class.checkErrors(OOLAG.scala:302)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$class.isError(OOLAG.scala:361)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory.edu$illinois$ncsa$daffodil$compiler$ProcessorFactory$$super$isError(Compiler.scala:151)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply$mcZ$sp(Compiler.scala:151)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply(Compiler.scala:142)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply(Compiler.scala:142)
at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:75)
at
[jira] [Created] (DAFFODIL-1894) NoSuchElementException when getting namedTypes in union restrictions
Steve Lawrence created DAFFODIL-1894:
Summary: NoSuchElementException when getting namedTypes in union
restrictions
Key: DAFFODIL-1894
URL: https://issues.apache.org/jira/browse/DAFFODIL-1894
Project: Daffodil
Issue Type: Bug
Reporter: Steve Lawrence
Accidentally trying to compile the iCalendar.xsd schema (note, not the
iCalendar.dfdl.xsd schema) results in the following exception:
{code:java}
java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:347)
at scala.None$.get(Option.scala:345)
at
edu.illinois.ncsa.daffodil.dsom.Union$$anonfun$namedTypes$1.apply(RestrictionUnion.scala:229)
at
edu.illinois.ncsa.daffodil.dsom.Union$$anonfun$namedTypes$1.apply(RestrictionUnion.scala:229)
at scala.collection.immutable.List.map(List.scala:273)
at
edu.illinois.ncsa.daffodil.dsom.Union.namedTypes$lzycompute(RestrictionUnion.scala:228)
at
edu.illinois.ncsa.daffodil.dsom.Union.namedTypes(RestrictionUnion.scala:228)
at
edu.illinois.ncsa.daffodil.dsom.Union.directMemberTypes$lzycompute(RestrictionUnion.scala:231)
at
edu.illinois.ncsa.daffodil.dsom.Union.directMemberTypes(RestrictionUnion.scala:231)
at
edu.illinois.ncsa.daffodil.dsom.Union.unionMemberTypes$lzycompute(RestrictionUnion.scala:234)
at
edu.illinois.ncsa.daffodil.dsom.Union.unionMemberTypes(RestrictionUnion.scala:233)
at
edu.illinois.ncsa.daffodil.dsom.Union.primType$lzycompute(RestrictionUnion.scala:196)
at
edu.illinois.ncsa.daffodil.dsom.Union.primType(RestrictionUnion.scala:195)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2$$anonfun$apply$1.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2$$anonfun$apply$1.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$$anonfun$primType$2.apply(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeBase$class.primType(SimpleTypes.scala:61)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase.primType(SimpleTypes.scala:135)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$simpleTypeRuntimeData$7.apply(SimpleTypes.scala:193)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$simpleTypeRuntimeData$7.apply(SimpleTypes.scala:193)
at
edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.primType$lzycompute(RuntimeData.scala:270)
at
edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.primType(RuntimeData.scala:270)
at
edu.illinois.ncsa.daffodil.processors.SimpleTypeRuntimeData.preSerialization(RuntimeData.scala:286)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply$mcV$sp(SimpleTypes.scala:142)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply(SimpleTypes.scala:142)
at
edu.illinois.ncsa.daffodil.dsom.SimpleTypeDefBase$$anonfun$1.apply(SimpleTypes.scala:142)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:600)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:598)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:596)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.valueAsAny(OOLAG.scala:594)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply$mcV$sp(OOLAG.scala:302)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:302)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:302)
at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:75)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$class.checkErrors(OOLAG.scala:302)
at
edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$class.isError(OOLAG.scala:361)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory.edu$illinois$ncsa$daffodil$compiler$ProcessorFactory$$super$isError(Compiler.scala:151)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply$mcZ$sp(Compiler.scala:151)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply(Compiler.scala:142)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply(Compiler.scala:142)
at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:75)
at
edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1.apply$mcZ$sp(Compiler.scala:142)
at
[jira] [Closed] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
[ https://issues.apache.org/jira/browse/DAFFODIL-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Thompson closed DAFFODIL-1892. --- Closing in lieu of DAFFODIL-1893. > iCalendar performance to run tdml test is very slow > --- > > Key: DAFFODIL-1892 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 > Project: Daffodil > Issue Type: Bug > Components: DFDL Schemas >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Major > Fix For: 2.1.0 > > > The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit > tests that use TDML runner. > These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. > Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
[ https://issues.apache.org/jira/browse/DAFFODIL-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Lawrence resolved DAFFODIL-1892. -- Resolution: Duplicate Assignee: (was: Michael Beckerle) Fix Version/s: (was: deferred) 2.1.0 The main cause appears to be due to DAFFODIL-1893. The compiled schema takes a while to compile due to it's size/complexity, so not caching it hurts alot. Resolving as a duplicate. Improvements to the schema may be possible (e.g.. simplifications, choice dispatch) that might make iCalendar more performant at compile time or runtime, but those issues are specific to iCalendar and should be tracked on the iCalendar bug tracker. > iCalendar performance to run tdml test is very slow > --- > > Key: DAFFODIL-1892 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 > Project: Daffodil > Issue Type: Bug > Components: DFDL Schemas >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Major > Fix For: 2.1.0 > > > The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit > tests that use TDML runner. > These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. > Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1893) TDML Runner not caching schema compiles - only caches schemas if compileAllTopLevel is true
[ https://issues.apache.org/jira/browse/DAFFODIL-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Lawrence updated DAFFODIL-1893: - Fix Version/s: (was: 2.1.0) 2.2.0 > TDML Runner not caching schema compiles - only caches schemas if > compileAllTopLevel is true > --- > > Key: DAFFODIL-1893 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1893 > Project: Daffodil > Issue Type: Bug > Components: TDML Runner >Affects Versions: 2.0.0 >Reporter: Michael Beckerle >Priority: Major > Fix For: 2.2.0 > > > The TDML runner is supposed to cache compiled schemas so that many tests > using the same DFDL schema will compile it only once. > It currently only does this when compileAllTopLevel is true, but the default > for this is false. > The cache should be modified to take the root element into account - i.e., > compile all top levels need not be true. The cache should hit, avoiding > recompilation, so long as the same root element is being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
[ https://issues.apache.org/jira/browse/DAFFODIL-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1892: --- Priority: Major (was: Blocker) > iCalendar performance to run tdml test is very slow > --- > > Key: DAFFODIL-1892 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 > Project: Daffodil > Issue Type: Bug > Components: DFDL Schemas >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Major > Fix For: deferred > > > The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit > tests that use TDML runner. > These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. > Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
[ https://issues.apache.org/jira/browse/DAFFODIL-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle updated DAFFODIL-1892: --- Fix Version/s: (was: 2.1.0) deferred > iCalendar performance to run tdml test is very slow > --- > > Key: DAFFODIL-1892 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 > Project: Daffodil > Issue Type: Bug > Components: DFDL Schemas >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Major > Fix For: deferred > > > The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit > tests that use TDML runner. > These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. > Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
[ https://issues.apache.org/jira/browse/DAFFODIL-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Beckerle reassigned DAFFODIL-1892: -- Assignee: Michael Beckerle > iCalendar performance to run tdml test is very slow > --- > > Key: DAFFODIL-1892 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 > Project: Daffodil > Issue Type: Bug > Components: DFDL Schemas >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Assignee: Michael Beckerle >Priority: Blocker > Fix For: 2.1.0 > > > The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit > tests that use TDML runner. > These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. > Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DAFFODIL-1891) TDML Runner XML comparisons should ignore xsi:noNamespaceSchemaLocation and xsi:schemaLocation attributes
[ https://issues.apache.org/jira/browse/DAFFODIL-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Lawrence updated DAFFODIL-1891: - Fix Version/s: (was: 2.1.0) deferred > TDML Runner XML comparisons should ignore xsi:noNamespaceSchemaLocation and > xsi:schemaLocation attributes > - > > Key: DAFFODIL-1891 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1891 > Project: Daffodil > Issue Type: Bug > Components: TDML Runner >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Assignee: Steve Lawrence >Priority: Major > Fix For: deferred > > > For the iCalendar schema, the infoset files used for unparser testing, and > for comparison of results for parse testing, they carry these attributes on > the root element: > xsi:noNamespaceSchemaLocation > or > xsi:schemaLocation > The Daffodil output from parse doesn't have these attributes, so comparisons > fail complaining that these attributes do not match. > The TDML runner should tolerate (i.e., ignore) these xsi namespace attributes > when comparing result sets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-1891) TDML Runner XML comparisons should ignore xsi:noNamespaceSchemaLocation and xsi:schemaLocation attributes
[ https://issues.apache.org/jira/browse/DAFFODIL-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Lawrence reassigned DAFFODIL-1891: Assignee: (was: Steve Lawrence) > TDML Runner XML comparisons should ignore xsi:noNamespaceSchemaLocation and > xsi:schemaLocation attributes > - > > Key: DAFFODIL-1891 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1891 > Project: Daffodil > Issue Type: Bug > Components: TDML Runner >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Major > Fix For: deferred > > > For the iCalendar schema, the infoset files used for unparser testing, and > for comparison of results for parse testing, they carry these attributes on > the root element: > xsi:noNamespaceSchemaLocation > or > xsi:schemaLocation > The Daffodil output from parse doesn't have these attributes, so comparisons > fail complaining that these attributes do not match. > The TDML runner should tolerate (i.e., ignore) these xsi namespace attributes > when comparing result sets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DAFFODIL-1893) TDML Runner not caching schema compiles - only caches schemas if compileAllTopLevel is true
Michael Beckerle created DAFFODIL-1893: -- Summary: TDML Runner not caching schema compiles - only caches schemas if compileAllTopLevel is true Key: DAFFODIL-1893 URL: https://issues.apache.org/jira/browse/DAFFODIL-1893 Project: Daffodil Issue Type: Bug Components: TDML Runner Affects Versions: 2.0.0 Reporter: Michael Beckerle Fix For: 2.1.0 The TDML runner is supposed to cache compiled schemas so that many tests using the same DFDL schema will compile it only once. It currently only does this when compileAllTopLevel is true, but the default for this is false. The cache should be modified to take the root element into account - i.e., compile all top levels need not be true. The cache should hit, avoiding recompilation, so long as the same root element is being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
[ https://issues.apache.org/jira/browse/DAFFODIL-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355675#comment-16355675 ] Michael Beckerle commented on DAFFODIL-1892: Note: other 'sbt test' on my machine run just fine. So it's not my current snapshot build of daffodil causing this. > iCalendar performance to run tdml test is very slow > --- > > Key: DAFFODIL-1892 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 > Project: Daffodil > Issue Type: Bug > Components: DFDL Schemas >Affects Versions: 2.1.0 >Reporter: Michael Beckerle >Priority: Blocker > Fix For: 2.1.0 > > > The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit > tests that use TDML runner. > These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. > Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DAFFODIL-1892) iCalendar performance to run tdml test is very slow
Michael Beckerle created DAFFODIL-1892: -- Summary: iCalendar performance to run tdml test is very slow Key: DAFFODIL-1892 URL: https://issues.apache.org/jira/browse/DAFFODIL-1892 Project: Daffodil Issue Type: Bug Components: DFDL Schemas Affects Versions: 2.1.0 Reporter: Michael Beckerle Fix For: 2.1.0 The iCalendar schema (on DI2E.net) has been enhanced to have a couple unit tests that use TDML runner. These tests are tiny, yet on current 2.1.0 snapshot they take minutes to run. Something is wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1878) Parse and unparse files/sec values vary significantly using compiled and saved parsers
[ https://issues.apache.org/jira/browse/DAFFODIL-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355618#comment-16355618 ] Michael Beckerle commented on DAFFODIL-1878: Ok, so multi-thread interactions may be at work here. Are there variations like this in runtime if we run only 1 thread? If not then we have a strong hint. Can we get that data point too? > Parse and unparse files/sec values vary significantly using compiled and > saved parsers > -- > > Key: DAFFODIL-1878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1878 > Project: Daffodil > Issue Type: Bug > Components: General >Affects Versions: 2.1.0 >Reporter: Dave Thompson >Assignee: Steve Lawrence >Priority: Major > Fix For: 2.2.0 > > Attachments: Performance report for 01-19-2018.docx > > > After updating the nightly scripts to use the pre-compiled/saved parsers the > result show that for some tests there are significant performance value > differences between using runtime compiled parsers and the saved parsers. > Attached is the report email. The Previous Val column is from non-saved > parser run. Curr Val column is from previously compiled/saved parsers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1878) Parse and unparse files/sec values vary significantly using compiled and saved parsers
[ https://issues.apache.org/jira/browse/DAFFODIL-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355603#comment-16355603 ] Dave Thompson commented on DAFFODIL-1878: - Additional information: Noticed that there were significant variances in the jpeg_10t_1m_1000 test between nightly runs, even when there was no change to daffodil when using saved parser. Looked at the raw csv data file and found that the average files/sec varied greatly between test iterations (e.g. 30 files/sec to 145 files/sec). Executed the test (ten iterations) numerous times varying the number files processed. Found that as the files processed was lowered the files/sec performance lowered as did the difference between the max and min files/sec lowered but remained significant. Executed with 5 threads and got similar results, lower files/sec and lower but still excessive difference. The difference when executing the test using the runtime compiled parser/schema was usually between 2 – 5 files/sec. > Parse and unparse files/sec values vary significantly using compiled and > saved parsers > -- > > Key: DAFFODIL-1878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1878 > Project: Daffodil > Issue Type: Bug > Components: General >Affects Versions: 2.1.0 >Reporter: Dave Thompson >Assignee: Steve Lawrence >Priority: Major > Fix For: 2.2.0 > > Attachments: Performance report for 01-19-2018.docx > > > After updating the nightly scripts to use the pre-compiled/saved parsers the > result show that for some tests there are significant performance value > differences between using runtime compiled parsers and the saved parsers. > Attached is the report email. The Previous Val column is from non-saved > parser run. Curr Val column is from previously compiled/saved parsers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] stevedlawrence opened a new pull request #33: Update wiki links to daffodil.apache.org
stevedlawrence opened a new pull request #33: Update wiki links to daffodil.apache.org URL: https://github.com/apache/incubator-daffodil/pull/33 User related documentation is moved from the wiki to the daffodil.apache.org website. Update the links to reference those pages. DFDL-1890 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (DAFFODIL-1890) Update user related links from the wiki to daffodil.apache.org
Steve Lawrence created DAFFODIL-1890: Summary: Update user related links from the wiki to daffodil.apache.org Key: DAFFODIL-1890 URL: https://issues.apache.org/jira/browse/DAFFODIL-1890 Project: Daffodil Issue Type: Bug Reporter: Steve Lawrence Assignee: Steve Lawrence Fix For: 2.1.0 User related documentation has been moved from the wiki to daffodil.apache.org. Those pages should be removed from the wiki, so links should be updated to reference the daffodil.apache.org pages. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
