olabusayoT commented on a change in pull request #453:
URL: https://github.com/apache/incubator-daffodil/pull/453#discussion_r520067783
##########
File path:
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/DaffodilUnparseContentHandler.scala
##########
@@ -70,10 +74,18 @@ class DaffodilUnparseContentHandler(
extends DFDL.DaffodilUnparseContentHandler {
private lazy val inputter = new SAXInfosetInputter(this, dp, output)
private var unparseResult: DFDL.UnparseResult = _
- private lazy val infosetEvent: DFDL.SAXInfosetEvent = new
DFDL.SAXInfosetEvent
private lazy val characterData = new StringBuilder
private var prefixMapping: NamespaceBinding = _
private lazy val prefixMappingTrackingStack = new MStackOf[NamespaceBinding]
+
+ private lazy val tunablesBatchSize =
dp.getTunables().saxUnparseEventBatchSize
+ private lazy val SAX_UNPARSE_EVENT_BATCH_SIZE = tunablesBatchSize + 1
Review comment:
So the tunablesBatchSixe + 1 is actually an implementation detail.
Under the hood, we always have an extra buffer in the array that we use for
the hasNext call. For each element, we need to know if it has a viable next, if
it doesn't, it triggers the context switch.
So for example, if the user provides 1 as the batchSize, under the hood
we'll have [event1, event2] batched.
- Daffodil unparse will call hasNext and getEventType for the initialization
call, and that hasNext will check if currentIndex (0) + 1 is non-empty. Then it
call getEventType for the event at currentIndex
- Subsequent calls will be next(), ...some processing of the current event
..., hasNext()
For out scenario, next() will update the currentIndex to 1, and event2 will
be processed, then hasNext will check if there is a viable index 2, as there is
not, it will perform the context switch
Without us having the extra buffer, things would happen like this:
user provides 1 as the batchSize, under the hood we'll have [event1] batched.
- Daffodil unparse will call hasNext and getEventType for the initialization
call, and that hasNext will check if currentIndex (0) + 1 is non-empty. As
there is no index 1. It will context switch to get a new batched event, which
would overwrite event1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
