mbeckerle commented on code in PR #1191: URL: https://github.com/apache/daffodil/pull/1191#discussion_r1570917604
########## daffodil-runtime1-layers/src/main/scala/org/apache/daffodil/layers/runtime1/FixedLengthLayer.scala: ########## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.layers.runtime1 + +import java.io.ByteArrayInputStream +import java.io.ByteArrayOutputStream +import java.io.InputStream +import java.io.OutputStream +import java.nio.ByteBuffer + +import org.apache.daffodil.lib.exceptions.Assert +import org.apache.daffodil.runtime1.layers.LayerNotEnoughDataException +import org.apache.daffodil.runtime1.layers.api.Layer + +import org.apache.commons.io.IOUtils + +/** + * Suitable only for small sections of data, not large data streams or whole files. + * See the maxFixedLength value defined herein for the maximum. + * + * The entire fixed length region of the data will be pulled into a byte buffer in memory. + * + * TODO: Someday, enhance to make this streaming. + * + * One DFDL Variable is a parameter + * - fixedLength - an unsignedInt giving the fixed length of this layer. + * This length is enforced on both parsing and unparsing the layer. + * There are no output/result DFDL variables from this layer. + */ +final class FixedLengthLayer + extends Layer("fixedLength", "urn:org.apache.daffodil.layers.fixedLength") { + + private var fixedLength: Int = -1 + + private def maxFixedLength = Short.MaxValue + + /** + * Captures the fixedLength DFDL variable value and saves it. + * + * Also validates whether it is in acceptable range. Because the fixedLength variable + * can be populated from an expression that consumes input data, it's possible for the + * fixedLength to be out of range as a result of speculative parsing down + * an incorrect path. Hence, it is a processing error, not a schema definition error + * if the fixedLength variable value is out of range. + * + * @param fixedLength matches the name of the DFDL variable which will supply this value. + */ + private[layers] def setLayerVariableParameters(fixedLength: Long): Unit = { + this.fixedLength = fixedLength.toInt + Assert.invariant(fixedLength >= 0) // variable is unsignedInt, so this can't be negative + if (fixedLength > maxFixedLength) + processingError( + s"fixedLength value of $fixedLength is above the maximum of $maxFixedLength.", + ) + } + + override def wrapLayerInput(jis: InputStream): InputStream = { + new FixedLengthInputStream(jis) Review Comment: Using the IOUtils library instead of our own - makes sense. I'll consider changing this or add a ticket to do this as a second change set. The second issue is more important. We need some tests to characterize what it means for each place a layer can cause a parse error. E.g., what happens if a parse error occurs on close of the layer stream, what happens if it happens on establishment of the layer stream, what happens if it occurs during access to layer data. The error on close issue is snarly for unparsing, but all processing errors during unparsing are fatal, so we only really case that these errors cannot be masked in that case. For parsing however, we need a failure to establish or to close the stream to cause the parse of the sequence having the layer to fail with a parse error. I will look into a couple more tests to verify this precisely, or document which tests we have now which verify this. Tests will need to know not just that a parse error occurred, but exactly what construct in the DFDL schema got the parse error. Some of these errors must be parse errors of the layered sequence, not of something inside the layered sequence which could be suppressed by a choice (for example). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
