mbeckerle commented on code in PR #1165: URL: https://github.com/apache/daffodil/pull/1165#discussion_r1503371997
########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/api/JLayerLengthKind.java: ########## @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers.api; + +public enum JLayerLengthKind { Review Comment: These properties go away in the "it's all just DFDL variables" formulation of this stuff. ########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/api/Layer.scala: ########## @@ -0,0 +1,275 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers.api + +import java.io.IOException +import java.io.InputStream +import java.io.OutputStream +import java.nio.charset.Charset +import java.util.Optional + +import org.apache.daffodil.runtime1.api.DFDLPrimType +import org.apache.daffodil.runtime1.layers.LayerException + +// TODO: Convert this whole file to Java ?? + +/** + * Descriptor of the DFDL variables used by the layer. + * + * The names and types must match the dfdl:defineVariable definitions used in + * an included/imported DFDL component schema which provides the layer definition for use + * by other schemas. + * + * The first such variable is sometimes distinguished as the variable written to + * as the single unique value of the layer, such as when the layer computes a + * checksum. This is, however, just a convention used by various checksum layers + * and classes/traits that support writing checksum layers. + * + * @param prefix preferred namespace prefix for the namespace of the variables + * @param namespace namespace of the variables + * @param variables list of pairs, each is the name (an NCName, that is without any + * namespace prefix) and type of a variable + */ +final case class LayerVariables( + prefix: String, + namespace: String, + variables: java.util.List[(String, DFDLPrimType)], +) + +/** + * This is the primary API class for writing layers. + * + * All layers are derived from this class, and must have no-args default constructors. + * + * Derived classes will be dynamically loaded by Java's SPI system. + * The names of concrete classes derived from Layer are listed in a resources/M.services file + * so that they can be found and dynamically loaded. + * + * The SPI creates an instance the class of which is used as a factory to create the + * instances actually used by the Daffodil runtime. Compilation of the static information about + * the layer occurs only once and is then shared by all runtime instances. + * + * Instances of derived layer classes can be stateful. They are private to threads, and each time a layer + * is encountered during parse/unparse, an instance is created for that situation. + * + * All the static information about the layer is provided in the arguments. + * + * The rest of the Layer class implements the + * layer decode/encode logic, which is done as part of deriving one's Layer class from the + * Layer base class. + * + * About variables: Layer logic may read and write variables. Variables being read are parameters to + * the layer algorithm. Variables being written are outputs (such as checksums) from the layer algorithm. + * Variables being written must be undefined, since variables in DFDL are single-assignment. + * Variables being read must be defined before being read by the layer, and this is true for both + * parsing and unparsing. When unparsing, variables being read cannot be forward-referencing to parts + * of the DFDL infoset that have not yet been unparsed. + * + * @param layerName the name that will appear in the DFDL schema to identify the layer + * @param supportedLayerLengthKinds list of the layer length kinds the layer supports + * @param supportedLayerLengthUnits list of the layer length units the layer supports + * @param isRequiredLayerEncoding true if the layer is textual and so needs the layerEncoding property + * @param optLayerVariables a LayerVariables structure describing the variables the layer accesses + */ +abstract class Layer( + val layerName: String, + val supportedLayerLengthKinds: java.util.List[JLayerLengthKind], + val supportedLayerLengthUnits: java.util.List[JLayerLengthUnits], + val isRequiredLayerEncoding: Boolean, + val optLayerVariables: Optional[LayerVariables], +) { + + final def name(): String = + layerName // name() method with empty args is required by SPI loader + + /** + * Called exactly once when the schema is compiled to do extra checking that the layer is being used properly. + * The thrown exception becomes a SchemaDefinitionError at schema compile time. + * + * Example checks are: + * - layerEncoding is constant and is a single-byte charset + * - layerLength, if constant, is within a maximum value range Review Comment: Yes, but there is a difference in behavior. The layerLengthKind 'explicit' is only explicit on parse. On unparse it ignores the layerLength. This was to avoid the problem DFDL v1.0 has where the output wants to be perhaps a different length than the input, but to recompute that on output you have to measure the length not of the original element, but an inner element inside it, to avoid the circularity. But I'm inclined to try getting rid of dfdl:layerLengthKind entirely and see how it goes. ########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/LayerException.scala: ########## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers + +abstract class LayerException(msg: String, cause: Throwable) extends Exception(msg, cause) { + def this(msg: String) = this(msg, null) + def this(cause: Throwable) = this(null, cause) +} + +class LayerCompilerException(msg: String, cause: Throwable) extends LayerException(msg, cause) { Review Comment: I will follow this UDF style. ########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/api/Layer.scala: ########## @@ -0,0 +1,275 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers.api + +import java.io.IOException +import java.io.InputStream +import java.io.OutputStream +import java.nio.charset.Charset +import java.util.Optional + +import org.apache.daffodil.runtime1.api.DFDLPrimType +import org.apache.daffodil.runtime1.layers.LayerException + +// TODO: Convert this whole file to Java ?? + +/** + * Descriptor of the DFDL variables used by the layer. + * + * The names and types must match the dfdl:defineVariable definitions used in + * an included/imported DFDL component schema which provides the layer definition for use + * by other schemas. + * + * The first such variable is sometimes distinguished as the variable written to + * as the single unique value of the layer, such as when the layer computes a + * checksum. This is, however, just a convention used by various checksum layers + * and classes/traits that support writing checksum layers. + * + * @param prefix preferred namespace prefix for the namespace of the variables + * @param namespace namespace of the variables + * @param variables list of pairs, each is the name (an NCName, that is without any + * namespace prefix) and type of a variable + */ +final case class LayerVariables( + prefix: String, + namespace: String, + variables: java.util.List[(String, DFDLPrimType)], Review Comment: The string in this case can be one of these "{urn:someNamespace}localName" type strings to incorporate a namespace. ########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/api/Layer.scala: ########## @@ -0,0 +1,275 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers.api + +import java.io.IOException +import java.io.InputStream +import java.io.OutputStream +import java.nio.charset.Charset +import java.util.Optional + +import org.apache.daffodil.runtime1.api.DFDLPrimType +import org.apache.daffodil.runtime1.layers.LayerException + +// TODO: Convert this whole file to Java ?? + +/** + * Descriptor of the DFDL variables used by the layer. + * + * The names and types must match the dfdl:defineVariable definitions used in + * an included/imported DFDL component schema which provides the layer definition for use + * by other schemas. + * + * The first such variable is sometimes distinguished as the variable written to + * as the single unique value of the layer, such as when the layer computes a + * checksum. This is, however, just a convention used by various checksum layers + * and classes/traits that support writing checksum layers. + * + * @param prefix preferred namespace prefix for the namespace of the variables + * @param namespace namespace of the variables + * @param variables list of pairs, each is the name (an NCName, that is without any + * namespace prefix) and type of a variable + */ +final case class LayerVariables( + prefix: String, + namespace: String, + variables: java.util.List[(String, DFDLPrimType)], +) + +/** + * This is the primary API class for writing layers. + * + * All layers are derived from this class, and must have no-args default constructors. + * + * Derived classes will be dynamically loaded by Java's SPI system. + * The names of concrete classes derived from Layer are listed in a resources/M.services file + * so that they can be found and dynamically loaded. + * + * The SPI creates an instance the class of which is used as a factory to create the + * instances actually used by the Daffodil runtime. Compilation of the static information about + * the layer occurs only once and is then shared by all runtime instances. + * + * Instances of derived layer classes can be stateful. They are private to threads, and each time a layer + * is encountered during parse/unparse, an instance is created for that situation. + * + * All the static information about the layer is provided in the arguments. + * + * The rest of the Layer class implements the + * layer decode/encode logic, which is done as part of deriving one's Layer class from the + * Layer base class. + * + * About variables: Layer logic may read and write variables. Variables being read are parameters to + * the layer algorithm. Variables being written are outputs (such as checksums) from the layer algorithm. + * Variables being written must be undefined, since variables in DFDL are single-assignment. + * Variables being read must be defined before being read by the layer, and this is true for both + * parsing and unparsing. When unparsing, variables being read cannot be forward-referencing to parts + * of the DFDL infoset that have not yet been unparsed. + * + * @param layerName the name that will appear in the DFDL schema to identify the layer + * @param supportedLayerLengthKinds list of the layer length kinds the layer supports + * @param supportedLayerLengthUnits list of the layer length units the layer supports + * @param isRequiredLayerEncoding true if the layer is textual and so needs the layerEncoding property + * @param optLayerVariables a LayerVariables structure describing the variables the layer accesses + */ +abstract class Layer( + val layerName: String, + val supportedLayerLengthKinds: java.util.List[JLayerLengthKind], + val supportedLayerLengthUnits: java.util.List[JLayerLengthUnits], + val isRequiredLayerEncoding: Boolean, + val optLayerVariables: Optional[LayerVariables], +) { + + final def name(): String = + layerName // name() method with empty args is required by SPI loader + + /** + * Called exactly once when the schema is compiled to do extra checking that the layer is being used properly. + * The thrown exception becomes a SchemaDefinitionError at schema compile time. + * + * Example checks are: + * - layerEncoding is constant and is a single-byte charset + * - layerLength, if constant, is within a maximum value range + * - layerBoundaryMark string, if constant, is not too long and contains only allowed characters. + * These things can be required to be constant by this check, or it can check their values for legality + * if they happen to be constant. Since these are runtime-valued properties (can be expressions), then if the + * layer allowed that, they must also be checked at runtime. Review Comment: This is what I will try next. More verbose, but that's mostly XSD's fault. Only one thing to learn: variables. ########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/api/Layer.scala: ########## @@ -0,0 +1,275 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers.api + +import java.io.IOException +import java.io.InputStream +import java.io.OutputStream +import java.nio.charset.Charset +import java.util.Optional + +import org.apache.daffodil.runtime1.api.DFDLPrimType +import org.apache.daffodil.runtime1.layers.LayerException + +// TODO: Convert this whole file to Java ?? + +/** + * Descriptor of the DFDL variables used by the layer. + * + * The names and types must match the dfdl:defineVariable definitions used in + * an included/imported DFDL component schema which provides the layer definition for use + * by other schemas. + * + * The first such variable is sometimes distinguished as the variable written to + * as the single unique value of the layer, such as when the layer computes a + * checksum. This is, however, just a convention used by various checksum layers + * and classes/traits that support writing checksum layers. + * + * @param prefix preferred namespace prefix for the namespace of the variables + * @param namespace namespace of the variables + * @param variables list of pairs, each is the name (an NCName, that is without any + * namespace prefix) and type of a variable + */ +final case class LayerVariables( + prefix: String, + namespace: String, + variables: java.util.List[(String, DFDLPrimType)], +) + +/** + * This is the primary API class for writing layers. + * + * All layers are derived from this class, and must have no-args default constructors. + * + * Derived classes will be dynamically loaded by Java's SPI system. + * The names of concrete classes derived from Layer are listed in a resources/M.services file Review Comment: The name in my source tree is M.services. I was wondering how it decides what files to read. The string M.services is not in our code base, so we aren't telling the SPI to look for a file of that name. It must be by convention that this file has "services" in its name or something like that. ########## daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/layers/api/Layer.scala: ########## @@ -0,0 +1,275 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.daffodil.runtime1.layers.api + +import java.io.IOException +import java.io.InputStream +import java.io.OutputStream +import java.nio.charset.Charset +import java.util.Optional + +import org.apache.daffodil.runtime1.api.DFDLPrimType +import org.apache.daffodil.runtime1.layers.LayerException + +// TODO: Convert this whole file to Java ?? + +/** + * Descriptor of the DFDL variables used by the layer. + * + * The names and types must match the dfdl:defineVariable definitions used in + * an included/imported DFDL component schema which provides the layer definition for use + * by other schemas. + * + * The first such variable is sometimes distinguished as the variable written to + * as the single unique value of the layer, such as when the layer computes a + * checksum. This is, however, just a convention used by various checksum layers + * and classes/traits that support writing checksum layers. + * + * @param prefix preferred namespace prefix for the namespace of the variables + * @param namespace namespace of the variables + * @param variables list of pairs, each is the name (an NCName, that is without any + * namespace prefix) and type of a variable + */ +final case class LayerVariables( + prefix: String, + namespace: String, + variables: java.util.List[(String, DFDLPrimType)], +) + +/** + * This is the primary API class for writing layers. + * + * All layers are derived from this class, and must have no-args default constructors. + * + * Derived classes will be dynamically loaded by Java's SPI system. + * The names of concrete classes derived from Layer are listed in a resources/M.services file + * so that they can be found and dynamically loaded. + * + * The SPI creates an instance the class of which is used as a factory to create the + * instances actually used by the Daffodil runtime. Compilation of the static information about + * the layer occurs only once and is then shared by all runtime instances. + * + * Instances of derived layer classes can be stateful. They are private to threads, and each time a layer + * is encountered during parse/unparse, an instance is created for that situation. + * + * All the static information about the layer is provided in the arguments. + * + * The rest of the Layer class implements the + * layer decode/encode logic, which is done as part of deriving one's Layer class from the + * Layer base class. + * + * About variables: Layer logic may read and write variables. Variables being read are parameters to + * the layer algorithm. Variables being written are outputs (such as checksums) from the layer algorithm. + * Variables being written must be undefined, since variables in DFDL are single-assignment. + * Variables being read must be defined before being read by the layer, and this is true for both + * parsing and unparsing. When unparsing, variables being read cannot be forward-referencing to parts + * of the DFDL infoset that have not yet been unparsed. + * + * @param layerName the name that will appear in the DFDL schema to identify the layer + * @param supportedLayerLengthKinds list of the layer length kinds the layer supports + * @param supportedLayerLengthUnits list of the layer length units the layer supports + * @param isRequiredLayerEncoding true if the layer is textual and so needs the layerEncoding property + * @param optLayerVariables a LayerVariables structure describing the variables the layer accesses + */ +abstract class Layer( + val layerName: String, + val supportedLayerLengthKinds: java.util.List[JLayerLengthKind], + val supportedLayerLengthUnits: java.util.List[JLayerLengthUnits], + val isRequiredLayerEncoding: Boolean, Review Comment: I think this can just be a DFDL variable, if a layer needs it then it can define a variable to use. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
