stevedlawrence commented on a change in pull request #422: URL: https://github.com/apache/incubator-daffodil/pull/422#discussion_r498472789
########## File path: daffodil-core/src/main/scala/org/apache/daffodil/runtime2/Runtime2DataProcessor.scala ########## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.daffodil.runtime2 + +import java.io.File +import java.io.InputStream +import java.io.OutputStream + +import org.apache.daffodil.api.DFDL +import org.apache.daffodil.api.DaffodilTunables +import org.apache.daffodil.api.DataLocation +import org.apache.daffodil.api.ValidationMode +import org.apache.daffodil.externalvars.Binding +import org.apache.daffodil.processors.Failure +import org.apache.daffodil.processors.ProcessorResult +import org.apache.daffodil.processors.Success +import org.apache.daffodil.processors.VariableMap +import org.apache.daffodil.processors.WithDiagnosticsImpl +import org.apache.daffodil.processors.parsers.ParseError +import org.apache.daffodil.processors.unparsers.UnparseError +import org.apache.daffodil.util.Maybe +import org.apache.daffodil.util.Maybe.Nope +import os.Pipe + +/** + * Effectively a scala proxy object that does its work via the underlying C-code. + * Will need to consider how to use features of underlying C-code to get infoset, + * walk infoset, generate XML for use by TDML tests. + */ +class Runtime2DataProcessor(executableFile: os.Path) extends DFDL.DataProcessorBase { + /** + * Returns a data processor with all the same state, but the validation mode changed to that of the argument. + * + * Note that the default validation mode is "off", that is, no validation is performed. + */ + override def withValidationMode(mode: ValidationMode.Type): DFDL.DataProcessor = ??? + + override def withTunable(name: String, value: String): DFDL.DataProcessor = ??? + + override def withTunables(tunables: Map[String, String]): DFDL.DataProcessor = ??? + + override def withExternalVariables(extVars: Map[String, String]): DFDL.DataProcessor = ??? + + override def withExternalVariables(extVars: File): DFDL.DataProcessor = ??? + + override def withExternalVariables(extVars: Seq[Binding]): DFDL.DataProcessor = ??? + + override def validationMode: ValidationMode.Type = ??? + + override def getTunables(): DaffodilTunables = ??? + + override def save(output: DFDL.Output): Unit = ??? + + override def variableMap: VariableMap = ??? + + override def setValidationMode(mode: ValidationMode.Type): Unit = ??? + + override def setExternalVariables(extVars: Map[String, String]): Unit = ??? + + override def setExternalVariables(extVars: File): Unit = ??? + + override def setExternalVariables(extVars: File, tunable: DaffodilTunables): Unit = ??? + + override def setExternalVariables(extVars: Seq[Binding]): Unit = ??? + + override def setTunable(tunable: String, value: String): Unit = ??? + + override def setTunables(tunables: Map[String, String]): Unit = ??? + + /** + * Returns an object which contains the result, and/or diagnostic information. + */ + def parse(input: InputStream): ParseResult = { Review comment: Regarding the TDML requirement, I think this can all be handled via the TDMLDFDLProcessor API. Perhaps different backends will also want different TDMLDFDLlProcessor implementations. The runtime2 implementation would generate and compile code, execute the binary with the appropriate command line arguments based on the test configuration, capture the output, and return it to the TDML Runner to check for correctness. We already have support for tests to define which implementation they work for and the ability to switch different implementations. So it's just a matter of defining this new implementation. So there is certainly a concept of a DFDLProcessor specific to runtime2, but it doesn't necessarily need to match the DataProcessor API that runtime1 uses. It really just needs to match the TDMLDFDLProcessor API. Agreed on the second requirment. Generating a shared object with some standard API that users can easily call ultimately seems like best approach. Generating another CLI implemented in C is easy for testing, but utlimately likely not how people will want to use it in the real world. I could imagine an implementation of the TDMLDFDLProcessor could use JNI to dynamically load these generated libraries and hook into the right function calls to parse/unparse. Regarding the save-parser, I'm wonder if options really become part of the runtime. Different runtimes can determine which options they expect and what actions to take. For example, the --debug/--trace options might not make sense with runtime2. Debugging the generated C code likely just means someone should compile the code and execute it with GDB, for example. And the save-parser command for the runtime2 likely wants additional parameters specific to it, like c compiler options, or output diretory for header files. The runtimes could very well require very different parameters. I'm not sure the following is a good idea, but maybe the current CLI wants to become something like this: ``` usage: daffodil <runtime> <runtime subcommands/options> daffodil runtime1 parse --schema foo.dfdl.xsd ... daffodil runtime2 --schema foo.dfdl.sd -copts=-Wfoo --incldir include/ ``` Note that different runtimes don't necessarily need to have subcommands. Runtime1 would likely have many of the existing subcommands. Anoher option would be to keep the root level parse/unparse/save-parser options, and just have a --runtime option, and depending on which runtime option you select different options will become available. That might be a bit easier and more familiar, but the usage likely gets more complicated since availability of parameters changes depending on the runtime. I'm not sure how well scallop can handle something like that. Another option to the save-parser thing To be even more pluggable, it would also be interesting if the Daffodil Scala CLI could ask the different backends for their CLI options. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
