tuxji commented on a change in pull request #681:
URL: https://github.com/apache/daffodil/pull/681#discussion_r756321225
##########
File path: build.sbt
##########
@@ -343,14 +343,38 @@ lazy val unidocSettings = Seq(
lazy val genExamplesSettings = Seq(
Compile / genExamples := {
- val cp = (runtime2 / Runtime / dependencyClasspath).value
- val forkOpts = ForkOptions().withBootJars(cp.files.toVector)
- val mainClass = "org.apache.daffodil.runtime2.CodeGenerator"
- val args = Seq(mainClass)
- val ret = Fork.java(forkOpts, args)
- val stream = streams.value
- if (ret != 0) {
- stream.log.error(s"Failed to generate examples")
+ val cp = (runtime2 / Test / dependencyClasspath).value
+ val inSrc = (runtime2 / Compile / sources).value
+ val inRSrc = (runtime2 / Compile / resources).value
+ val inTSrc = (runtime2 / Test / resources).value
+ val stream = (runtime2 / streams).value
+ val filesToWatch = (inSrc ++ inRSrc ++ inTSrc).toSet
+ val cachedFun = FileFunction.cached(stream.cacheDirectory / "genExamples")
{ _ =>
+ val out = new java.io.ByteArrayOutputStream()
+ val forkOpts = ForkOptions()
+ .withOutputStrategy(Some(CustomOutput(out)))
+ .withBootJars(cp.files.toVector)
+ val mainClass = "org.apache.daffodil.runtime2.CodeGenerator"
+ val args = Seq(mainClass)
+ val ret = Fork.java(forkOpts, args)
+ if (ret != 0) {
+ stream.log.error(s"failed to generate example files")
+ }
+ val bis = new java.io.ByteArrayInputStream(out.toByteArray)
+ val isr = new java.io.InputStreamReader(bis)
+ val br = new java.io.BufferedReader(isr)
+ val iterator = Iterator.continually(br.readLine()).takeWhile(_ !=
null).filterNot(_.startsWith("WARN"))
+ val files = iterator.map { f =>
+ new File(f)
+ }.toSet
+ stream.log.info(s"generated ${files.size} runtime2 example files")
Review comment:
I'll add $outDir to the message, but it needs to be something like `val
outDir = (runtime2 / Test / "c" / "examples").value` (which doesn't work) or
`val outDir = (runtime2 / Test).value / "c" / "examples"` (which doesn't work
either). How do I define outDir correctly?
I'll pass outDir as a parameter to CodeGenerator.main (I don't want to have
to change the examples directory's location in both build.sbt and
CodeGenerator.scala), but I'll miss being able to run CodeGenerator.main from
the IDE using whatever default runtime configuration it creates without having
to edit the runtime configuration to pass the directory. Currently only
Rat.scala and CodeGenerator.scala had to know the examples directory's
location; now it'll be build.sbt and Rat.scala.
##########
File path: .github/workflows/main.yml
##########
@@ -149,6 +149,9 @@ jobs:
- name: Run Integration Tests
run: $SBT coverage IntegrationTest/test
+ - name: Run Modified Example Files Check
+ run: git diff --color --exit-code
Review comment:
The check has passed in the last couple of CI jobs even though it's
running over the entire checkout, so there's no problem right now. And the
check would alert us if a change introduced such a problem in the future, so I
think that's a good thing. Also, if we specify a directory, then that's
another file which we have to remember to update if we move the examples again.
##########
File path:
daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/CodeGenerator.scala
##########
@@ -198,3 +200,60 @@ class CodeGenerator(root: Root) extends DFDL.CodeGenerator
{
override def getDiagnostics: Seq[Diagnostic] = diagnostics
override def isError: Boolean = errorStatus
}
+
+/** Runs from "sbt compile" to keep all example generated code files up to
date */
+object CodeGenerator {
+ // Update one set of example generated code files from an example schema
+ private def updateGeneratedCodeExample(schemaFile: os.Path, rootName:
Option[String],
+ exampleCodeHeader: os.Path,
exampleCodeFile: os.Path): Unit = {
+ // Generate code from the example schema file
+ val pf = Compiler().compileFile(schemaFile.toIO, rootName)
+ assert(!pf.isError, pf.getDiagnostics.map(_.getMessage()).mkString("\n"))
+ val cg = pf.forLanguage("c")
+ val rootNS =
QName.refQNameFromExtendedSyntax(rootName.getOrElse("")).toOption
+ val tempDir = os.temp.dir(dir = null, prefix = "daffodil-runtime2-")
+ val codeDir = cg.generateCode(rootNS, tempDir.toString)
+ assert(!cg.isError, cg.getDiagnostics.map(_.getMessage()).mkString("\n"))
+
+ // Replace the example generated files with the newly generated files
+ val generatedCodeHeader = codeDir/"libruntime"/"generated_code.h"
+ val generatedCodeFile = codeDir/"libruntime"/"generated_code.c"
+ os.copy(generatedCodeHeader, exampleCodeHeader, replaceExisting = true,
createFolders = true)
+ os.copy(generatedCodeFile, exampleCodeFile, replaceExisting = true,
createFolders = true)
+
+ // Print the example generated files' names so "sbt 'show genExamples'"
can list them
+ System.out.println(exampleCodeHeader)
+ System.out.println(exampleCodeFile)
+
+ // tempDir should be removed automatically after main exits; this is just
in case
+ os.remove.all(tempDir)
+ }
+
+ // Make sure "sbt compile" calls this main method
+ def main(args: Array[String]): Unit = {
Review comment:
I don't think it's a big deal that this main method works only in the
daffodil root directory or the daffodil-runtime2 directory (see the if
expression on line 235). The only two times I expect anyone to run this main
method is from "sbt compile" (where the daffodil root directory is the current
directory) or from their IDE as a runtime configuration (where the
daffodil-runtime2 directory is the current directory).
##########
File path: build.sbt
##########
@@ -339,3 +339,40 @@ lazy val unidocSettings = Seq(
}
},
)
+
+lazy val genExamplesSettings = Seq(
+ Compile / genExamples := {
+ val cp = (runtime2 / Test / dependencyClasspath).value
+ val inSrc = (runtime2 / Compile / sources).value
+ val inRSrc = (runtime2 / Test / resources).value
+ val stream = (runtime2 / streams).value
+ val filesToWatch = (inSrc ++ inRSrc).toSet
Review comment:
True, we might want to watch Scala files in daffodil-core and
daffodil-runtime1 too although I think the likelihood of a dsom or grammar
change making it to the PR check is below 10%, not only because it would have
to change generated_code.[ch] without breaking any test, also because a
contributor would have to submit a PR without having done a clean build ("sbt
clean") at some point. Their PR will fail the "git diff --exit-code" check, so
the change still won't slip past the PR check even then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
