Note that the repo isn't closed. It's a public github repo at: https://github.com/OpenDFDL/examples
But access is restricted and not controlled by Daffodil PPMC or anything ASF related, so it is private in that sense. The main thing we use OpenDFDL is as place to store examples of how to use the Daffodil API, but don't really belong in the Daffodil codebase. I sort of view this as what another person/project might do if they wanted to use the Daffodil API and publish it. This one just happens to be controlled by PPMC members of Daffodil. Using Daffodil doesn't require any of this stuff at all. Though, I'm not against merging some stuff form OpenDFDL into a repo. Do you know how other projects handle this kind of thing? Does it make sense to have a "daffodil-examples" repository, or just directly include it in our code base? The other thing we've been using OpenDFDL is for things that have incompatible licenses. For example, we have a tool that uses the IBM DFDL implementation to run tests in the Daffodil resository, as a way to test cross compatibility. The IBM tool requires agreeing to IBM terms and downloading jars from IBM's website. The terms are pretty restrictive and are certainly category X, so doesn't belong in Daffodil. Another example is this XSLT example. That depends on Saxon-EE, which is also category X. I guess we could include these in the an ASF repo, and just add instructions on how to get the dependencies and build everything. But it feels safer to just not include that stuff at all. None of these are needed to use Daffodil so feels like a separate thing to me. On 7/15/20 12:39 PM, Dave Fisher wrote: > Hi - > > I’m somewhat concerned about this private closed repository being used for > daffodil examples. It really would be more helpful if it was more publicly > accessible and perhaps part of the project’s repositories. > > Please explain. > > Regards, > Dave > >> On Jul 7, 2020, at 6:41 AM, Steve Lawrence <slawre...@apache.org> wrote: >> >> All makes sense. I've created a handful a bugs in the OpenDFDL repo to >> track these issues. >> >> On 7/7/20 8:39 AM, Beckerle, Mike wrote: >>> I suggest adding an Issue/ticket to the OpenDFDL github for this. >>> >>> There are a few enhancements to this thing that would be helpful. >>> >>> As is, it only requires Saxon-PE, which is affordable for most usages. >>> >>> Another feature would be a cache of the compiled DFDL schema so that the >>> penalty >>> to compile it isn't being paid every time you run the XSLT. (Something >>> many >>> usages of Daffodil need - probably should be a feature supplied in a >>> Daffodil >>> library so that it's not being reinvented.) Daffodil compilation times are >>> acceptable now for long-running processes, but not for quick start/stop >>> things >>> as XSLT commonly is. >>> >>> We should look into whether Java's built in XSLT could be used to do this >>> also. >>> >>> Roger suggests you just need XSLT 2.0, and maybe Java's built in XSLT is >>> 2.0 ? I >>> know it is not XSLT 3.0. But I wasn't able to quickly determine if it is >>> stuck >>> at version 1.0 or has been updated to 2.0. >>> >>> I am unclear if or how Java's XSLT supports extension functions however. >>> But I >>> suspect it does. The trick is the parse-xml() function or other way to >>> construct >>> the XML again that the XSLT operates on. >>> >>> >>> >>> >>> -------------------------------------------------------------------------------- >>> *From:* Steve Lawrence <slawre...@apache.org> >>> *Sent:* Tuesday, July 7, 2020 8:15 AM >>> *To:* users@daffodil.apache.org <users@daffodil.apache.org> >>> *Subject:* Re: Here's how XSLT programs can use DFDL/Daffodil >>> Great! Thanks Mike! >>> >>> The only downside to this that I see is that it requires Saxon-EE to use >>> reflexive extension functions, which takes a little bit of effort to get. >>> >>> I'm wondering if it would be worth updating this to use Saxon's newer >>> integrated extension functions [1] rather than the reflexive extension >>> functions? It looks like it requires a bit more code to setup the >>> extension function and register with Saxon, but it is supported in >>> Saxon-HE which is much easier to get via maven. And I'd like to >>> integrate this into a Daffodil regression suite, which would be easier >>> if all dependencies were in maven. >>> >>> I can take a look at making this update if this seems reasonable. >>> >>> - Steve >>> >>> [1] >>> http://www.saxonica.com/documentation/#!extensibility/integratedfunctions >>> >>> >>> On 7/6/20 7:04 PM, Beckerle, Mike wrote: >>>> I captured Roger's example from this email and put in on github here with >>>> a test. >>>> >>>> https://github.com/OpenDFDL/examples/tree/master/xslt-csv >>>> >>>> This isn't part of Daffodil, but it's an interesting example combining >>>> XSLT with >>>> Daffodil. >>>> >>>> -------------------------------------------------------------------------------- >>>> *From:* Roger L Costello <coste...@mitre.org> >>>> *Sent:* Tuesday, June 2, 2020 12:10 PM >>>> *To:* users@daffodil.apache.org <users@daffodil.apache.org> >>>> *Subject:* Here's how XSLT programs can use DFDL/Daffodil >>>> >>>> Hi Folks, >>>> >>>> Below is an XSLT program that processes XML-formatted CSV. The XSLT >>>> program >>>> calls an external Java program, passing it the name of a DFDL file >>>> (csv.dfdl.xsd) and the name of a CSV file (csv.txt). The Java program >>>> calls >>>> Daffodil which produces XML-formatted CSV. The Java program returns the >>>> XML as a >>>> string. The XSLT program uses the parse-xml() function to convert the >>>> string to >>>> XML. >>>> >>>> <xsl:stylesheetxmlns:xsl="http://www.w3.org/1999/XSL/Transform"; >>>> xmlns:xs="http://www.w3.org/2001/XMLSchema"; >>>> xmlns:dfdl="java:runDaffodil" >>>> version="3.0"> >>>> >>>> <xsl:variablename="dfdl"select="'csv.dfdl.xsd'"/> >>>> <xsl:variablename="input"select="'csv.txt'"/> >>>> >>>> <xsl:templatematch="/"> >>>> <!-- Convert the CSV text file to XML using DFDL --> >>>> <xsl:variablename="csv-string"select="/dfdl:dfdlParse/(*$dfdl*,*$input*)"as="xs:string"/> >>>> <xsl:variablename="csv-xml"select="/parse-xml/(*$csv-string*)"as="document-node()"/> >>>> <!-- Process the XML-formatted CSV here --> >>>> <xsl:variablename="numRecords"select="/count/(*$csv-xml*//*record*)"as="xs:integer"/> >>>> <!-- Output the number of records in the CSV file --> >>>> <xsl:sequenceselect="*$numRecords*"/> >>>> <!-- Output the model (field 3) and year (field 1) of each Chevy auto --> >>>> <xsl:for-eachselect="*$csv-xml*//*record*[*field*[2] eq'Chevy']"> >>>> <xsl:sequenceselect="(*field*[3]//data/(),' ',*field*[1]//data/())"/> >>>> </xsl:for-each> >>>> </xsl:template> >>>> </xsl:stylesheet> >>>> >>>> The XSLT program uses XSLT version 3.0. You can use any version except 1.0 >>>> (the >>>> parse-xml() function is not present in XSLT 1.0). I used Saxon as the XSLT >>>> processor. >>>> >>>> The Java program calls Daffodil which parses the CSV file using the DFDL >>>> schema. >>>> Here is the Java program: >>>> >>>> *import*java.io.IOException; >>>> *import*java.net.URISyntaxException; >>>> *import*java.net.URL; >>>> >>>> *import*org.jdom2.Document; >>>> *import*org.jdom2.output.XMLOutputter; >>>> >>>> *import*org.apache.daffodil.japi.Compiler; >>>> *import*org.apache.daffodil.japi.Daffodil; >>>> *import*org.apache.daffodil.japi.DataProcessor; >>>> *import*org.apache.daffodil.japi.Diagnostic; >>>> *import*org.apache.daffodil.japi.ParseResult; >>>> *import*org.apache.daffodil.japi.ProcessorFactory; >>>> *import*org.apache.daffodil.japi.infoset.JDOMInfosetOutputter; >>>> *import*org.apache.daffodil.japi.io.InputSourceDataInputStream; >>>> >>>> *public**class*runDaffodil { >>>> >>>> *public**static*String dfdlParse(String dfdl,String >>>> input)*throws*IOException,URISyntaxException { >>>> >>>> URL dfdlURL =runDaffodil.*class*.getResource(dfdl); >>>> URL inputURL =runDaffodil.*class*.getResource(input); >>>> >>>> // >>>> // First, compile the DFDL Schema >>>> // >>>> Compiler c =Daffodil.compiler(); >>>> ProcessorFactory pf =c.compileSource(dfdlURL.toURI()); >>>> DataProcessor dp =pf.onPath("/"); >>>> >>>> // >>>> // Parse - parse data to XML >>>> // >>>> java.io.InputStream is =inputURL.openStream(); >>>> InputSourceDataInputStream dis =*new*InputSourceDataInputStream(is); >>>> >>>> // >>>> // Setup JDOM outputter >>>> // >>>> JDOMInfosetOutputter outputter =*new*JDOMInfosetOutputter(); >>>> >>>> // >>>> // Do the parse >>>> // >>>> ParseResult res =dp.parse(dis,outputter); >>>> >>>> // >>>> // Return the XML as a string >>>> // >>>> Document doc =outputter.getResult(); >>>> *return**new*XMLOutputter().outputString(doc); >>>> } >>>> } >>>> >>> >> >
