Thanks for great insights and experience reports Mike.
We’ve had good experience with OpenEXI (Nagasena) and Exificient, also preferring Schema Aware. Am finally recovering from a laptop failure… lots of things to check here. We expect to revisit our code applying jars of each, will report back results sometime next month. * https://openexi.sourceforge.net * http://exificient.github.io all the best, Don -- Don Brutzman Naval Postgraduate School, Code USW/Br brutz...@nps.edu Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149 X3D graphics, virtual worlds, navy robotics https://faculty.nps.edu/brutzman From: Mike Beckerle <mbecke...@apache.org> Sent: Friday, April 14, 2023 9:30 AM To: users@daffodil.apache.org Cc: Brutzman, Donald (Don) (CIV) <brutz...@nps.edu> Subject: Re: How to generate an EXI file from the command line? Hi Don, long time no email. I concur EXI is very important. We maybe have not waved the flag hard enough about this. I've become a big advocate of EXI. We did a few internal studies at Owl, and even very intricate data formats like VMF(mil-std-6017) which is full of flag bits and everything is optional or repeating, nothing is byte aligned, everything is bit packed, when you parse with DFDL and convert back to EXI, you are at about 200% of native size even with regular schema-un-aware EXI, and can get down to about 110% of native size using SA (schema-aware) EXI. XML text is typically 1000% relative to native size (or worse) for detailed fine-grained data formats like this where a single bit can turn into XML text like: "<big_element_name>NORMAL</big_element_name>". And this is without turning on the EXI "compression" feature, so it's just the power of the EXI encoding alone. This has rather big performance implications. I wish EXI-SA was easier to use, but EXI-regular (regular meaning not SA/Schema-Aware) is already a huge win over XML text. Open source Exificient is fine to use for EXI-regular. For EXI-SA I have to recommend the closed-source tools from Agile Delta, because they have a schema compiler that pre-digests the XSD (or DFDL) schema. Some of our schemas are really big. Our VMF schema is 180K lines of DFDL/XSD spread over 135 files, composed of 5 separate schema components (separate schema modules that are testable separately, being combined together). It's important to go through digesting this exactly once and save it, and not once per process each time it is processing the EXI-SA data. -mike beckerle On Fri, Apr 14, 2023 at 11:29 AM Brutzman, Donald (Don) (CIV) <brutz...@nps.edu <mailto:brutz...@nps.edu> > wrote: Great to hear, very important, thanks! all the best, Don -- Don Brutzman Naval Postgraduate School, Code USW/Br brutz...@nps.edu <mailto:brutz...@nps.edu> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149 X3D graphics, virtual worlds, navy robotics https://faculty.nps.edu/brutzman From: Mike Beckerle <mbecke...@apache.org <mailto:mbecke...@apache.org> > Sent: Friday, April 14, 2023 8:22 AM To: users@daffodil.apache.org <mailto:users@daffodil.apache.org> Subject: Re: How to generate an EXI file from the command line? Yes, the release note says: The Daffodil CLI adds two new infoset types–-I exi and -I exisa–to support infosets represented as EXI binary XML for non-schema aware and schema aware EXI, respectively. I guess the update to that web page never got propagated. I'll open a ticket to fix. On Fri, Apr 14, 2023 at 10:48 AM Roger L Costello <coste...@mitre.org <mailto:coste...@mitre.org> > wrote: I looked on the web page for CLI arguments: https://daffodil.apache.org/cli/ <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdaffodil.apache.org%2Fcli%2F&data=05%7C01%7Cbrutzman%40nps.edu%7Cafcc620328a241fcd50e08db3d05903e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638170866326429359%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9CzdtabvQPoq3lEm%2FgjEKTrGApi103dFN4tJ3DyCZMc%3D&reserved=0> and it says nothing about EXI. I thought that Daffodil 3.4 supports EXI? >From the command line, can I instruct Daffodil to output EXI?
smime.p7s
Description: S/MIME cryptographic signature
