I like Gehard's suggestion as well.
This design allows for PDs and the output module to split the work. Each
output module will need to document 1) what metadata it needs and 2) what the
PD's binary stream will look like. In most cases, I would expect the PD's
binary stream to be very close to the final format (ex: WAV and MIDI).
Theoretically, as Gerhard suggested earlier, an output module could go a step
further and translate from one format to another similar format (ex: all
audio-based PDs output WAV data but the output module can write it as WAV or AU
format).
Sent from my Verizon, Samsung Galaxy smartphone
-------- Original message --------
From: Soeren Apel <soe...@apelpie.net>
Date: 10/11/16 10:50 AM (GMT-07:00)
To: Gerhard Sittig <gerhard.sit...@gmx.net>, sigrok-devel@lists.sourceforge.net
Subject: Re: [sigrok-devel] Outputting files from Protocol Decoders
I like this approach.
What do you guys think about a meta packet that describes
the meaning/content of the binary output? With that, the
I2S PD could announce sample rate/channels/resolution
(before or after starting output, wouldn't really matter)
and the sigrok client could pass this meta data to an
output module along with the binary data that was captured
from the PD.
That way, the PD can stream whatever it has, the output
module can create an output file in any way it wants and
the libsigrok/libsigrokdecode infrastructure can remain
virtually unchanged.
The icing on the cake would be that a "MIDI" output module
(or any other) could easily be made to deal with binary data
from different PDs - in contrast to having the need to
implement such I/O in every single PD directly.
Any thoughts?
All the best,
-Soeren
On Tue, 2016-10-11 at 12:54 +0200, Gerhard Sittig wrote:
> [ summary: use an output module, see srzip for prior art ]
>
> On Tue, Oct 11, 2016 at 06:03 +0000, Chris Dreher wrote:
> >
> >
> > It seems the streaming design of sigrok eliminates a lot of
> > file formats. Namely, most file formats with a size field that
> > precedes a large variable amount of data can not be supported
> > in sigrok. This includes file formats that are 99%
> > stream-oriented. For example, MIDI files are 99% compatible
> > with sigrok's stream design other than a single 32-bit size
> > field in the track chunk. For the MIDI protocol, there are not
> > good alternatives to saving data other than in a MIDI file. It
> > is unfortunate that such files currently can not be supported
> > by sigrok.
> >
> > That said, I get that sigrok's primary goal is support for
> > logic analysis and oscilloscope. Being able to extract and
> > save data to a file format is secondary.
>
> This might be too strict an interpretation. I still feel that
> what you want to achieve _is_ possible, just not from within
> _decoders_. It's simply that seek operations and manipulation of
> previously written data needs to occur in the most appropriate
> location, which is not from within a pipe that operates on
> strictly linear streams.
>
> In theory or rather strictly speaking, output modules might
> assume mere streaming, too. It's good practice to avoid
> unnecessary dependencies when possible. But there is prior art
> where some output module has direct access to a file object, and
> does manipulate previously written data as more chunks of
> information become available.
>
>
> It's true that in the current set of output modules most fill in
> an "out" string parameter in their receive() method in straight
> forward ways (mere formatting of what they received). Those
> strings then are put into an output stream in strict linear ways.
>
> The CSV output mostly passes on the samples, and later writes
> additional gnuplot(1) information that was accumulated in the
> process (if the respective feature was enabled). Multiple files
> appear to get written by that one output module (optional).
>
> The "srzip" output module for sigrok's native .sr file format
> stands out in that it operates on a ZIP archive that gets created
> initially, and gets updated in several steps at several points in
> time while the input stream is being received and processed. See
> the zip_create() and zip_append() routines.
>
> So I wouldn't consider it too far a stretch when the WAV (and
> later MIDI) output modules insist in accessing a file object
> directly, open the file with an initial header which reflects the
> available information at the time of file creation, and update
> that header as more chunks or samples get appended to the file.
> This seems to perfectly be in line with what existing output
> modules do already. It's reflecting on a property of the very
> format that the user requested for the output.
>
> For the record: The notion of "internal I/O handling" in output
> modules and the associated "output flag" was introduced in commit
> 3cd4b381 as of 2015-08-15.
>
>
> Note that I might be slightly off in my reading of the CSV output
> module's logic, only had a cursory look. But the srzip output
> module certainly does have a test for "filename available?" in
> init(), and calls zip_add(), zip_rename(), and zip_replace()
> routines which do more than merely append data at the end of a
> previously written file. Even if one interprets the ZIP archive
> as "just a directory" -- still previously written members get
> updated as required. And none of this feels wrong to me, it all
> looks appropriate. Just not for a decoder. :)
>
>
> >
> > Do people think addressing this issue is worth fixing or is
> > time better spent on other issues and features?
>
> I think that the decoder API doesn't need fixing. When
> individual decoder implementations assume conditions that are not
> met, then the implementations need fixing but not the design.
>
> "The issue" goes away if you move the respective manipulation
> logic from the decoder to an output module.
>
>
> virtually yours
> Gerhard Sittig
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
sigrok-devel mailing list
sigrok-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sigrok-devel
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
sigrok-devel mailing list
sigrok-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sigrok-devel
