Bert Vermeulen wrote: > Zoubek Ondřej wrote: >> Hello sigrok developers! > > Hi! Welcome to the sigrok project -- sorry for the late reply.
Hi, > >> I would like to post few notes on proposed API >> (http://www.sigrok.org/wiki/Formats_and_structures). >> >> struct samplerates >> ------------------ >> >> generally it is not good idea to represent sample rate in frequency >> units (1/sec), instead of period (sec). >> >> This is because lot (if not all) LA are based on some type of FPGA. >> Idea, that there is some sort of PLL is odd. Most of analyzers will >> have capability to capture samples at some base_rate and its >> multiples. For example Sigma can capture at sample rates 50MHz, 25MHz, >> 16.666MHz, 12.5MHz, 10MHz, 8.333MHz, 7.143Hz etc... In case of Sigma, >> the divisor is 8-bit, so the values in proposed structure would be: >> low=196078; high=50000000; step=0; and list would contain 255 values. >> Imagine what would happen if I say the divisor is 16-bit. >> >> On the other hand, if the units would be, say picoseconds, this would >> give: low=5100000; high=20000; step=20000, which is much simplier, >> despite the fact, that frequecies like 50/3MHz would not never be >> recorded exactly. >> >> We fight another problem, that periods, like 3.333ns are also often >> used. So one should define the time unit as 15 = 1ps or similar to >> cover divisors 3 and 5. > > Yeah, that struct really isn't good enough to express thecapabilities of > any but the simplest of devices. The problem is we want both an accurate > way of passing on a device's capabilities to the frontend, but still > have the frontend be able present "nice" choices to the user. For > example it's ok to have the user select "8 Mhz" when it's really 8.333 > -- but it's NOT ok to then work out timing based on that 8 Mhz, because > that's just incorrect (more on this below). > > We can cover some of this transparently -- have the driver quietly > convert to the nearest requested rate, if it's not too far off, or have > the frontend ask the driver in real time if a requested rate is valid > (this is a cheap operation, after all). But still not enough. > > I'm not sure I understand what you mean by 15 = 1ps? Hiding actual rate is sufficient, but not good enough, because experienced user want to count with antialiasing effects at given sample rate. By 15 = 1ps I meant scale which would mean that sample rate 10ns would be expressed as 15*10000=150000. The good thing on this is for example sample rate 3.333ns=50000, which is accurate. Multiplier 15 covers many common imaginable sample rates. > >> Please note, that Sigma can also measure at samplerate defined by >> external clock. This means thereI meant should be (at least in context of >> DF_HEADER packet) value of samplerate meaning "unknown". > > Yup, and this is a feature that is needed by all but the simplest of > logic analyzers really. The OLS needs it as well, for example. Haven't > quite worked out how we're going to encode that, but it won't be a big > issue. > Great. > >> struct datafeed >> --------------- >> >> I don't see anywhere in DF_LOGICxx packets usage of timestamps. Or am >> I missing something? For RLE (or maybe something better) enabled logic >> analyzers, this is crucial. With Sigma, typical 2 minute test have 6G >> of samples. > > The idea is that the timestamp of any given sample can be calculated by > using the offset in the datafeed and the samplerate of that feed. There > is no problem with that as such, but as I said, the samplerate does have > to match the hardware's real samplerate *exactly* for this to work well. > > However, the datafeed packets are going to get an extra "offset" field > soon, because we'll need that for the protocol decoders. The > input/output into the PD interface code is just datafeed as well (with a > different DF_ type), and the offset is needed to sync output of a > decoder with the raw data (and that of other decoders). > > I will admit, that 6G has got me thinking -- never thought of such an > amount of data. Currently we're filtering out unused probes (channels) > from the driver's output, which can reduce the # of bytes per sample, > but that's just a matter of cleaning up, not compression. We also have > the datstore backend, which is at least useful for storing large amounts > of data without requiring a large continuous chunk of memory. > > You're making me think we need compression now -- 6G really is too much. > The right place to introduce this would be in that datastore, I think. > Well, in the software for Sigma, which is also designed to be universal, the data are maintained in the logic analyzer code. This idea is because the logic analyzer plugin and only it knows best how to effectively and quickly compress (maybe with some hardware aid) the data. The data are not streamed from plugin, instead of it others asks for them (in decompressed form very similar to the form of yours DF_ packets), but with ability to express uncertainty and analog signals. PD also use this interface. When gui asks PD the data in view, it intensively asks LA for data to decode something. To make it simplier next time, PD remembers some hints, like start bit on uart, start on i2c etc.. So the amount of memory is kept as low as possible, everything is computed realtime. This is good also for some type of "virtual" analyzer, which reads only file from disk. There is no need to load entire file into memory, so the file can be very large. It only needs to be indexed by some sort of B-tree. >> I don't see any reference to ability to rewrite already received data. >> The communication of Sigma works in this way: >> The logic analyzer makes a test. The test is stored in up to 32MB >> memory. A "fingerprint" of the test is downloaded and shown to the >> user. This fingerprint is 294912bytes. Memory is divided into 32768 >> chunks and downloaded is 9 bytes per chunk. This takes about half >> second. Each chunk now knows its min and max vector (probes which >> change within it). >> >> The rest of memory is downloaded quietly in background (newly created >> thread). If user zooms onto some segment of test, the corresponding >> chunk is downloaded in preference. Typically, user have its view >> within two seconds. Downloading whole test takes about 1 minute. > > That's amazingly good, I never thought of that. This is definitely > something I want to support in sigrok. I'm thinking along the lines of > adding a "flags" field in in the data, with a "preview" bit to support > this. The priority downloading thing is very nifty, but I'm not sure how > best to handle that though. > >> If I understand it, how the API will work, the analyzer (plugin) upon >> receiving stop_acquisition() will start calling receive_data_callback >> with the measured data in DF_LOGICxx and finally DF_END. But this >> would take Sigma hardware one minute and user will fall asleep. > > Actually no, everything in sigrok is streaming, always. From the driver, > to the session bus, to the protocol decoders and into the frontend -- > it's all datafeed packets. The idea is that while doing acquisition, the > driver is usually idle waiting for data from the hardware, and in that > time the other parts of sigrok can do their thing. Protocol decoders > will decode what they have on the fly, in real time, even as data is > coming in. > > In general, a datafeed always starts with a DF_HEADER packet (there will > likely be more than one header type eventually), followed by one or more > data packets -- currently only DF_LOGIC*, maybe a DF_TRIGGER, and the > stream always ends with a DF_END packet. Actually, this is very good idea. Makes logic analyzers very simple. But I see very hard limitation of that. I have implemented USB 1.1 protocol analyzer (for sigma software indeed). It will be released soon. USB is so complex and fast protocol, so one Sigma test may contain hundreds of packets. To find there something, there is "navigation" window with tree structure. Highest tree level encapsulate whole transfers, lowest level are raw bits. This tree contains about 40 bytes per node (two int64 for time range, 4 pointers to neighborhoods, some flags, ...). In worst case the tree has about 700MB (almost every sample is USB bit). What I want to say is, amount of data coming out of PD may be enormous and you should think about it. > >> Best regards, >> Ondřej > > Let me just say, I'm very, very impressed by the way you guys at Asix > are engaging with us on this! > > Best regards, Ondřej ------------------------------------------------------------------------------ _______________________________________________ sigrok-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sigrok-devel
