Hi all again,
Just wanted to wrap this up. I finally looked up in the source code, where does the
"Rate from timestamp" appear in the source code - and I found it in:
https://github.com/sigrokproject/libsigrok/blob/master/src/input/csv.c
... which at the time of writing, points at commit cd11e33, that is:
https://github.com/sigrokproject/libsigrok/blob/cd11e33/src/input/csv.c
Here are the critical quotes:
* column_formats: Specifies the data formats and channel counts for the
* input file's text columns. [...]
...
* The 't' format marks timestamp values, which could help in automatic
* determination of the input stream's samplerate. This "column_formats"
* option is most versatile, other forms of specifying the column layout
* only exist for backwards compatibility, and are rather limited. They
* exclusively support logic input data in strictly adjacent columns,
* with further constraints on column layout for multi-bit data.
...
* Typical examples of using these options:
...
* - ... -I csv:column_formats=t,2a ...
* The first column contains timestamps, the next two columns contain
* analog values. The capture's samplerate could get determined from
* the timestamp values if not provided by the user by means of the
* 'samplerate' option. This assumes a mere number in units of seconds,
* and equidistant rows, there is no fancy support for textual unit
* suffixes nor gaps in the stream of samples nor other non-linearity,
* just '-' ignore the column if the format is not supported).
...
* Implementor's notes on timestamp interpretation. Use a simple
* approach for improved maintainability which covers most cases
* of input data. There is not much gain in adding complexity,
* users can easily provide the rate when auto-detection fails.
...
* - If a previous timestamp was seen, determine the difference
* between them, and derive the samplerate. Update internal
* state (the value automatically gets sent to the datafeed),
* and clear previous knowledge. Subsequent calls will ignore
* following input data (see above, rate is known).
So, to answer my question:
So - is it possible to "persuade" Pulseview, to somehow interpret the
timestamp column as floating-point seconds?
Well, it sort of does already - but it does **not** interpret them "absolutely"
for each timestamp: PulseView/sigrok at this time assumes that the .csv file contains
samples/timestamps obtained at a constant sampling rate, which is why it simply tries to
calculate the delta between the first two timestamps (it can parse as floating point
seconds), and take that as the sampling period (rate). And once the sampling rate is
known, all the timestamp information is *ignored*, and the subsequent samples are taken
to be consecutive in time (as per constant sampling rate).
So, in the sense of how I intended the question: that is, can PulseView interpret the
timestamp column as floating-point seconds "absolutely" - the answer is: no, it
cannot.
Which means, that the approach I tried to take previously - add two "fake"
entries in the .csv just to cheat the part that determines the sampling rate, and then
leave the rest of the capture as is:
```
Time[s], Ch2, Ch3, Ch4, Ch5, Ch6, Ch7
0.000008640000000, 0, 0, 0, 0, 0, 0
0.000009120000000, 0, 0, 0, 0, 0, 0
1.616108640000000, 0, 0, 0, 0, 1, 0
1.616109120000000, 1, 0, 0, 0, 1, 0
...
```
... can **not** work - because going from timestamp 0.000009120000000 to 1.616108640000000
represents a "gap", if the sampling rate was determined from the delta between
0.000009120000000 and 0.000008640000000 (and as the source code states, "gaps" are not
handled).
In other words - as far as PulseView/sigrok is concerned, everything else
following the first two timestamps happens at the sampling rate determined from
the first two timestamps - so it doesn't really matter if the third timestamp
is 1.616108640000000 or whatever value it is.
Now that I've found the docs on it, I'm OK with this - however, I'd wish the
wiki https://sigrok.org/wiki/File_format:Csv or the help text in `sigrok-cli -I
csv --show` worded this more strongly; for instance, with a statement like:
Note that the timestamp column, at this time, is never interpreted as absolute
timestamps, since constant sampling rate is always assumed, and gaps in the
.csv file are not supported (see comments in libsigrok/src/input/csv.c for
more).
Otherwise, I kept looking at this information, and whenever I see "timestamp", I assume it is an "absolute"
timestamp, since there are no strong wordings that would persuade me to the contrary - and then once I try it out, and things
don't work, I assume I have made an error in "instructing" PulseView/sigrok how to interpret timestamps (instead of
realizing I'm in error of understanding: that PulseView cannot really interpret "absolute" timestamps "with
gaps" in .csv at this time).
Anyways - good to have this finally clarified!
On 2020-10-16 2:28 PM, sdbbs wrote:
Many thanks again, Michael,
The above line is the issue: sigrok treats 0 as an invalid timestamp.
Start your file with 2 entries that have a non-zero timestamp and sigrok
should pick up the correct sample rate.
Afterwards, you can leave gaps as you wish.
...
So the timestamp difference of 4.8e-007 = 4.8 x 10-7 = 0.00000048 seems
correct to me.
So, aside from sigrok skipping the line with timestamp 0, everything seems
to work as intended :)
This looked very promising - so I tried to change the file, so it starts like
this (full file test.csv is attached):
```
Time[s], Ch2, Ch3, Ch4, Ch5, Ch6, Ch7
0.000008640000000, 0, 0, 0, 0, 0, 0
0.000009120000000, 0, 0, 0, 0, 0, 0
1.616108640000000, 0, 0, 0, 0, 1, 0
1.616109120000000, 1, 0, 0, 0, 1, 0
...
```
... just copied the first two 1.* timestamps, and zeroed out the most
significant digits; sigrok does not seem to report any errors anymore,
unfortunately Pulseview still gives me a trace which is 5 us in range (similar
to the pv_test_csv_shown.png attached in the first post).
Is there anything else I could try?
Thanks!
```
$ "C:\Program Files (x86)\sigrok\sigrok-cli\sigrok-cli.exe" -l 5 -I
csv:header=yes:column_formats=t,l,l,l,l,l,l -i test.csv
sr: [00:00.000000] log: libsigrok loglevel set to 5.
sr: [00:00.000000] backend: libsigrok 0.6.0-git-d7df9dc/4:0:0.
sr: [00:00.000000] backend: Libs: glib 2.44.1 (rt: 2.44.1/4401:1), libzip
1.5.2, libserialport 0.1.1/1:0:1 (rt: 0.1.1/1:0:1), libusb-1.0 1.0.20.11003-rc3
API 0x01000104, hidapi 0.8.0-rc1, libftdi 1.4.
sr: [00:00.000000] backend: Host: x86_64-w64-mingw32.static.posix,
little-endian.
sr: [00:00.000000] backend: SCPI backends: TCP, serial, USBTMC.
sr: [00:00.000000] backend: Firmware search paths:
sr: [00:00.019000] backend: - C:\Users\user\AppData\Local\sigrok-firmware
sr: [00:00.019000] backend: - C:\ProgramData\sigrok-firmware
sr: [00:00.019000] backend: - C:\Users\Public\Documents\sigrok-firmware
sr: [00:00.019000] backend: - C:\Program Files
(x86)\sigrok\sigrok-cli\share\sigrok-firmware
sr: [00:00.019000] backend: - C:\Program Files
(x86)\sigrok\sigrok-cli\share\sigrok-firmware
sr: [00:00.019000] backend: Sanity-checking all drivers.
sr: [00:00.019000] backend: Sanity-checking all input modules.
sr: [00:00.020000] backend: Sanity-checking all output modules.
sr: [00:00.020000] backend: Sanity-checking all transform modules.
srd: libsigrokdecode loglevel set to 5.
sr: [00:00.024000] input/csv: User specified column_formats: t,l,l,l,l,l,l.
sr: [00:00.024000] input: Sending 398 bytes to csv module.
sr: [00:00.024000] input/csv: Got 7 columns in text line: Time[s], Ch2, Ch3,
Ch4, Ch5, Ch6, Ch7.
sr: [00:00.024000] input/csv: fmt t -> 1 cols, timestamp fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.024000] input/csv: Column format t,l,l,l,l,l,l -> 7 columns, 6
logic, 0 analog channels.
sr: [00:00.024000] input: Calling end() on csv module.
sr: [00:00.024000] session: bus: Received SR_DF_HEADER packet.
cli: Received SR_DF_HEADER.
sr: [00:00.024000] input/csv: Header line 1 skipped.
sr: [00:00.024000] input/csv: First timestamp value 8.64e-006 in line 2.
sr: [00:00.024000] input/csv: Second timestamp value 9.12e-006 in line 3.
sr: [00:00.024000] input/csv: Timestamp difference 4.8e-007 in line 3.
sr: [00:00.024000] input/csv: Rate from timestamp 2.08333e+006 in line 3.
sr: [00:00.024000] input/csv: Blank line 12 skipped.
sr: [00:00.024000] session: bus: Received SR_DF_META packet.
cli: Received SR_DF_META.
cli: Got samplerate 2083333 Hz.
META samplerate: 2083333
sr: [00:00.024000] session: bus: Received SR_DF_LOGIC packet (10 bytes,
unitsize = 1).
cli: Received SR_DF_LOGIC (10 bytes, unitsize = 1).
libsigrok 0.6.0-git-d7df9dc
Acquisition with 6/6 channels at 2.083333 MHz
sr: [00:00.025000] session: bus: Received SR_DF_END packet.
Ch2:00010001 00
Ch3:00000100 01
Ch4:00000000 00
Ch5:00000000 00
Ch6:00111111 11
Ch7:00000000 00
cli: Received SR_DF_END.
sr: [00:00.025000] hwdriver: Cleaning up all drivers.
```
On 2020-10-16 2:12 PM, Michael Stapelberg wrote:
On Fri, Oct 16, 2020 at 1:53 PM Smilen Dimitrov <s...@brothers-sons.dk> wrote:
Hi Michael,
Thanks a lot for the feedback!
The csv import logs a few debug messages that might be helpful here.
On Linux, specifying -l 5 when calling sigrok-cli makes these visible.
I’m not sure how to do it on Windows. Maybe sigrok-cli works in the
command
line there, too?
Sure it does - here are the results on my machine, of a sigrok-cli command
resembling the "(automatic samplerate)" example in the wiki; hopefully
they'll reveal something to get PulseView working (note, I'm running this
in a MSYS2 bash shell on Windows 10):
```
$ "C:\Program Files (x86)\sigrok\sigrok-cli\sigrok-cli.exe" --version
sigrok-cli 0.8.0-git-0e2f95c
Libraries and features:
- libsigrok 0.6.0-git-d7df9dc/4:0:0 (rt: 0.6.0-git-d7df9dc/4:0:0).
- Libs:
- glib 2.44.1 (rt: 2.44.1/4401:1)
- libzip 1.5.2
- libserialport 0.1.1/1:0:1 (rt: 0.1.1/1:0:1)
- libusb-1.0 1.0.20.11003-rc3 API 0x01000104
- hidapi 0.8.0-rc1
- libftdi 1.4
- Host: x86_64-w64-mingw32.static.posix, little-endian.
- SCPI backends: TCP, serial, USBTMC.
- libsigrokdecode 0.6.0-git-115f19c/4:0:0 (rt: 0.6.0-git-115f19c/4:0:0).
- Libs:
- glib 2.44.1 (rt: 2.44.1/4401:1)
- Python 3.4.4 / 0x30404f0 (API 1013, ABI 3)
- Host: x86_64-w64-mingw32.static.posix, little-endian.
$ "C:\Program Files (x86)\sigrok\sigrok-cli\sigrok-cli.exe" -l 5 -I
csv:header=yes:column_formats=t,l,l,l,l,l,l -i test.csv
sr: [00:00.000000] log: libsigrok loglevel set to 5.
sr: [00:00.000000] backend: libsigrok 0.6.0-git-d7df9dc/4:0:0.
sr: [00:00.000000] backend: Libs: glib 2.44.1 (rt: 2.44.1/4401:1), libzip
1.5.2, libserialport 0.1.1/1:0:1 (rt: 0.1.1/1:0:1), libusb-1.0
1.0.20.11003-rc3 API 0x01000104, hidapi 0.8.0-rc1, libftdi 1.4.
sr: [00:00.000000] backend: Host: x86_64-w64-mingw32.static.posix,
little-endian.
sr: [00:00.000000] backend: SCPI backends: TCP, serial, USBTMC.
sr: [00:00.000000] backend: Firmware search paths:
sr: [00:00.019000] backend: - C:\Users\user\AppData\Local\sigrok-firmware
sr: [00:00.019000] backend: - C:\ProgramData\sigrok-firmware
sr: [00:00.019000] backend: - C:\Users\Public\Documents\sigrok-firmware
sr: [00:00.019000] backend: - C:\Program Files
(x86)\sigrok\sigrok-cli\share\sigrok-firmware
sr: [00:00.019000] backend: - C:\Program Files
(x86)\sigrok\sigrok-cli\share\sigrok-firmware
sr: [00:00.019000] backend: Sanity-checking all drivers.
sr: [00:00.019000] backend: Sanity-checking all input modules.
sr: [00:00.019000] backend: Sanity-checking all output modules.
sr: [00:00.019000] backend: Sanity-checking all transform modules.
srd: libsigrokdecode loglevel set to 5.
sr: [00:00.027000] input/csv: User specified column_formats: t,l,l,l,l,l,l.
sr: [00:00.028000] input: Sending 362 bytes to csv module.
sr: [00:00.028000] input/csv: Got 7 columns in text line: Time[s], Ch2,
Ch3, Ch4, Ch5, Ch6, Ch7.
sr: [00:00.028000] input/csv: fmt t -> 1 cols, timestamp fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: fmt l -> 1 cols, binary fmt, 1 bits, rc 0
sr: [00:00.028000] input/csv: Column format t,l,l,l,l,l,l -> 7 columns, 6
logic, 0 analog channels.
sr: [00:00.029000] input: Calling end() on csv module.
sr: [00:00.029000] session: bus: Received SR_DF_HEADER packet.
cli: Received SR_DF_HEADER.
The following lines are relevant:
sr: [00:00.030000] input/csv: Header line 1 skipped.
sr: [00:00.030000] input/csv: Cannot convert timestamp text
0.000000000000000 in line 2 (or zero value).
The above line is the issue: sigrok treats 0 as an invalid timestamp.
Start your file with 2 entries that have a non-zero timestamp and sigrok
should pick up the correct sample rate.
Afterwards, you can leave gaps as you wish.
sr: [00:00.030000] input/csv: First timestamp value 1.61611 in line 3.
sr: [00:00.030000] input/csv: Second timestamp value 1.61611 in line 4.
sr: [00:00.030000] input/csv: Timestamp difference 4.8e-007 in line 4.
sr: [00:00.030000] input/csv: Rate from timestamp 2.08333e+006 in line 4.
sr: [00:00.030000] input/csv: Blank line 11 skipped.
sr: [00:00.030000] session: bus: Received SR_DF_META packet.
cli: Received SR_DF_META.
cli: Got samplerate 2083333 Hz.
META samplerate: 2083333
You mentioned your file looks like this:
Time[s], Ch2, Ch3, Ch4, Ch5, Ch6, Ch7
0.000000000000000, 0, 0, 0, 0, 0, 0
1.616108640000000, 0, 0, 0, 0, 1, 0
1.616109120000000, 1, 0, 0, 0, 1, 0
So the timestamp difference of 4.8e-007 = 4.8 x 10-7 = 0.00000048 seems
correct to me.
So, aside from sigrok skipping the line with timestamp 0, everything seems
to work as intended :)
sr: [00:00.030000] session: bus: Received SR_DF_LOGIC packet (9 bytes,
unitsize = 1).
cli: Received SR_DF_LOGIC (9 bytes, unitsize = 1).
libsigrok 0.6.0-git-d7df9dc
Acquisition with 6/6 channels at 2.083333 MHz
sr: [00:00.030000] session: bus: Received SR_DF_END packet.
Ch2:00100010 0
Ch3:00001000 1
Ch4:00000000 0
Ch5:00000000 0
Ch6:01111111 1
Ch7:00000000 0
cli: Received SR_DF_END.
sr: [00:00.031000] hwdriver: Cleaning up all drivers.
```
On Fri, Oct 16, 2020 at 12:46 PM sdbbs <s...@brothers-sons.dk> wrote:
Hi all,
Apologies in advance if I'm in the wrong forum for asking this - if so,
please let me know where I can ask this instead.
I have seen https://sigrok.org/wiki/File_format:Csv - and from the
example called:
Mixed signal data in arbitrary order. Timestamps and automatic
samplerate.
... it is implied, that it can accept time in floating point format -
but
it is unclear what the units are.
So, I have this test.csv file, which is attached - and here is a
snippet:
```
Time[s], Ch2, Ch3, Ch4, Ch5, Ch6, Ch7
0.000000000000000, 0, 0, 0, 0, 0, 0
1.616108640000000, 0, 0, 0, 0, 1, 0
1.616109120000000, 1, 0, 0, 0, 1, 0
...
```
Here, time column has floating point values, with the meaning/unit of
seconds.
So, using Pulseview 0.5.0-git-9d307c6 on Windows 10, I click "Import
Comma-separated values...", choose test.csv, and then enter
`t,l,l,l,l,l,l`
for the "Column format specs" argument, and leave everything else as is
(see attached pv_test_csv_import.png)
Once the import is done, I get the visualization shown on the attached
pv_test_csv_shown.png - as you can see, the entire range of the capture
shown is about 4500 ns, that is 4.6 us - which has nothing to do with
the
timestamps as given in the .csv file (first timestamp is 0, next one is
at
1.61 seconds!).
So - is it possible to "persuade" Pulseview, to somehow interpred the
timestamp column as floating-point seconds?
Thanks in advance for any answers!
_______________________________________________
sigrok-devel mailing list
sigrok-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sigrok-devel
_______________________________________________
sigrok-devel mailing list
sigrok-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sigrok-devel
_______________________________________________
sigrok-devel mailing list
sigrok-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sigrok-devel
