Re: [Owfs-developers] Understanding BAE's read access to generic/memory
Hi list, I wanted to close the loop on this with one final comment: On 07/07/2011 10:55 PM, Eloy Paris wrote: On 07/07/2011 04:41 PM, Paul Alfille wrote: I think I'm starting to see your question. You are wondering how to read and write in the middle of a file (memory buffer). Yes, with the added complexity (for me to understand how things work) that I was trying to do this from the command line, with regular Unix utilities like cat . The command line is an awkward place to do partial file operations. That's what I learned today. Perhaps dd, which allows to specify an offset and a read size, would work, but definitely not cat. As Paul mentioned, there are several ways to read a block of memory from a device (for those 1-Wire devices that support this operation). To read from the command line one could use owread. However, cat cannot be used because it does not result in an operating system read() function call, which uses offset and size parameters that end up in a 1-Wire query (owq) structure. However, in addition to owread, I found that the dd utility also allows to do memory reads. Here are two ways I was able to do a device memory read using dd: dd bs=1 count=2304 if=/owfs2/99.010203040506/generic/memory of=file.bin dd bs=2304 count=1 if=/owfs2/99.010203040506/generic/memory of=file.bin The first dd invocation took considerable longer than the second because, since the block size is 1 (bs=1), it actually issues 2304 1-Wire read memory commands to read a single byte each time. This results in a *lot* of overhead. It's a good stress-test of the system, though ;-) The second invocation is faster because it issues less 1-Wire read memory commands (less overhead). The number of 1-Wire read memory commands issued depends on how much memory can be read at once -- on the BAE and on my device it is 32 bytes (limitation set on the owfs side). Thanks to Paul and Pascal for their insightful comments. Cheers, Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
owq is filled by the read command -- i.e. a filesystem read has size and offset as part of the call. Buffer (for the result) is allocated as well. You don't need to worry about how it's filled in, that comes from the OW_read routines (or OW_write). Those MACROs: OWQ_buffer, OWQ_size, OWQ_offset call the relevant fields from the structure. OW_r_mem, in this case, takes those parameters, and knows what 1-wire commands to send to the device to read the memory. In the case of BAE, as I recall, there was the added complexity that the chip and memory read is designed as a long linear address space, while the various fields exposed (EEPROM, user mem, variables) are subsets of that space. We remap the start to correspond to the correct chip address space while keeping the bounds checking correct. All this is a thin object-oriented layer written in C. owq stands for one-wire-query, and hold the field's size, offset, data buffer, and all the flags and parsed path information. This way we use a similar data structure independent of the data type (float, integer, boolean, data, memory). Paul Alfille On Thu, Jul 7, 2011 at 3:15 PM, Eloy Paris pe...@chapus.net wrote: Hi list, I am working on a micro-controller-based 1-Wire device and am looking at ow_bae.c as a guide in implementing OWFS support for my device. OWFS BAE support has the capability to read micro-controller memory by accessing generic/memory and I'd like to implement that capability for my device. There is something I don't understand, though -- the FS_r_mem() function does this: OW_r_mem( (BYTE *) OWQ_buffer(owq), OWQ_size(owq), OWQ_offset(owq), PN(owq)) owq is a struct one_wire_query pointer. What has me puzzled is: how does one put into owq the size and offset of the memory block to be read? In other words, it is not clear to me how one would access generic/memory to read a certain number of bytes starting at some arbitrary offset. Writing the micro-controller code to handle the memory read (and subsequent 1-Wire transfer from micro-controller to bus master) is easy but I am struggling with the OWFS side of things. Thanks in advance for any hints. Cheers, Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
I think I'm starting to see your question. You are wondering how to read and write in the middle of a file (memory buffer). The command line is an awkward place to do partial file operations. From a C program, you can use lseek and read From the owshell routines, there is owread --size=nnn --start=nnn --hex owwrite --start=nnn --hex From the language bindings there is ow_lread and ow_lwrite Also OWNet (like OWNet.pm the perl routine) use the ownet protocol (the tcp protocol) that includes size and offset. I don't think you can use the web interface to do partial reads or writes, and I'm not sure about ftp. What we did with BAE is make distinct memory areas (like the eeprom area) have their own property, and then expect users to use them as atomic blocks. The fact that the chip only permits smaller pages to be written at a time is hidden. Can you explain your requirements more extensively? Paul Alfille On Thu, Jul 7, 2011 at 3:22 PM, Eloy Paris pe...@chapus.net wrote: On 07/07/2011 03:15 PM, Eloy Paris wrote: [...] Writing the micro-controller code to handle the memory read (and subsequent 1-Wire transfer from micro-controller to bus master) is easy but I am struggling with the OWFS side of things. I could do something like: $ echo 0 /owfs/xx.n/generic/offset $ echo 128 /owfs/xx.n/generic/length $ cat /owfs/xx.n/generic/memory file.bin However, the OWFS BAE support does not have such an interface, i.e. offset and length directory entries; instead, it seems like it expects offset and length parameters as part of the same read operation, so I am confused :-( Puzzled, Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
Hi Paris, you may want to look at the 'writebyte' I've utilized to test some features sometime ago. the code is now commented out but still present. The principle is to consider the writebyte property as a word value (two bytes) the hi-byte is parsed as the offset and the lo-byte is parsed as the value to write. for example echo 513 /owfs/xx.n/writebyte will write the value 1 (513 |0Xff) to offset 2 (5138) for more advanced use of a generic memory, you may create a byte array of 256 elements wich will be individually addressable in both read and write function. (see aggregate examples) Pascal 2011/7/7 Paul Alfille paul.alfi...@gmail.com I think I'm starting to see your question. You are wondering how to read and write in the middle of a file (memory buffer). The command line is an awkward place to do partial file operations. From a C program, you can use lseek and read From the owshell routines, there is owread --size=nnn --start=nnn --hex owwrite --start=nnn --hex From the language bindings there is ow_lread and ow_lwrite Also OWNet (like OWNet.pm the perl routine) use the ownet protocol (the tcp protocol) that includes size and offset. I don't think you can use the web interface to do partial reads or writes, and I'm not sure about ftp. What we did with BAE is make distinct memory areas (like the eeprom area) have their own property, and then expect users to use them as atomic blocks. The fact that the chip only permits smaller pages to be written at a time is hidden. Can you explain your requirements more extensively? Paul Alfille On Thu, Jul 7, 2011 at 3:22 PM, Eloy Paris pe...@chapus.net wrote: On 07/07/2011 03:15 PM, Eloy Paris wrote: [...] Writing the micro-controller code to handle the memory read (and subsequent 1-Wire transfer from micro-controller to bus master) is easy but I am struggling with the OWFS side of things. I could do something like: $ echo 0 /owfs/xx.n/generic/offset $ echo 128 /owfs/xx.n/generic/length $ cat /owfs/xx.n/generic/memory file.bin However, the OWFS BAE support does not have such an interface, i.e. offset and length directory entries; instead, it seems like it expects offset and length parameters as part of the same read operation, so I am confused :-( Puzzled, Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers -- Pascal www.brain4home.eu to subscribe, send a mail to list-subscr...@brain4home.eu -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
Hi Paul, On 07/07/2011 04:26 PM, Paul Alfille wrote: owq is filled by the read command -- i.e. a filesystem read has size and offset as part of the call. Buffer (for the result) is allocated as well. You don't need to worry about how it's filled in, that comes from the OW_read routines (or OW_write). Ah, yes -- this all makes sense. What was confusing me is that I was looking at all this from a command line point of view. You just don't have a read offset and read size when you use cat to read files ;-) I think the above simple explanation is what made me understand what I was dealing with. Those MACROs: OWQ_buffer, OWQ_size, OWQ_offset call the relevant fields from the structure. Yup. I saw the macros in the corresponding header file. I just was not seeing how the owq structures was supposed to get filled when accessing the file from the command line with an utility like cat. The answer is that it doesn't get filled if you use cat. OW_r_mem, in this case, takes those parameters, and knows what 1-wire commands to send to the device to read the memory. Yup; once you explained everything made sense and all pieces fell into place. The rest is just microcontroller code. In the case of BAE, as I recall, there was the added complexity that the chip and memory read is designed as a long linear address space, while the various fields exposed (EEPROM, user mem, variables) are subsets of that space. We remap the start to correspond to the correct chip address space while keeping the bounds checking correct. I have not reached this point yet with my own device but I think that at some point I will. All this is a thin object-oriented layer written in C. owq stands for one-wire-query, and hold the field's size, offset, data buffer, and all the flags and parsed path information. This way we use a similar data structure independent of the data type (float, integer, boolean, data, memory). Yup; thanks for the explanation. It's easier to understand things now after your explanation. Cheers, Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
Hi Paul, On 07/07/2011 04:41 PM, Paul Alfille wrote: I think I'm starting to see your question. You are wondering how to read and write in the middle of a file (memory buffer). Yes, with the added complexity (for me to understand how things work) that I was trying to do this from the command line, with regular Unix utilities like cat . The command line is an awkward place to do partial file operations. That's what I learned today. Perhaps dd, which allows to specify an offset and a read size, would work, but definitely not cat. From a C program, you can use lseek and read From the owshell routines, there is owread --size=nnn --start=nnn --hex owwrite --start=nnn --hex Once you mentioned these methods it was clear to me how things work. Thanks! [...] Can you explain your requirements more extensively? It was actually pretty simple -- I wanted to be able to read an arbitrary number of microcontroller memory starting at a specified offset. After your explanations I am accomplishing what I need like this (for example): shell$ owread --hex --size=16 --start=0 /99.010203040506/generic/memory Cheers, Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
Hi Pascal, On 07/07/2011 05:37 PM, Pascal Baerten wrote: you may want to look at the 'writebyte' I've utilized to test some features sometime ago. the code is now commented out but still present. The principle is to consider the writebyte property as a word value (two bytes) the hi-byte is parsed as the offset and the lo-byte is parsed as the value to write. for example echo 513 /owfs/xx.n/writebyte will write the value 1 (513 |0Xff) to offset 2 (5138) for more advanced use of a generic memory, you may create a byte array of 256 elements wich will be individually addressable in both read and write function. (see aggregate examples) Thanks for the info. The writebyte seems to be a bit limited in that it only allows to write 1 byte at a time, but thanks for the idea. I'll take a look at aggregate examples; I see that there are some in ow_bae.c. Cheers! Eloy Paris.- -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Owfs-developers mailing list Owfs-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/owfs-developers
Re: [Owfs-developers] Understanding BAE's read access to generic/memory
Hi Paul,
On 07/07/2011 10:55 PM, Eloy Paris wrote:
[...]
After your explanations I am accomplishing what I need like this (for
example):
shell$ owread --hex --size=16 --start=0 /99.010203040506/generic/memory
This had me scratching my head -- I was reading from microcontroller
memory, using owshell's owread, the following 7 bytes:
0x63, 0x00, 0x4C, 0x08, 0x6C, 0x08, and 0xAC
However, the output from owread --hex was:
63004C086C08FFAC
There were obviously some mysterious bytes in there, but after realizing
that the last byte has its most significant bit set, and realizing that
the extra bytes are 0xff, it was obvious that there was an unintended
integer conversion from a small size to a larger size (signed char to
integer).
Below you'll find the patch that fixed the problem for me. We could
change the type of the argument of the function (from char to unsigned
char) instead of using a cast, but that probably would require changes
to other places inside ow_server.c (function prototype, add casts to
arguments when the function is called, or change the type of arguments
being passed to the function) so I thought casting in one place would be
easier.
Cheers,
Eloy Paris.-
---
--- module/owshell/src/c/ow_server.c.orig 2011-07-07 23:23:53.568305295
-0400
+++ module/owshell/src/c/ow_server.c2011-07-07 23:30:03.824315008 -0400
@@ -367,6 +367,6 @@
{
int i ;
for ( i=0 ; ilength ; ++i ) {
- printf(%.2X,buffer[i]);
+ printf(%.2X, (unsigned char) buffer[i]);
}
}
--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
Owfs-developers mailing list
Owfs-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/owfs-developers
