Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Sunday 16 August 2009 15:55:31 Grant Edwards wrote: On 2009-08-15, Hendrik van Rooyen hend...@microcorp.co.za wrote: I am still confused about pyserial and serial - I found serial in my distribution library, (on the SuSe machine, not on the 2.5 in Slackware) but I had to download pyserial. That's very interesting. Is the pre-existing serial a version of pyserial that the packager had pre-installed or is it something else? I didn't know any distributions shipped I am not too sure now - when I looked in the packages directory over the weekend it was there. But I am really not sure how it got there - I cannot recall downloading it myself, but that means nothing as I have been using this machine for a while now, so it is possible that I did it some time ago and forgotten about it, without ever using it. Possible, but unlikely. I would like to think that if I downloaded it, I would have tried it, and then I would have remembered. Sorry to be so vague. Does this help? h...@linuxbox:~ python Python 2.5.1 (r251:54863, Dec 6 2008, 10:49:39) [GCC 4.2.1 (SUSE Linux)] on linux2 Type help, copyright, credits or license for more information. import serial dir(serial) ['EIGHTBITS', 'FCNTL', 'FIVEBITS', 'FileLike', 'PARITY_EVEN', 'PARITY_NAMES', 'PARITY_NONE', 'PARITY_ODD', 'SEVENBITS', 'SIXBITS', 'STOPBITS_ONE', 'STOPBITS_TWO', 'Serial', 'SerialBase', 'SerialException', 'SerialTimeoutException', 'TERMIOS', 'TIOCINQ', 'TIOCMBIC', 'TIOCMBIS', 'TIOCMGET', 'TIOCMSET', 'TIOCM_CAR', 'TIOCM_CD', 'TIOCM_CTS', 'TIOCM_DSR', 'TIOCM_DTR', 'TIOCM_DTR_str', 'TIOCM_RI', 'TIOCM_RNG', 'TIOCM_RTS', 'TIOCM_RTS_str', 'TIOCM_zero_str', 'VERSION', 'XOFF', 'XON', '__builtins__', '__doc__', '__file__', '__name__', '__path__', 'device', 'errno', 'fcntl', 'os', 'plat', 'portNotOpenError', 'select', 'serialposix', 'serialutil', 'string', 'struct', 'sys', 'termios', 'writeTimeoutError'] serial.VERSION '1.27' - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
Hendrik van Rooyen wrote: On Saturday 15 August 2009 14:40:35 Michael Ströder wrote: Hendrik van Rooyen wrote: In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, and nobody contradicted me. Absolutely false. Despite all the good comments here by other skilled people I'd recommend to determine whether the transmission line to the devices accessed support full duplex. All standard PC serial ports are full-duplex devices. Here's a program I wrote which uses pyserial to drive Baudot teletypes as full-duplex devices. https://sourceforge.net/projects/baudotrss/ This uses an input thread and an output thread. It reads RSS feeds and prints them on antique Teletype machines. (Reuters RSS feeds produce a classic news ticker. Twitter RSS feeds work but look silly when hammered out on yellow paper at 45.45 baud.) You raise a good point, that is probably not well known amongst the youngsters here, as simple serial multidropping has gone out of fashion. Actually, no. Dynamixel servos as used on the latest Bioloid robots are multidrop serial RS-485. But outside the embedded world, nobody uses that stuff any more. (Embedded is going Ethernet; it's overkill but works fine and is now cheap.) John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port? - conclusions
On Sunday 16 August 2009 08:20:34 John Nagle wrote: Hendrik van Rooyen wrote: On Saturday 15 August 2009 14:40:35 Michael Ströder wrote: Hendrik van Rooyen wrote: In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, and nobody contradicted me. Absolutely false. No its true, if you open the serial port with the standard open() instead of os.open(). And it is also true that I was not contradicted in the past. :-) Despite all the good comments here by other skilled people I'd recommend to determine whether the transmission line to the devices accessed support full duplex. All standard PC serial ports are full-duplex devices. I know this, and I started the thread because they would not work full duplex for me. 8 pyserial baudot program link --- You raise a good point, that is probably not well known amongst the youngsters here, as simple serial multidropping has gone out of fashion. Actually, no. Dynamixel servos as used on the latest Bioloid robots are multidrop serial RS-485. But outside the embedded world, nobody uses that stuff any more. (Embedded is going Ethernet; it's overkill but works fine and is now cheap.) Exactly - it is no longer part of mainstream fashion. We also still use RS-485 because it is cheaper, and we have better control over timing. I enclose two naive test implementations. To run them you will need a loopback connector. One can be made by shorting pin 2 and 3 together on the standard DB9. The results are illuminating - on my development machine, (dual 64 bit, some gigs), the last lines look like this: test.py: The quick brown fox jumps over the lazy dog 1023 That took 3.91238284111 seconds - 3.82068636827 Millisecs per record, 261.733077152 recs per sec h...@linuxbox:~/dos/JOBS/sms/lib test1.py: The quick brown fox jumps over the lazy dog 1023 That took 3.90402388573 seconds - 3.81252332591 Millisecs per record, 262.293477185 recs per sec h...@linuxbox:~/dos/JOBS/sms/lib Almost no difference between the two implementations. This is basically because there is enough processing power to keep the link running at full speed in both instances. On the eBox, (486, 400MHz, 128Mb, no FP), the difference is startling: test.py: The quick brown fox jumps over the lazy dog 1023 That took 69.2863521576 seconds - 67.6624532789 Millisecs per record, 14.7792453797 recs per sec r...@ebox:/home/user/sms/lib# About eighteen times slower than the development machine. test1.py: The quick brown fox jumps over the lazy dog 1023 That took 10.391780138 seconds - 10.148222791 Millisecs per record, 98.5394211964 recs per sec r...@ebox:/home/user/sms/lib# Less than three times slower than the development machine. The little processor + Slackware + python cannot keep the link busy. Python, as a character handler, well let us say that it is suboptimal, because saying that something sucks, sucks. An almost seven times ratio between the implementations is not to be sneezed at. So the conclusions I have come to are the following: 1) Thou shalt not use ordinary python files for serial ports, on pain of death. 2) Thou shalt strive mightily to minimize python calls, doing all in thy power to move away from character input to string input. 3) Thou shalt expend real treasure for real processing power as there is no such thing as a free lunch. I would like to thank everybody who took the trouble to respond to teach me the error of my ways. - Hendrik test1.py Description: application/python test.py Description: application/python -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-15, Hendrik van Rooyen hend...@microcorp.co.za wrote: On Saturday 15 August 2009 16:25:03 Grant Edwards wrote: Are you using python file operations open/read/write or OS file-descriptor operations os.open/os.read/os.write? The former - that seems to be the source of my trouble. I have now written a little test that uses serial.Serial and it works a treat. Good to hear. I am still confused about pyserial and serial - I found serial in my distribution library, (on the SuSe machine, not on the 2.5 in Slackware) but I had to download pyserial. That's very interesting. Is the pre-existing serial a version of pyserial that the packager had pre-installed or is it something else? I didn't know any distributions shipped Python with pyserial installed. In either case, serial isn't something that ships with the standard Python library. I see that you were the the original author. Thank you for letting this stuff loose in the wild. My pleasure. -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 15:58:37 exar...@twistedmatrix.com wrote: One strategy you might employ to get rid of the busy looping is to use Twisted and its serial port support. This also addresses the full- duplex issue you've raised. I know - vaguely - about twisted and I have been dancing around the fire, not really ready to put the time in to understand it properly. Looks like the time has come though - my weekend is really going to hell. I am now going to make a loopback connector and start playing. Thanks to everybody for the feedback. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 16:19:04 Grant Edwards wrote:
On 2009-08-14, Hendrik van Rooyen hend...@microcorp.co.za wrote:
In the meantime I have had another idea which I have also not tried yet,
namely to do independent opens for reading and writing, to give me two
file instances instead of one, and to try with that. I have no idea if
it would make any difference, or even work at all.
That should work (and shouldn't make any difference)
My normal stuff works, but I do not like it as it is
essentially busy looping with short sleeps in between. In the
eBox, it uses most of the processor just to move a few bytes
of I/O in and out between the serial port and the TCP/IP, and
struggles to do that better than five times a second, while
the message time on the 115200 baud port is only about 2
milliseconds.
What platform are you using? I suppose it's possible that
there's something broken in the serial driver for that
particular hardware.
Your experience seems to be exactly the opposite to mine - you are saying it
should just work and I am seeing half duplex functionality.
I have seen this on my development machine which is a dual processor of some
gigs running SuSe Linux 10.3, as well as on the other end of a the scale -
the eBox (a 400MHz 486 without floating point with 128 Mb of memory) running
Slackware.
Maybe it is in the way I set the port up, because that is the common thing.
What I do is this:
reterror = os.system('stty -F /dev/ttyS0 sane 115200 cread clocal raw -echo')
It does not seem to make a difference if I do this before or after opening the
port.
Any comments from a Linux Guru?
- Hendrik
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 16:03:22 Diez B. Roggisch wrote: You should *really* just use pyserial. No hassle, instant satisfaction. :-) I have downloaded and had a quick look, and I see it is based on the standard library's serial.Serial class - another battery that I have not used before. And I see that serial.Serial looks like it uses os. calls, which is one of the things Greg mentioned. Curioser and Curioser. There was one thing I saw in a quick read of pyserial that I did not like as I cannot understand why it is done - if a timeout is set to less than a tenth of a second, then it is changed to be a tenth. - In a polling protocol that will limit you to poll only ten terminals a second, or less, and is a very long time if a message takes only a couple of millis to send. I am getting there - this time around I want to kill this problem dead because I seem to keep doing something wrong somewhere and I want to understand what it is and stop doing it. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 16:19:36 Grant Edwards wrote: On 2009-08-14, exar...@twistedmatrix.com exar...@twistedmatrix.com wrote: One strategy you might employ to get rid of the busy looping is to use Twisted and its serial port support. This also addresses the full- duplex issue you've raised. There are no such full-dulex issues. I will put an example together as soon as I have finished reading and answering the mail - maybe I am crazy and chasing angels. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 16:28:26 Grant Edwards wrote: On 2009-08-14, greg g...@cosc.canterbury.ac.nz wrote: Hendrik van Rooyen wrote: 8--- Doh! It didn't even occur to me that somebody would use python file objects for serial ports, and I completely overlooked the fact that the OP was doing that. In short: don't do that -- it just messes things up *grin* All right that makes me feel better - you were so adamant that there is no problem that I was starting to doubt my sanity. - So I hereby cancel the promise I have just made to put an example together. - It is no longer needed. Do not use Python file objects. Use the underlying file descriptors: os.open(), os.read(), os.write(). That will almost certainly solve your problems. If you want examples of os.x() usage, below is the PosixSerial.py module that I use for Linux-only applications. For cross-platform work, use pyserial (whose Posix support is based on the code below). 8 -PosixSerial.py Thanks that looks, on first inspection, similar to the serialposix.py module in the stdlib, but less cluttered. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Saturday 15 August 2009 04:03:42 Terry Reedy wrote: greg wrote: You can't read and write with the same stdio file object at the same time. Odd things tend to happen if you try. I believe the C standard specifies that the behavior of mixed reads and writes is undefined without intervening seek and/or flush, even if the seek is ignored (as it is on some Unix systems). With two threads, the timing is undetermined. For a serial port, flush on write makes some sense, but seek is complete nonsense because it is undefined, and besides- the point you try to seek to may never come around. So the message I am getting loud and clear is that the basic thing I am doing wrong is to use the ordinary python open() instead of os.open(). As for the timing in two threads - Yes you are right, but there is not a lot one can do about it - The right solution depends to a large extent on what you are doing - for instance, if you are writing a polling protocol (such as Burroughs poll-select, or Uniscope), then you want a loop that transmits something, and waits for an answer or time out. This is essentially half duplex, and in a high level language the natural structure to write this is in one thread. On the other hand, if you are writing a sliding window type protocol that is capable of pouring stuff into a link asynchronously from both ends, then the natural way to do it is to use two threads - one to handle incoming stuff, and the other to squirt out the data that must go out. If, as is true in my case, the source of outgoing data and the sink for incoming data is a TCP/IP socket, then one can accomplish this with blocking I/O quite efficiently, provided you have a third thread looking after overall timing Issues. For such a case, the timing is essentially determined by the flow of the data (provided of course that you can keep up with the link speed). When one introduces another variable into the equation, namely the requirement to do a transmission at least every n milliseconds, (a feel-good keepalive) then you need a time out on the sources, so that you can either do a transmission or raise an alarm because a reporting period was missed. So then you are back at a loop waiting for input or timeout, and doing a transmission afterwards. Only now there are two of them, facing in opposite directions. I think this sort of thing is better written at a lower level where one has access to the interrupts from the ports, as well as a timer interrupt to handle timing and timeout issues. But that is a lot of work, so I make do with python. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
Terry Reedy wrote: I believe the C standard specifies that the behavior of mixed reads and writes is undefined without intervening seek and/or flush, even if the seek is ignored (as it is on some Unix systems). With two threads, the timing is undetermined. It's also possible that the stdio object is being locked while one of the threads is using it, which would also account for the observed half-duplex behaviour. Another good reason to steer clear of file objects! -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
Hendrik van Rooyen wrote: In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, and nobody contradicted me. Despite all the good comments here by other skilled people I'd recommend to determine whether the transmission line to the devices accessed support full duplex. My knowledge is a bit rusty on this topic. But I vaguely remember having to deal with symmetric two-wire connections (RS-485) which were definitely limited to half-duplex by the wire. So the PC hardware was a normal serial port with the usual UART hardware device but the transmission protocols were limited to half-duplex. Ciao, Michael. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-15, Hendrik van Rooyen hend...@microcorp.co.za wrote: On Friday 14 August 2009 16:19:04 Grant Edwards wrote: What platform are you using? I suppose it's possible that there's something broken in the serial driver for that particular hardware. Your experience seems to be exactly the opposite to mine - you are saying it should just work and I am seeing half duplex functionality. If you're using Python's normal open()/read()/write() calls, then that introduces all sorts of issues. I always use direct OS calls os.open(), os.read(), os.write(). Any comments from a Linux Guru? Are you using python file operations open/read/write or OS file-descriptor operations os.open/os.read/os.write? -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-15, Hendrik van Rooyen hend...@microcorp.co.za wrote: On Friday 14 August 2009 16:03:22 Diez B. Roggisch wrote: You should *really* just use pyserial. No hassle, instant satisfaction. :-) I have downloaded and had a quick look, and I see it is based on the standard library's serial.Serial class - another battery that I have not used before. There is no serial module in the standard library. The serial module is provided by pyserial. And I see that serial.Serial looks like it uses os. calls, which is one of the things Greg mentioned. Indeed. You shouldn't try to use normal open/read/write calls with serial ports. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-15, Hendrik van Rooyen hend...@microcorp.co.za wrote: 8 -PosixSerial.py Thanks that looks, on first inspection, similar to the serialposix.py module in the stdlib, but less cluttered. pyserial is a bit more complex because it is cross-platform and supports Windows, as well as RFC2272 telnet backends (and OS X?). The Posix support in pyserial is based on an earlier version of PosixSerial. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Saturday 15 August 2009 14:40:35 Michael Ströder wrote: Hendrik van Rooyen wrote: In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, and nobody contradicted me. Despite all the good comments here by other skilled people I'd recommend to determine whether the transmission line to the devices accessed support full duplex. My knowledge is a bit rusty on this topic. But I vaguely remember having to deal with symmetric two-wire connections (RS-485) which were definitely limited to half-duplex by the wire. So the PC hardware was a normal serial port with the usual UART hardware device but the transmission protocols were limited to half-duplex. You raise a good point, that is probably not well known amongst the youngsters here, as simple serial multidropping has gone out of fashion. There is nothing wrong with your memory as far as RS-485 goes - you have to turn the line around, same as for *shudder* Burroughs TDI (Two Wire Direct Interface). Otherwise, if two or more parties talk at once you have cacophony. An RS-422 link is to some extent worse, as it is capable of full duplex, but the slaves cannot hear each other, so they have to listen and play very nicely with the master. This instance Is not one of those, thank heaven - I am on both sides of the link - once in the eBox in python, and on the other side there is just one Dallas chip - a fast (30 Mhz single cycle) 8051 lookalike that I programmed in assembler. It is a thing that does discrete I/O that we have made for a customer. The link in between is just RS-232 receive and transmit without hardware flow control or anything fancy. This is why I was so certain that there was something wrong in my python part, because I could use the second port on the Dallas to do monitoring, by spewing stuff out into Hyper Terminal. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Saturday 15 August 2009 16:25:03 Grant Edwards wrote: Are you using python file operations open/read/write or OS file-descriptor operations os.open/os.read/os.write? The former - that seems to be the source of my trouble. I have now written a little test that uses serial.Serial and it works a treat. I am still confused about pyserial and serial - I found serial in my distribution library, (on the SuSe machine, not on the 2.5 in Slackware) but I had to download pyserial. I see that you were the the original author. Thank you for letting this stuff loose in the wild. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Is it possible to use python to get True Full Duplex on a Serial port?
In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, and nobody contradicted me. I am running into the self same issue again. What I normally do is to open the port like this: port = open(/dev/ttyS0,r+b,0) and then I unblock it with: def unblock(f): Given file 'f', sets its unblock flag to true. fcntl.fcntl(f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) Then I can write a loop that uses a try-except to see if there are characters available, and that examines a queue to see if there is something to transmit, to give the appearance of full duplex functionality. What I would really like is to have two threads - one that does blocking input waiting for a character, and one that examines an output queue and transmits the stuff it finds. When I try to do this, it does not seem to work - as far as I can see, it is as if the underlying implementation is somehow single threaded - if it is waiting for a received character, it waits until something comes in before it will transmit anything. So if you are talking to a device that does not respond, the whole thing freezes up waiting for a character that never comes, and nothing is transmitted either, despite the call to port.write(somestring). The write blocks, and everything stops, waiting for the receipt to finish. Is there a way to get full duplex, so that the transmit and receive are independent of each other? Or are we stuck with a disk-like model that forces a sequence on reads and writes? - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
Hendrik van Rooyen schrieb: In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, and nobody contradicted me. I am running into the self same issue again. What I normally do is to open the port like this: port = open(/dev/ttyS0,r+b,0) How about using pyserial? With that, I never had any problems accessing the the serial ports, and AFAIK no duplex-problems as well. And I seriously doubt that these are a python-related problem - python only has a very thin, direct layer above the posix-calls, and doesn't do anything that would explain your observed behavior. The GIL is not the issue here either - it won't interfer with blocking IO. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
Hendrik van Rooyen wrote: port = open(/dev/ttyS0,r+b,0) What I would really like is to have two threads - one that does blocking input waiting for a character, and one that examines an output queue and transmits the stuff it finds. You can't read and write with the same stdio file object at the same time. Odd things tend to happen if you try. You need to open *two* file objects, one for reading and one for writing: fr = open(/dev/ttyS0,rb,0) fw = open(/dev/ttyS0,wb,0) and give fr to the reading thread and fw to the writing thread. You could also try avoiding file objects altogether and use the raw system calls in the os module. Since you're not using any buffering, there's little reason to use the stdio layer. If you do that, you should be able to use the same file descriptor for reading and writing without any trouble. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 12:54:32 Diez B. Roggisch wrote: How about using pyserial? With that, I never had any problems accessing the the serial ports, and AFAIK no duplex-problems as well. And I seriously doubt that these are a python-related problem - python only has a very thin, direct layer above the posix-calls, and doesn't do anything that would explain your observed behavior. The GIL is not the issue here either - it won't interfer with blocking IO. I will have a look at pyserial - have never used it before. I agree that it is probably not a Python issue, and that the GIL is irelevant - I was hoping that someone had already travelled the road and could give me a signpost. In the meantime I have had another idea which I have also not tried yet, namely to do independent opens for reading and writing, to give me two file instances instead of one, and to try with that. I have no idea if it would make any difference, or even work at all. My normal stuff works, but I do not like it as it is essentially busy looping with short sleeps in between. In the eBox, it uses most of the processor just to move a few bytes of I/O in and out between the serial port and the TCP/IP, and struggles to do that better than five times a second, while the message time on the 115200 baud port is only about 2 milliseconds. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On Friday 14 August 2009 14:13:46 greg wrote: You can't read and write with the same stdio file object at the same time. Odd things tend to happen if you try. You need to open *two* file objects, one for reading and one for writing: fr = open(/dev/ttyS0,rb,0) fw = open(/dev/ttyS0,wb,0) and give fr to the reading thread and fw to the writing thread. Does this actually work without somehow falling foul of the fact that the /dev/ttyS0 is only one thing? - I know that there is no physical reason for not being able to go in and out at the same time - in my embedded stuff I do that routinely - but that is all assembler running on bare metal so it is under my own control. You could also try avoiding file objects altogether and use the raw system calls in the os module. Since you're not using any buffering, there's little reason to use the stdio layer. If you do that, you should be able to use the same file descriptor for reading and writing without any trouble. Thanks for this - I now have my weekend cut out for me... - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 01:38 pm, hend...@microcorp.co.za wrote: On Friday 14 August 2009 12:54:32 Diez B. Roggisch wrote: How about using pyserial? With that, I never had any problems accessing the the serial ports, and AFAIK no duplex-problems as well. And I seriously doubt that these are a python-related problem - python only has a very thin, direct layer above the posix-calls, and doesn't do anything that would explain your observed behavior. The GIL is not the issue here either - it won't interfer with blocking IO. I will have a look at pyserial - have never used it before. I agree that it is probably not a Python issue, and that the GIL is irelevant - I was hoping that someone had already travelled the road and could give me a signpost. In the meantime I have had another idea which I have also not tried yet, namely to do independent opens for reading and writing, to give me two file instances instead of one, and to try with that. I have no idea if it would make any difference, or even work at all. My normal stuff works, but I do not like it as it is essentially busy looping with short sleeps in between. In the eBox, it uses most of the processor just to move a few bytes of I/O in and out between the serial port and the TCP/IP, and struggles to do that better than five times a second, while the message time on the 115200 baud port is only about 2 milliseconds. One strategy you might employ to get rid of the busy looping is to use Twisted and its serial port support. This also addresses the full- duplex issue you've raised. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
Hendrik van Rooyen schrieb: On Friday 14 August 2009 14:13:46 greg wrote: You can't read and write with the same stdio file object at the same time. Odd things tend to happen if you try. You need to open *two* file objects, one for reading and one for writing: fr = open(/dev/ttyS0,rb,0) fw = open(/dev/ttyS0,wb,0) and give fr to the reading thread and fw to the writing thread. Does this actually work without somehow falling foul of the fact that the /dev/ttyS0 is only one thing? - I know that there is no physical reason for not being able to go in and out at the same time - in my embedded stuff I do that routinely - but that is all assembler running on bare metal so it is under my own control. You could also try avoiding file objects altogether and use the raw system calls in the os module. Since you're not using any buffering, there's little reason to use the stdio layer. If you do that, you should be able to use the same file descriptor for reading and writing without any trouble. Thanks for this - I now have my weekend cut out for me... You should *really* just use pyserial. No hassle, instant satisfaction. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-14, Hendrik van Rooyen hend...@microcorp.co.za wrote: In the past, on this group, I have made statements that said that on Linux, the serial port handling somehow does not allow transmitting and receiving at the same time, That's not true. Linux/Unix does and always has supported full-duplex communications on serial ports. and nobody contradicted me. Um, sorry, I guess. What I would really like is to have two threads - one that does blocking input waiting for a character, and one that examines an output queue and transmits the stuff it finds. That's the traditional way of doing full-duplex serial IO on Unix back in the day before the select/poll system calls were available. When I try to do this, it does not seem to work - as far as I can see, it is as if the underlying implementation is somehow single threaded - if it is waiting for a received character, it waits until something comes in before it will transmit anything. Nope. I'll try to dig up an example, but that approach has always worked for me. So if you are talking to a device that does not respond, the whole thing freezes up waiting for a character that never comes, and nothing is transmitted either, despite the call to port.write(somestring). The write blocks, and everything stops, waiting for the receipt to finish. I've never observed that behavior. Is there a way to get full duplex, so that the transmit and receive are independent of each other? That's the way serial ports work on Unix. Or are we stuck with a disk-like model that forces a sequence on reads and writes? No. -- Grant Edwards grante Yow! Like I always say at -- nothing can beat visi.comthe BRATWURST here in DUSSELDORF!! -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-14, Hendrik van Rooyen hend...@microcorp.co.za wrote: In the meantime I have had another idea which I have also not tried yet, namely to do independent opens for reading and writing, to give me two file instances instead of one, and to try with that. I have no idea if it would make any difference, or even work at all. That should work (and shouldn't make any difference) My normal stuff works, but I do not like it as it is essentially busy looping with short sleeps in between. In the eBox, it uses most of the processor just to move a few bytes of I/O in and out between the serial port and the TCP/IP, and struggles to do that better than five times a second, while the message time on the 115200 baud port is only about 2 milliseconds. What platform are you using? I suppose it's possible that there's something broken in the serial driver for that particular hardware. -- Grant Edwards grante Yow! I feel ... JUGULAR ... at visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-14, exar...@twistedmatrix.com exar...@twistedmatrix.com wrote: One strategy you might employ to get rid of the busy looping is to use Twisted and its serial port support. This also addresses the full- duplex issue you've raised. There are no such full-dulex issues. -- Grant Edwards grante Yow! The PINK SOCKS were at ORIGINALLY from 1952!! visi.comBut they went to MARS around 1953!! -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 2009-08-14, greg g...@cosc.canterbury.ac.nz wrote:
Hendrik van Rooyen wrote:
port = open(/dev/ttyS0,r+b,0)
What I would really like is to have two threads - one that
does blocking input waiting for a character, and one that
examines an output queue and transmits the stuff it finds.
You can't read and write with the same stdio file object at
the same time. Odd things tend to happen if you try.
You need to open *two* file objects, one for reading and one
for writing:
fr = open(/dev/ttyS0,rb,0)
fw = open(/dev/ttyS0,wb,0)
Doh! It didn't even occur to me that somebody would use python
file objects for serial ports, and I completely overlooked
the fact that the OP was doing that.
In short: don't do that -- it just messes things up.
and give fr to the reading thread and fw to the writing
thread.
You could also try avoiding file objects altogether and use
the raw system calls in the os module.
That's definitely the way you should do serial port I/O.
Since you're not using any buffering, there's little reason to
use the stdio layer. If you do that, you should be able to use
the same file descriptor for reading and writing without any
trouble.
Do not use Python file objects. Use the underlying file
descriptors: os.open(), os.read(), os.write(). That will
almost certainly solve your problems.
If you want examples of os.x() usage, below is the
PosixSerial.py module that I use for Linux-only applications.
For cross-platform work, use pyserial (whose Posix support is
based on the code below).
-PosixSerial.py--
# Posix serial port class
# Copyright 2001-2009 Grant B. Edwards gra...@visi.com
# You may use this code in any way you like so long as you
# leave this copyright notice.
# Though you are not required to, it would be nice if you send
# me a copy of bufixes or enhancements and allowed me to
# incorporate and distribute them.
import sys
import fcntl
import os
import struct
import termios
import string
import select
if string.split(sys.version)[0] '2':
TERMIOS = termios
else:
import TERMIOS
# construct dictionaries for baud rate lookups
baudEnumToInt = {}
baudIntToEnum = {}
for rate in
(0,50,75,110,134,150,200,300,600,1200,1800,2400,4800,9600,19200,38400,57600,115200,230400,460800,50,576000,921600,100,1152000,150,200,250,300,350,400):
try:
i = eval('TERMIOS.B'+str(rate))
baudEnumToInt[i]=rate
baudIntToEnum[rate] = i
except:
pass
# Do not know if these are right for anything except Linux
if sys.platform[:5] == 'linux':
TIOCMGET = 0x5415
TIOCMBIS = 0x5416
TIOCMBIC = 0x5417
TIOCMSET = 0x5418
TIOCM_LE = 0x001
TIOCM_DTR = 0x002
TIOCM_RTS = 0x004
TIOCM_ST = 0x008
TIOCM_SR = 0x010
TIOCM_CTS = 0x020
TIOCM_CAR = 0x040
TIOCM_RNG = 0x080
TIOCM_DSR = 0x100
TIOCM_CD = TIOCM_CAR
TIOCM_RI = TIOCM_RNG
TIOCM_OUT1 = 0x2000
TIOCM_OUT2 = 0x4000
TIOCM_zero_str = struct.pack('I',0)
TIOCM_one_str = struct.pack('I',1)
TIOCM_RTS_str = struct.pack('I',TIOCM_RTS)
TIOCM_DTR_str = struct.pack('I',TIOCM_DTR)
portNotOpenError = ValueError('port not open')
class Port:
An object wrapper for Posix serial ports
def __init__(self,path=None,noinit=False):
self.fd = None
if path:
self.open(path,noinit)
def __tcsetattr(self):
termios.tcsetattr(self.fd,TERMIOS.TCSANOW,[self.iflag,self.oflag,self.cflag,self.lflag,self.ispeed,self.ospeed,self.cc])
def __tcgetattr(self):
self.iflag,self.oflag,self.cflag,self.lflag,self.ispeed,self.ospeed,self.cc =
termios.tcgetattr(self.fd)
def open(self,path,noinit):
if self.fd:
self.close()
self.path = path
self.fd = os.open(path,os.O_RDWR)
self.__tcgetattr()
if not noinit:
self.iflag = 0
self.oflag = 0
self.lflag = 0
self.__tcsetattr()
def close(self):
if self.fd:
os.close(self.fd)
self.fd = None
def fileno(self):
return self.fd;
def _write(self,data):
if not self.fd: raise portNotOpenError
return os.write(self.fd,data)
def write(self,data):
if not self.fd: raise portNotOpenError
t = len(data)
d = data
while t0:
n = os.write(self.fd,d)
d = d[n:]
t = t - n
def read(self,size=1024,timeout=None):
if not self.fd: raise portNotOpenError
if timeout is None:
return os.read(self.fd,size)
else:
r,w,e = select.select([self.fd],[],[self.fd],timeout)
if r:
return os.read(self.fd,size)
else:
raise timeout
def baud(self,rate=None):
if not
Re: Is it possible to use python to get True Full Duplex on a Serial port?
On 02:19 pm, inva...@invalid wrote: On 2009-08-14, exar...@twistedmatrix.com exar...@twistedmatrix.com wrote: One strategy you might employ to get rid of the busy looping is to use Twisted and its serial port support. This also addresses the full- duplex issue you've raised. There are no such full-dulex issues. There was a perceived issues. Obviously it's possible to do full-duplex with Linux's serial port support (and all the other major platforms too, as far as I know), as long as you know how. :) Twisted makes the how a lot simpler. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to use python to get True Full Duplex on a Serial port?
greg wrote: You can't read and write with the same stdio file object at the same time. Odd things tend to happen if you try. I believe the C standard specifies that the behavior of mixed reads and writes is undefined without intervening seek and/or flush, even if the seek is ignored (as it is on some Unix systems). With two threads, the timing is undetermined. -- http://mail.python.org/mailman/listinfo/python-list
