Re: ptys - I give up
On 7/25/2013 11:13 PM, Christopher Faylor wrote: It has been suggested here a couple of times that it might be a good idea for Cygwin to fill out the block that it sends to subprocesses with information that fools msvcrt programs into thinking that its ptys are really consoles. My suggestion was to use the lpReserved2 CRT block to send fake filetype information, but there's something better. Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. I got as far as implementing the actual console system hooks, but didn't wire them up to anything (like Cygwin ptys). I was about to start writing a test suite for it and got distracted. In principle, something like this makes it possible to also grant job control to groups of regular windows processes. (Since you then know which ones belong to which process groups.) https://github.com/dcolascione/conio I have no idea whether this even builds anymore. I haven't compiled it in months. signature.asc Description: OpenPGP digital signature
Re: ptys - I give up
On Thu, Jul 25, 2013 at 11:44:32PM -0700, Daniel Colascione wrote: On 7/25/2013 11:13 PM, Christopher Faylor wrote: It has been suggested here a couple of times that it might be a good idea for Cygwin to fill out the block that it sends to subprocesses with information that fools msvcrt programs into thinking that its ptys are really consoles. My suggestion was to use the lpReserved2 CRT block to send fake filetype information, but there's something better. Yes, that's why I mentioned your name. Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. This is what I was holding out for. The last time it came up here, people seemed vehemently opposed to the idea since implementations that do this (like the Console app and a couple of libraries floating around) seem to have to poll the console looking for data and that could result in data loss. I have had code (not mine) sitting in my Cygwin directory for years which does this but I couldn't 100% convince myself that the detractors for this idea weren't right. Does your code work around this? cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: ptys - I give up
On 7/26/2013 8:27 AM, Christopher Faylor wrote: On Thu, Jul 25, 2013 at 11:44:32PM -0700, Daniel Colascione wrote: Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. This is what I was holding out for. The last time it came up here, people seemed vehemently opposed to the idea since implementations that do this (like the Console app and a couple of libraries floating around) seem to have to poll the console looking for data and that could result in data loss. I have had code (not mine) sitting in my Cygwin directory for years which does this but I couldn't 100% convince myself that the detractors for this idea weren't right. Does your code work around this? I'm also against screen scrapers. That's why I wrote this library: it doesn't poll. It actually intercepts console APIs and implements console handles as pseudohandles, just like Windows = 7 did, forwarding console API requests to a server. Any console program that works with Windows 7 should work with this library. I'm not aware of any other implementations that use this approach. There's no chance of data loss. signature.asc Description: OpenPGP digital signature
Re: ptys - I give up
On Jul 26 09:21, Daniel Colascione wrote: On 7/26/2013 8:27 AM, Christopher Faylor wrote: On Thu, Jul 25, 2013 at 11:44:32PM -0700, Daniel Colascione wrote: Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. This is what I was holding out for. The last time it came up here, people seemed vehemently opposed to the idea since implementations that do this (like the Console app and a couple of libraries floating around) seem to have to poll the console looking for data and that could result in data loss. I have had code (not mine) sitting in my Cygwin directory for years which does this but I couldn't 100% convince myself that the detractors for this idea weren't right. Does your code work around this? I'm also against screen scrapers. That's why I wrote this library: it doesn't poll. It actually intercepts console APIs and implements console handles as pseudohandles, just like Windows = 7 did, forwarding console API requests to a server. Any console program that works with Windows 7 should work with this library. Out of curiosity, can you expand what you mean here? I see a difference between Windows pre-7 and Windows 7 and later. Before Windows 7, the consoles seemed to be handled entirely by the csrss process. If you started bash directly, there was no other visible process involved. Starting with Windows 7 there's suddenly a conhost process for each console. Just as a side note, Windows 7's new conhost also broke the method to start an invisible console by creating it in a new, invisible WindowStation, and that still hasn't been fixed in Windows 8, despite my bug report during the W7 beta... Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: ptys - I give up
On 7/26/2013 9:35 AM, Corinna Vinschen wrote: On Jul 26 09:21, Daniel Colascione wrote: On 7/26/2013 8:27 AM, Christopher Faylor wrote: On Thu, Jul 25, 2013 at 11:44:32PM -0700, Daniel Colascione wrote: Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. This is what I was holding out for. The last time it came up here, people seemed vehemently opposed to the idea since implementations that do this (like the Console app and a couple of libraries floating around) seem to have to poll the console looking for data and that could result in data loss. I have had code (not mine) sitting in my Cygwin directory for years which does this but I couldn't 100% convince myself that the detractors for this idea weren't right. Does your code work around this? I'm also against screen scrapers. That's why I wrote this library: it doesn't poll. It actually intercepts console APIs and implements console handles as pseudohandles, just like Windows = 7 did, forwarding console API requests to a server. Any console program that works with Windows 7 should work with this library. Out of curiosity, can you expand what you mean here? I see a difference between Windows pre-7 and Windows 7 and later. Conhost exists to solve the problem Raymond Chen described here: http://blogs.msdn.com/b/oldnewthing/archive/2007/12/31/6909007.aspx. Conhost exists so there's something running with lower privileges that can draw console UI --- it's a broker. The actual console-ing still happens entirely in user mode though, with the functions in kernel32 conspiring to provide the illusion that the system has something called a console handle. In Windows 8, the implementation is completely different. There, console handles are real kernel objects. Anyway, that pre-8 Windows used pseudo-handles for console handles is a good thing now. It means that any library that tries to intercept all console functions doesn't need to emulate all possible operations on console handles: instead, it just needs to emulate the operations Windows actually allowed on console handles, which was actually a fairly limited subset of what's allowed for handles in general. Another way of saying it is that it's easy to fake console handles because Windows, too, was faking all along. Before Windows 7, the consoles seemed to be handled entirely by the csrss process. If you started bash directly, there was no other visible process involved. Starting with Windows 7 there's suddenly a conhost process for each console. Sure --- just like you might have an rxvt for each login shell. Now imagine if an X server happened to provide an rxvt implementation in-process and that the pty interface were entirely private to X and implemented by sending special WM hints to X. That's the Windows pre-7 architecture. Just as a side note, Windows 7's new conhost also broke the method to start an invisible console by creating it in a new, invisible WindowStation, and that still hasn't been fixed in Windows 8, despite my bug report during the W7 beta... I have no idea. signature.asc Description: OpenPGP digital signature
Re: ptys - I give up
On Jul 26 09:55, Daniel Colascione wrote: On 7/26/2013 9:35 AM, Corinna Vinschen wrote: On Jul 26 09:21, Daniel Colascione wrote: On 7/26/2013 8:27 AM, Christopher Faylor wrote: On Thu, Jul 25, 2013 at 11:44:32PM -0700, Daniel Colascione wrote: Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. This is what I was holding out for. The last time it came up here, people seemed vehemently opposed to the idea since implementations that do this (like the Console app and a couple of libraries floating around) seem to have to poll the console looking for data and that could result in data loss. I have had code (not mine) sitting in my Cygwin directory for years which does this but I couldn't 100% convince myself that the detractors for this idea weren't right. Does your code work around this? I'm also against screen scrapers. That's why I wrote this library: it doesn't poll. It actually intercepts console APIs and implements console handles as pseudohandles, just like Windows = 7 did, forwarding console API requests to a server. Any console program that works with Windows 7 should work with this library. Out of curiosity, can you expand what you mean here? I see a difference between Windows pre-7 and Windows 7 and later. Conhost exists to solve the problem Raymond Chen described here: http://blogs.msdn.com/b/oldnewthing/archive/2007/12/31/6909007.aspx. Conhost exists so there's something running with lower privileges that can draw console UI --- it's a broker. The actual console-ing still happens entirely in user mode though, with the functions in kernel32 conspiring to provide the illusion that the system has something called a console handle. In Windows 8, the implementation is completely different. There, console handles are real kernel objects. Really? That's entirely new to me. Do you know if there's some description available? What are the implications? Are console handles now handles to something like named pipes as well? Is there a console FS comparable to \Device\NamedPipe\? Anyway, that pre-8 Windows used pseudo-handles for console handles is a good thing now. It means that any library that tries to intercept all console functions doesn't need to emulate all possible operations on console handles: instead, it just needs to emulate the operations Windows actually allowed on console handles, which was actually a fairly limited subset of what's allowed for handles in general. Another way of saying it is that it's easy to fake console handles because Windows, too, was faking all along. But then again, doesn't that mean that your method stops working on W8? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: ptys - I give up
On 7/26/2013 10:10 AM, Corinna Vinschen wrote: On Jul 26 09:55, Daniel Colascione wrote: On 7/26/2013 9:35 AM, Corinna Vinschen wrote: On Jul 26 09:21, Daniel Colascione wrote: On 7/26/2013 8:27 AM, Christopher Faylor wrote: On Thu, Jul 25, 2013 at 11:44:32PM -0700, Daniel Colascione wrote: Ugly, only half-implemented, but better: a hook-based pseudoconsole system for Windows. This is what I was holding out for. The last time it came up here, people seemed vehemently opposed to the idea since implementations that do this (like the Console app and a couple of libraries floating around) seem to have to poll the console looking for data and that could result in data loss. I have had code (not mine) sitting in my Cygwin directory for years which does this but I couldn't 100% convince myself that the detractors for this idea weren't right. Does your code work around this? I'm also against screen scrapers. That's why I wrote this library: it doesn't poll. It actually intercepts console APIs and implements console handles as pseudohandles, just like Windows = 7 did, forwarding console API requests to a server. Any console program that works with Windows 7 should work with this library. Out of curiosity, can you expand what you mean here? I see a difference between Windows pre-7 and Windows 7 and later. Conhost exists to solve the problem Raymond Chen described here: http://blogs.msdn.com/b/oldnewthing/archive/2007/12/31/6909007.aspx. Conhost exists so there's something running with lower privileges that can draw console UI --- it's a broker. The actual console-ing still happens entirely in user mode though, with the functions in kernel32 conspiring to provide the illusion that the system has something called a console handle. In Windows 8, the implementation is completely different. There, console handles are real kernel objects. Really? That's entirely new to me. Do you know if there's some description available? What are the implications? Are console handles now handles to something like named pipes as well? Is there a console FS comparable to \Device\NamedPipe\? I don't know the details, and I'm not aware of any new facilities exposed to applications. I'd have loved a pseudoconsole API, but we didn't get one. You can see the difference in a debugger, though: console pseudohandles *look* strange and don't show up in the process handle table. The difference shouldn't really matter, though: applications shouldn't really care what kind of handles they get as long as system APIs accept them. The problem with writes larger than 8k (I think that was the limit?) to console handles failing seems to have disappeared, though. Anyway, that pre-8 Windows used pseudo-handles for console handles is a good thing now. It means that any library that tries to intercept all console functions doesn't need to emulate all possible operations on console handles: instead, it just needs to emulate the operations Windows actually allowed on console handles, which was actually a fairly limited subset of what's allowed for handles in general. Another way of saying it is that it's easy to fake console handles because Windows, too, was faking all along. But then again, doesn't that mean that your method stops working on W8? Not really --- it'll work fine. The difference means that operations on console handles that used not to work on Windows 7 might work on Windows 8. The public API contract hasn't changed, however, so applications aren't supposed to be doing these things anyway, and as long as application developers care about testing their applications on Windows 7, they won't. If you eventually do start seeing applications that do things with console handles that can't be done with console pseudohandles, well, you can hook more APIs to preserve the console handle illusion. But that time is a long way away, if it ever arrives at all. signature.asc Description: OpenPGP digital signature
Re: ptys - I give up
On Fri, Jul 26, 2013 at 02:13:34AM -0400, Christopher Faylor wrote: It has been suggested here a couple of times that it might be a good idea for Cygwin to fill out the block that it sends to subprocesses with information that fools msvcrt programs into thinking that its ptys are really consoles. Back in 2010, there was a flurry of discussion that I shot down, hoping for a better solution. Since none has materialized, I have concluded that I was wrong to hold hostage a solution that would have made things better for a lot of people. I have done some googling and see that Andy Koppe and Daniel Colascione seemed to have something working (maybe that's where rlwrap comes from?) but I don't either of them submitted a patch. I thought that I remembered seeing an actual code change for Cygwin but, it's late, and I can't find it. Does anyone remember if there was actual code submitted for the Cygwin DLL? If so, a link would be appreciated. Back to my original question: Does anyone remember this. Andy does this ring a bell? cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: ptys - I give up
On Jul 26 10:24, Daniel Colascione wrote: On 7/26/2013 10:10 AM, Corinna Vinschen wrote: On Jul 26 09:55, Daniel Colascione wrote: In Windows 8, the implementation is completely different. There, console handles are real kernel objects. Really? That's entirely new to me. Do you know if there's some description available? What are the implications? Are console handles now handles to something like named pipes as well? Is there a console FS comparable to \Device\NamedPipe\? I don't know the details, and I'm not aware of any new facilities exposed to applications. I'd have loved a pseudoconsole API, but we didn't get one. I see a new device called \Device\ConDrv and \GLOBAL?? now contains symlinks CON - \Device\ConDrv\Console CONIN$ - \Device\ConDrv\CurrentIn CONOUT$ - \Device\ConDrv\CurrentOut You can see the difference in a debugger, though: console pseudohandles *look* strange and don't show up in the process handle table. The difference shouldn't Console handle value % 4 != 0 really matter, though: applications shouldn't really care what kind of handles they get as long as system APIs accept them. The problem with writes larger than 8k (I think that was the limit?) to console handles failing seems to have disappeared, though. Uhm... there was a problem? I don't see a hint on the net, nor on MSDN. The Cygwin DLL uses a maximum buffer size of 16K for console writes. If there's some documented problem, maybe we should reduce the size to 8K. But then again, doesn't that mean that your method stops working on W8? Not really --- it'll work fine. The difference means that operations on console handles that used not to work on Windows 7 might work on Windows 8. The public API contract hasn't changed, however, so applications aren't supposed to be doing these things anyway, and as long as application developers care about testing their applications on Windows 7, they won't. If you eventually do start seeing applications that do things with console handles that can't be done with console pseudohandles, well, you can hook more APIs to preserve the console handle illusion. But that time is a long way away, if it ever arrives at all. I guess you're right. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
