RE: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Josh, Rich, et al., On 1/5/07, Rich Teer [EMAIL PROTECTED] wrote: On Fri, 5 Jan 2007, Christoph Hellwig wrote: Note that lsof doesn't have to do kmem craling. For example on Linux it only uses proper procfs interfaces. As such proper interfaces seem to exist on Solaris as well for use with pfiles it shouldn't be a problem for people knowledgeable in this area to adjust lsof to do the right thin on Solaris aswell. Have we invited Vic Abel Vic's name is spelled 'Abell', two e- I've been called worse. :-) No apology is necessary, Rich. (lsof's author) to join the OpenSolaris community? With the right support framework in place, he'd probably be the best person to make the required changes (in lsof anyway). Better: Sun could hire him! I am very happily retired and not looking for employment. But let me explain why lsof has continued to use /dev/kmem. The foremost reason is that I can test it as a guest on systems where I couldn't (or wouldn't want to) have root permission. Sys group membership is all I need and most administrators are willing to give that with a guest login. (I also have a innate distaste for setuid(root) programs.) There are other reasons, however. On Linux, for example, which was cited as prototypical reason for using the Solaris /proc, lsof isn't fully capable in one important respect, because of a deficiency in the Linux /proc file system. It can't deliver current file position (as found in the file structure), so lsof can't report it, something I have found extremely valuable when using lsof to watch a file's progress -- e.g., growth. A second example is the HP-UX PSTAT extensions for lsof in whose development I played a direct role. Bugs have surfaced in that interface over the several years lsof has been using it and one significant one has been the subject of considerable discussion between me, customers and HP. After over a year of that, I think the bug may have finally been accepted as a bug and may even get fixed. Those two examples show that API's have their own limitations, primarily sluggish response to necessary fixes. At one time I proposed to Sun the construction of an lsof API for Solaris, but that proposal never went anywhere. I made it right after completing the HP PSTAT API, thinking its existence might be a sufficient prod. Now that I have struggled getting HP to fix a simple PSTAT bug for a year, my enchantment with lsof APIs has lost some of its edge. Having said all that, I would be interested in joining in some sort of effort to make lsof a part of OpenSolaris. Regards, Vic Abell, lsof author ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Peter Tribble wrote: On 1/1/07, wb [EMAIL PROTECTED] wrote: I have a /tmp FS for swap, and a really big file crout* inside. The /tmp was 95% up. I decided to remove the crout file. The problem, is the /tmp is not decreasing, but still growing. How could I make it decrease? Find and kill the process that's writing to that file. Somehow I am wondering why Solaris doesn't ship lsof in /usr/sbin/ ... is this just noone had time yet or something else ? No; it's something that we won't ship, ever because of the nature how lsof trawls for its events. While we've hardened the system against panics induced by lsof and other kernel browsers, a piece of code which reads data structures without respect for locking and follows pointers which may no longer be there can report bogus data as well as reveal information that should not be reported. That's why we've elected, in the past, to mimic some of lsof's functionality in other tools, notably pfiles. I understand some of the information is still wanting so perhaps we should improve there. Casper ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Roland Mainz [EMAIL PROTECTED] wrote: Somehow I am wondering why Solaris doesn't ship lsof in /usr/sbin/ ... is this just noone had time yet or something else ? Wasn't there a blog where someone explained why lsof is a perfornance pig and should be avoided in favor of other tools? Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Joerg Schilling wrote: Roland Mainz [EMAIL PROTECTED] wrote: Somehow I am wondering why Solaris doesn't ship lsof in /usr/sbin/ ... is this just noone had time yet or something else ? Wasn't there a blog where someone explained why lsof is a perfornance pig and should be avoided in favor of other tools? Jörg I don't know of a performance problem. What I had seen is, that you apparently have to rebuild it for Solaris10 06/06, versions built for older releases don't work correctly under 06/06 anymore. But that's certainly not a reason, why SUNW couldn't ship it. Martin ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On Fri, 5 Jan 2007, Martin Bochnig wrote: [ ... ] What I had seen is, that you apparently have to rebuild it for Solaris10 06/06, versions built for older releases don't work correctly under 06/06 anymore. But that's certainly not a reason, why SUNW couldn't ship it. Well, so what exactly does lsof give that for example something like: #!/bin/sh cd /proc for i in *; do pfiles $i done doesn't do as well ? On the other hand, the request for lsof in Solaris is anything but new, and the reference to pfiles, the mentioning that lsof does nasty digging in the kernel's intestines as a reply is neither, and the arguments that the output format is different, the system load of pfiles is higher, lsof is common operator knowledge, ... - all that isn't new either ... See: 4286979 Solaris should include a utility like lsof lsof isn't perfect, and it doesn't claim to be. It's just neat and helpful. I think Sun has to overcome both not invented here and the instinctive reject via not perfect, therefore not 'good enough' no matter what the users say, before lsof goes into mainstream Solaris. Just my personal opinion there. FrankH. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On Fri, Jan 05, 2007 at 11:59:27AM +, Frank Hofmann wrote: On the other hand, the request for lsof in Solaris is anything but new, and the reference to pfiles, the mentioning that lsof does nasty digging in the kernel's intestines as a reply is neither, and the arguments that the output format is different, the system load of pfiles is higher, lsof is common operator knowledge, ... - all that isn't new either ... Note that lsof doesn't have to do kmem craling. For example on Linux it only uses proper procfs interfaces. As such proper interfaces seem to exist on Solaris as well for use with pfiles it shouldn't be a problem for people knowledgeable in this area to adjust lsof to do the right thin on Solaris aswell. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On Fri, 5 Jan 2007, Christoph Hellwig wrote: On Fri, Jan 05, 2007 at 11:59:27AM +, Frank Hofmann wrote: On the other hand, the request for lsof in Solaris is anything but new, and the reference to pfiles, the mentioning that lsof does nasty digging in the kernel's intestines as a reply is neither, and the arguments that the output format is different, the system load of pfiles is higher, lsof is common operator knowledge, ... - all that isn't new either ... Note that lsof doesn't have to do kmem craling. For example on Linux it only uses proper procfs interfaces. As such proper interfaces seem to exist on Solaris as well for use with pfiles it shouldn't be a problem for people knowledgeable in this area to adjust lsof to do the right thin on Solaris aswell. I know. That's all in the RFE I've referred to anyway. Just noone has done it yet. Maybe because Solaris developers know the Solaristics and don't miss lsof overly much. Again, personally, I can get what I want with pfiles and a bit of scripting; but thinking about e.g. having to write such a script portable between Solaris and Linux, admittedly, that'd be a pain, and that's where the usefulness of lsof comes in. FrankH. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
[EMAIL PROTECTED] writes: No; it's something that we won't ship, ever because of the nature how lsof trawls for its events. That's not true. Though there are folks around who have expressed the never opinion, that does not represent the opinion of all of Solaris development. Our group has discussed integrating lsof off and on many times. We *want* to have it in Solaris, but doing it *right* is substantially difficult. That's why we've elected, in the past, to mimic some of lsof's functionality in other tools, notably pfiles. I understand some of the information is still wanting so perhaps we should improve there. Having utilities with command line interfaces that work sort of like lsof but not quite would be foolish in my opinion. It represents a completely needless barrier to entry for folks new to Solaris, and the sorts of basic questions that lsof answers (who is writing to this file right now? and why is this port open?) are the ones that real administrators frequently need to answer. Pfiles, unfortunately, answers the wrong questions for those users (what things does this process have open?). The right answer, I think, is to introduce proper kernel interfaces that provide the basic information that lsof needs (where those interfaces might be missing), and then rewrite lsof so that it uses only proper interfaces rather than groveling about in internal data structures. Designing those new interfaces and figuring out how to rewrite lsof, though, is not a simple matter, and that's been part of the problem here. We're caught between the seductive simplicity of just integrating the already working lsof (which we can't really do) and rewriting it to conform to expected quality standards (which we don't have the time to do). +1 for a project to integrate the lsof command line interface. Design and implementation to be determined. ;-} -- James Carlson, KISS Network[EMAIL PROTECTED] Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
[EMAIL PROTECTED] writes: No; it's something that we won't ship, ever because of the nature how lsof trawls for its events. That's not true. +1 for a project to integrate the lsof command line interface. Design and implementation to be determined. ;-} Strange as it may seem, this is pretty much the same answer as I gave. (Never in its current form) Casper ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On 1/5/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Peter Tribble wrote: On 1/1/07, wb [EMAIL PROTECTED] wrote: I have a /tmp FS for swap, and a really big file crout* inside. The /tmp was 95% up. I decided to remove the crout file. The problem, is the /tmp is not decreasing, but still growing. How could I make it decrease? Find and kill the process that's writing to that file. Somehow I am wondering why Solaris doesn't ship lsof in /usr/sbin/ ... is this just noone had time yet or something else ? No; it's something that we won't ship, ever because of the nature how lsof trawls for its events. You could send patches to fix lsfo if you do not like it's implementation :) Josh ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On 1/5/07, Frank Hofmann [EMAIL PROTECTED] wrote: On Fri, 5 Jan 2007, Martin Bochnig wrote: I think Sun has to overcome both not invented here and the instinctive reject via not perfect, therefore not 'good enough' no matter what the users say, before lsof goes into mainstream Solaris. If Opensolaris rejects software which is not prefect or troublesome it may be time to kill the /bin/sh-rubbish with a less crappy version (hey, ksh93 would be a cool candidate) Josh ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On 1/5/07, Rich Teer [EMAIL PROTECTED] wrote: On Fri, 5 Jan 2007, Christoph Hellwig wrote: Note that lsof doesn't have to do kmem craling. For example on Linux it only uses proper procfs interfaces. As such proper interfaces seem to exist on Solaris as well for use with pfiles it shouldn't be a problem for people knowledgeable in this area to adjust lsof to do the right thin on Solaris aswell. Have we invited Vic Abel Vic's name is spelled 'Abell', two e- (lsof's author) to join the OpenSolaris community? With the right support framework in place, he'd probably be the best person to make the required changes (in lsof anyway). Better: Sun could hire him! Josh ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Josh Hurst wrote: On 1/5/07, Frank Hofmann [EMAIL PROTECTED] wrote: On Fri, 5 Jan 2007, Martin Bochnig wrote: I think Sun has to overcome both not invented here and the instinctive reject via not perfect, therefore not 'good enough' no matter what the users say, before lsof goes into mainstream Solaris. If Opensolaris rejects software which is not prefect or troublesome it may be time to kill the /bin/sh-rubbish with a less crappy version (hey, ksh93 would be a cool candidate) The problem with lsof is an architectural one, I don't think this is a fair comparison with /bin/sh which isn't architecturally flawed just not to some peoples liking. In the current implementation for Solaris it pokes around in kernel memory and it has no business doing so, it can and will break. An implementation of lsof that used documented and Committed interfaces (heck even ones marked Volatile rather than some form of Private) would be much more likely to be accepted. Just because /bin/sh isn't your shell of choice doesn't make it crappy. Now I'm not saying that /bin/sh shouldn't be replaced, IMO a POSIX compliant /bin/sh would be very nice - however that happens to get implemented be it ksh93 or something else. -- Darren J Moffat ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Josh Hurst writes: On 1/5/07, Frank Hofmann [EMAIL PROTECTED] wrote: On Fri, 5 Jan 2007, Martin Bochnig wrote: I think Sun has to overcome both not invented here and the instinctive reject via not perfect, therefore not 'good enough' no matter what the users say, before lsof goes into mainstream Solaris. If Opensolaris rejects software which is not prefect or troublesome it may be time to kill the /bin/sh-rubbish with a less crappy version (hey, ksh93 would be a cool candidate) There might be a difference between rejecting the integration of _new_ things that don't match current quality (or other) standards, and removing or replacing existing things that retroactively fail to live up to expectations. No? -- James Carlson, KISS Network[EMAIL PROTECTED] Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On Fri, 5 Jan 2007, Josh Hurst wrote: Vic's name is spelled 'Abell', two e- Whoops, my bad! Sorry Vic (I thought something didn't look quite right when I wrote that post)... -- Rich Teer, SCNA, SCSA, SCSECA, OpenSolaris CAB member . * * . * .* . . * . .* President, * . . /\ ( . . * Rite Online Inc. . . / .\ . * . .*. / * \ . . . /* o \ . Voice: +1 (250) 979-1638* '''||''' . URL: http://www.rite-group.com/rich ** ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
No; it's something that we won't ship, ever because of the nature how lsof trawls for its events. You could send patches to fix lsfo if you do not like it's implementation :) You could check the source code and look for my name there. While I understand James' aversion against my comments, I think my comment is well-tempered by my continuous contributions and testing. (I'm generally the person inside Sun who makes sure that it compiles and runs under newer releases, in the old days long before they hit customers) I like lsof and think it is a useful tool; but the current implementation has some serious architectural shortcomings. In the past we have talked about making sufficient information available, but it is always difficult to define a proper set of interfaces and implement them such that they don't step on performance (by looking at objects in use by others). Solaris implements netstat without groping through the kernel; this makes it reliable but also caused some performance issues. Casper ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
On 1/5/07, Darren J Moffat [EMAIL PROTECTED] wrote: Just because /bin/sh isn't your shell of choice doesn't make it crappy. Really? Take 1: /bin/sh /dev/urandom Illegal Instruction (core dumped) Take 2-29: cat /var/adm/messages* | grep -F core.sh | sort Dec 10 11:09:41 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[17713] core dumped: /var/coredumps/core.sh.17713 Dec 10 11:11:14 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[17730] core dumped: /var/coredumps/core.sh.17730 Dec 11 11:19:16 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[19118] core dumped: /var/coredumps/core.sh.19118 Dec 11 11:30:47 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[10798] core dumped: /var/coredumps/core.sh.10798 Dec 11 13:11:06 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[13615] core dumped: /var/coredumps/core.sh.13615 Dec 11 13:13:31 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[15465] core dumped: /var/coredumps/core.sh.15465 Dec 11 16:10:41 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[819] core dumped: /var/coredumps/core.sh.819 Dec 11 16:47:18 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[1181] core dumped: /var/coredumps/core.sh.1181 Dec 11 16:48:16 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[3961] core dumped: /var/coredumps/core.sh.3961 Dec 11 17:11:13 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[15754] core dumped: /var/coredumps/core.sh.15754 Dec 11 17:13:09 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[17434] core dumped: /var/coredumps/core.sh.17434 Dec 13 01:10:19 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[17595] core dumped: /var/coredumps/core.sh.17595 Dec 13 01:11:04 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[19439] core dumped: /var/coredumps/core.sh.19439 Dec 13 08:10:31 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[14917] core dumped: /var/coredumps/core.sh.14917 Dec 17 04:44:47 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[19766] core dumped: /var/coredumps/core.sh.19766 Dec 17 06:44:04 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[51] core dumped: /var/coredumps/core.sh.51 Dec 17 11:17:44 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[10611] core dumped: /var/coredumps/core.sh.10611 Dec 17 11:17:49 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[10637] core dumped: /var/coredumps/core.sh.10637 Dec 17 11:18:14 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[10639] core dumped: /var/coredumps/core.sh.10639 Dec 17 11:18:16 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[10641] core dumped: /var/coredumps/core.sh.10641 Dec 19 06:44:30 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[116] core dumped: /var/coredumps/core.sh.116 Dec 19 06:48:04 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[14186] core dumped: /var/coredumps/core.sh.14186 Dec 19 06:49:53 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[16316] core dumped: /var/coredumps/core.sh.16316 Dec 19 10:07:07 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[19503] core dumped: /var/coredumps/core.sh.19503 Dec 19 10:08:08 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[1131] core dumped: /var/coredumps/core.sh.1131 Dec 19 10:18:09 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[11835] core dumped: /var/coredumps/core.sh.11835 Dec 19 10:18:09 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[11836] core dumped: /var/coredumps/core.sh.11836 Dec 19 10:18:47 fido genunix: [ID 603404 kern.notice] NOTICE: core_log: sh[11841] core dumped: /var/coredumps/core.sh.11841 Popular root of the problem: Memory corruption, memory misalignment, buffer corruption This is Solaris 10 Update 2. Nice production quality, eh? /bin/sh in Solaris 11 is no better Josh ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Shipping lsof with Solaris ? / was: Re: [osol-discuss] problem with /tmp FS still up
Peter Tribble wrote: On 1/1/07, wb [EMAIL PROTECTED] wrote: I have a /tmp FS for swap, and a really big file crout* inside. The /tmp was 95% up. I decided to remove the crout file. The problem, is the /tmp is not decreasing, but still growing. How could I make it decrease? Find and kill the process that's writing to that file. Somehow I am wondering why Solaris doesn't ship lsof in /usr/sbin/ ... is this just noone had time yet or something else ? Bye, Roland -- __ . . __ (o.\ \/ /.o) [EMAIL PROTECTED] \__\/\/__/ MPEG specialist, CJAVASunUnix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
