Happy Birthday to Perl!
Perl 1.0 was released on 18 December, 1987. So today is Perl's 20th Birthday: * http://use.perl.org/article.pl?sid=07/12/17/2046212 * http://use.perl.org/~Shlomi+Fish/journal/35129 At work I'm usually working on PHP code, but yesterday I worked exclusively on Perl code, and may have to tweak it some more today. That seems appropriate considering the date. Regards, Shlomi Fish - Shlomi Fish [EMAIL PROTECTED] Homepage:http://www.shlomifish.org/ I'm not an actor - I just play one on T.V. = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
On Dec 18, 2007 2:36 AM, Oded Arbel [EMAIL PROTECTED] wrote: how much of that virtual memory the process actually tries to use but can't get it all in physical RAM because other processes are also hogging the memory. Does such a thing exist in Linux? To phrase it differently: In the last second (your question inherently calls for a measurement period to be specified), how many pages/megs of memory were touched by this process?
Re: Linux memory monitoring compared to MS-Windows
On Tue, 2007-12-18 at 12:47 +0200, Ilya Konstantinov wrote: On Dec 18, 2007 2:36 AM, Oded Arbel [EMAIL PROTECTED] wrote: how much of that virtual memory the process actually tries to use but can't get it all in physical RAM because other processes are also hogging the memory. Does such a thing exist in Linux? To phrase it differently: In the last second (your question inherently calls for a measurement period to be specified), how many pages/megs of memory were touched by this process? Yes, or at least that's how I understand it. I'm not sure about the time frame as the MSDN documentation does not specify. I would expect it to be much larger then a second - maybe a minute or a few. -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
There's something in your question I don't understand: If a process has 1gb in virtual memory, of which 500mb in physical, then this means that it has 500mb in swap. Or in other words: 500mb that the process wanted in physical but couldn't. (isn't that what you asked to know) Obviously I'm missing something here. Anyway, few notes: - MM starts swapping *EVEN BEFORE* all processes hog the whole physical memory. That's because the kernel wants to leave some memory space for buffers/cache. So you may find yourself swapping idle pages while theoretically some physical memory might be freed. This behavior tunable via /proc/sys/vm/swappiness - free command has an interesting 2nd line, which shows the net memory usage (i.e. without buffers/cache) - The amazing top command can add a swap column; press O, then p and enter. (If 'top' can do it, the per-process swap-vs.-physical data can probably be fetched from /proc/ some way, I don't know how, though) On Tuesday 18 December 2007 02:36, Oded Arbel wrote: Hi List. I heard (but haven't actually seen) that in MS-Windows the system keeps track of some notion of working set, which is supposedly (if I understand correctly) the total size of pages that an application referenced recently - whether these are currently resident or swapped out (see http://msdn2.microsoft.com/en-us/library/ms684891.aspx which is an MSDN article I found on the subject). The way I understand processes normally work (in Linux anyway) is that as long as there is enough memory available the memory manager keeps all pages that an application constantly references in physical RAM, and pages that are not references are swapped out after a while. A good example of such is a long running Java virtual machine process (at least the Sun implementation anyway) that doesn't return unused memory to the operating system letting it being swapped out until its needed again - so I have some jvm process which takes up some 1.5GB of virtual but less then 150MB resident: it was processing a lot of data some time in the past but now its idling. Now (again - according to my understanding) under contention - i.e. when processes need to use more physical memory then what is available - the memory manager keeps swapping stuff in and out of memory in an attempt to satisfy all requests. Under such conditions its might be useful to know - for each process - the amount of physical memory in use, the amount of virtual mapped to the process, but also how much of that virtual memory the process actually tries to use but can't get it all in physical RAM because other processes are also hogging the memory. Does such a thing exist in Linux? Thanks in advance -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
On Dec 18, 2007 2:47 PM, Oren Held [EMAIL PROTECTED] wrote: If a process has 1gb in virtual memory, of which 500mb in physical, then this means that it has 500mb in swap. Or in other words: 500mb that the process wanted in physical but couldn't. (isn't that what you asked to know) BTW, are you sure the Virtual column indicates real+swapped pages? I'm not sure what the terminology really refers to, but something called virtual is likely to refer to all pages, even totally non-committed ones -- i.e. when you malloc 1GB, your address space is expanded but until you touch those pages, the kernel wouldn't bother allotting real nor swap memory for it.
Extra Features of MP3 Drivers
Hi, I'm not sure if it's off-topic or not, so please excuse me if it is: Although MP3 players comply with standards, so they can work with PCs without installing any driver, most of them come with drivers: Some for Windows, some for Linux, some for both, and some without drivers. My question: Are there MP3 players which allow their drivers to access their display and/or buttons? (so programs running on the PC can write directly to the LCD of the MP3, and/or read which buttons are pressed). There are more than 1000 MP3 models in Israel (according to ZAP), so even if only 1% of them support this feature, it gives enough choices... Thanks, -- Eli Marmor [EMAIL PROTECTED] Netmask (El-Mar) Internet Technologies Ltd. __ Tel.: +972-9-766-1020 8 Yad-Harutzim St. Fax.: +972-9-766-1314 P.O.B. 7004 Mobile: +972-50-5237338 Kfar-Saba 44641, Israel = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
Oren Held wrote: There's something in your question I don't understand: If a process has 1gb in virtual memory, of which 500mb in physical, then this means that it has 500mb in swap. No, that is not what it means. Virtual memory amount might be different then physical memory amount due a whole bunch of different reason, swapping being just a single instance and not even the common one. Some other reasons are: - Lazy allocation. Malloc allocates virtual memory addresses, but only use does physical memory allocation. - Copy on write semantics, as is done with fork() for example. - Shared memory, including that of shared libraries text and data sections. - Memory mapped IO. And so on... In addition, memory may exist in both physical memory and swap at the same time, possibly in different versions (e.g. dirty anonymous page which has been previously swapped). In short, please ask your question again using more exact terms so that we understand what you want to ask. Gilad -- Gilad Ben-Yossef [EMAIL PROTECTED] Codefidence. A name you can trust(tm) Web: http://codefidence.com | Cel: +972.52.8260388 IL: +972.3.7515563 ext. 201 | Fax:+972.3.7515503 = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
On Tue, 2007-12-18 at 17:03 +0200, Gilad Ben-Yossef wrote: Oren Held wrote: There's something in your question I don't understand: If a process has 1gb in virtual memory, of which 500mb in physical, then this means that it has 500mb in swap. No, that is not what it means. Virtual memory amount might be different then physical memory amount due a whole bunch of different reason, swapping being just a single instance and not even the common one. In short, please ask your question again using more exact terms so that we understand what you want to ask. I was using the 1.5GB process only as a (bad) example. If you really want specifics, then said process has about 1.5GB under the virtual column (which can indeed be a lot of different things other then just real + swap, although interestingly this is exactly how 'man top' defines VIRT), about 200MB under resident column, some 8MB under shared, and 1.2GB under SWAP (according to top, not htop - I couldn't get htop to list this). Now according to 'free' only 250MB of swap are in actual use. The way I see it, 'top's SWAP is computed from virtual - resident regardless of how much swap space the process actually uses - so as Gilad said - talking about the virtual image size is next to useless. Anyway, the original question wasn't really about that specific process - I simply figured it as a good example. Obviously I was wrong, but that's ok - I was expecting something like this :-) The real question - as emphasized by all the comments I received - is: can I know how much memory the process is accessing (within some time period) specifically when its more then the total of pages actually held in physical memory. *accessing = reading to or writing from, not just having them assigned to the process. -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Processing time spent in IRQ handling and what to do about it
On Tue, 2007-12-18 at 15:21 +0200, Dotan Shavit wrote: I don't think that swapping has anything to do with the IRQ behavior I'm seeing, In that case, it probably is network related... Can you provide more details regarding this? Is the Apache server you mentioned located on the same machine? Indeed. Are you connected to a private vlan (or seeing non relevant traffic)? Its infrastructure I don't really have access to so I wouldn't know, but I'm on a good switch (maybe with a vlan) and I don't see traffic that isn't meant for me. Do you get this (a lot of time is spent in the hard-IRQ region) all the time or just when the server is accessed by it's clients? I'm always seeing some traffic, so its hard to say if I wouldn't see hard-IRQ when there aren't any clients. But interestingly enough a second identical machine which is currently doing nothing except maintaining a replica of the MySQL database on the first is also seeing high hard-IRQ counts. A third completely different computer on a different network with different work loads that also maintains a replica of the first MySQL database is also seeing high IRQ usage. What is the difference between this machine and the other (I understand the other machine works OK) ? Hardware wise and OS wise - nothing. Software wise there are many different things, but most prominently: * it doesn't see the same kind of traffic (which I currently don't think is the issue as the second server above doesn't see any traffic) * It doesn't replicate its databases. -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Processing time spent in IRQ handling and what to do about it
On Tue, 2007-12-18 at 07:48 +0200, Yedidyah Bar-David wrote: On Tue, Dec 18, 2007 at 02:49:29AM +0200, Oded Arbel wrote: Running some static benchmarks that should mimic the behavior on real load, on identical hardware at the office, I see very little hard-IRQ time if at all. The main difference between the static benchmark and real usage is that the static benchmark only tests the application logic and IO, while real usage also fetches some files served by Apache over HTTP with each request - maybe ~50Kbytes worth of responses are served by Apache for each request to the application. I was thinking that the high IRQ usage is due to high network traffic - could that be the case and could that be affecting the server's performance ? I am not an expert on this, but what you want might be NAPI - a new network driver infrastructure designed to solve just that. Google a bit - I do not know exactly when it entered 2.6 (and you did not state your kernel version) and which drivers use it already. Searching for NAPI I see some discussion on it entering 2.4 or 2.5, so I'm assuming 2.6 had it from the start. I also see some patches for the bnx2 NIC module which talk about NAPI related fixes for 2.6 - but only quite recently: October this year. I'm using Fedora 7 with kernel 2.6.22.1 which is fairly recent so I'm assuming I have this NAPI. can it possibly be currently turned off and I need to turn it on ? -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
On Tue, Dec 18, 2007 at 05:54:56PM +0200, Oded Arbel wrote: The real question - as emphasized by all the comments I received - is: can I know how much memory the process is accessing (within some time period) specifically when its more then the total of pages actually held in physical memory. *accessing = reading to or writing from, not just having them assigned to the process. Look at the pagemap patches, which will give you the raw info you need to calculate this. See http://lwn.net/Articles/230975/ for an introduction. Cheers, Muli = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
Oded Arbel [EMAIL PROTECTED] writes: Now (again - according to my understanding) under contention - i.e. when processes need to use more physical memory then what is available - the memory manager keeps swapping stuff in and out of memory in an attempt to satisfy all requests. Under such conditions its might be useful to know - for each process - the amount of physical memory in use, the amount of virtual mapped to the process, but also how much of that virtual memory the process actually tries to use but can't get it all in physical RAM because other processes are also hogging the memory. All of the above is basically correct. Does such a thing exist in Linux? Internally to the kernel, look at the active_list and inactive_list members of struct zone. I have no idea whether it is exposed to userspace in any way. You can find out, I suppose. So the kernel knows it, what would you do with it as a user? If you need it and the kernel does not allow you, I suppose you can write your own /proc file module... May be a nice exercise for students... -- Oleg Goldshmidt | [EMAIL PROTECTED] | http://www.goldshmidt.org = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux memory monitoring compared to MS-Windows
On Tue, 2007-12-18 at 18:36 +0200, Muli Ben-Yehuda wrote: On Tue, Dec 18, 2007 at 05:54:56PM +0200, Oded Arbel wrote: The real question - as emphasized by all the comments I received - is: can I know how much memory the process is accessing (within some time period) specifically when its more then the total of pages actually held in physical memory. *accessing = reading to or writing from, not just having them assigned to the process. Look at the pagemap patches, which will give you the raw info you need to calculate this. See http://lwn.net/Articles/230975/ for an introduction. Thanks ! this is really interesting. Using the clear_refs item one can easily implement the supposed working set functionality. I'll see if I can compile it for my kernel. -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Processing time spent in IRQ handling and what to do about it
On Tuesday, 18 בDecember 2007, Yedidyah Bar-David wrote: I am not an expert on this, but what you want might be NAPI - a new network driver infrastructure designed to solve just that. Google a bit - I do not know exactly when it entered 2.6 (and you did not state your kernel version) and which drivers use it already. 1. NAPI was new at kernel 2.3.x when it was developed towards 2.4 2. It gives the *driver* the option to toggle between interrupt driven and polling mode at runtime. E.g: - A GB ethernet at full speed may better poll the hardware every once in a while. - The same card is better off using interrupt driven mode if the trafic is low. 3. You cannot turn it on/off. The driver may support this optional API or not. If it supports it, it's the driver sole decision when it's better to use polling/interrupt-per-packet according it its hardware specifics. 4. I don't think a single fast ethernet card can severely affect your hardware interrupt load. So either: - You have a GB (or maybe 2GB?) ethernet with high load. - You have several fast-ethernet cards working at full speed. 5. A far better suspect would be the disk controller (e.g: working without DMA etc.) 6. Why guess? watch -n10 -d cat /proc/interrupts And calculate how many interrupts per-sec occured for various devices. That would give you a rough idea who are the possible suspects. -- Oron Peled Voice/Fax: +972-4-8228492 [EMAIL PROTECTED] http://www.actcom.co.il/~oron ICQ UIN: 16527398 Linux lasts longer! -- Kim J. Brand [EMAIL PROTECTED] To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
[job offer] SysAdmin
EPIX Pharmaceutical is looking for a system administrator for the computational chemistry department. Send CV to [EMAIL PROTECTED] Description The Unix System Administrator's role is to operate and tune Unix systems, servers, and related components, ensuring high levels of availability and security to support EPIX Pharmaceuticals drug discovery research. This individual also participates in the planning and implementation of policies and procedures to ensure Unix system provisioning and maintenance that is consistent with company goals, industry best practices, and regulatory requirements. Responsibilities Strategy Planning Participate in and support capacity planning and the development of long-term strategic goals for Unix systems and software in conjunction with scientists and department managers. Acquisition Deployment Work with scientists and software developers in support of existing internal research applications as well as install external scientific software applications, compilers and parallel processing (MPI) support software on the cluster. Install and configure Unix systems and software. Operational Management Perform standard configuration, management, and maintenance tasks for related file systems, input/output systems, networking, clustering, storage, and applications. Perform and test system configuration backups and restores to ensure system recovery from error or outage. Perform and test data backups and restores to ensure full data retrieval. Review and deploy Unix system releases and vendor-supplied patches according to best practices. Anticipate, mitigate, identify, troubleshoot, and resolve hardware and software problems on all Unix systems. Support application development teams throughout project lifecycles. Monitor, test, and tune system performance; preserve and provide system log files as needed. Recommend and execute modifications to Unix systems in order to improve efficiency, reliability, and performance. Conduct research on Unix-related hardware and software in support of procurement and system development efforts. Ensure secure user access and role validation processes. Perform Unix server/database transaction and security audits leading to identification and mitigation of security threats to enterprise data. Recommend and implement changes where necessary. Create required reports in response to business user needs. Create Unix shell scripts. Develop, document, and maintain Unix system training materials and systems documentation for educating end users and new IT staff. Position Requirements Formal Education Certification Academic or industrial experience is required. Knowledge Experience Minimum 2 years Unix Administration experience installing, configuring, and maintaining Unix servers, networks, and clusters. Hands-on hardware and software troubleshooting experience. Working technical knowledge of coding using Perl, shell scripting, Powershell and Python. At least 1 year Commercial Unix System (IRIX, Solaris, HPUX, AIX) an advantage. Experience in utilizing PBS Pro or other scheduling systems. Experience in FLEXlm administration a plus. Experience administrating high capacity storage. Good technical knowledge of current network hardware and standards, including Windows System Administration. Extensive application support experience with supporting internal and/or commercial scientific applications – an advantage. Familiarity with TCP/IP protocols, firewall management, and database administration. Personal Attributes Strong customer service orientation. Proven analytical and problem-solving abilities. Ability to effectively prioritize and execute tasks in a high-pressure environment. Good written, oral, and interpersonal communication skills. Ability to conduct research into Unix issues and products as required. Highly self motivated and directed, with a keen attention to detail. Team-oriented and skilled in working within a collaborative environment. To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: [job offer] SysAdmin
Gil, I love you like a brother, but I simply must Translation in body, below: - Gil Freund [EMAIL PROTECTED] wrote: Responsibilities Strategy Planning Translation: Plan and then see the plan go up in smoke Work with scientists and software developers in support of existing Translation: Herd cats systems documentation for educating end users and new IT staff. Translation: trying to imparts a few morsels of wisdom to people who do not want it and yourself Academic or industrial experience is required. Translation: No PhD, you lowly serf? including Windows System Administration. Translation: Outlook is more important Personal Attributes Translation: We're an equal-opportunity bigot! Strong customer service orientation. Translation: Groveling is a way of life and a universal panacea Proven analytical and problem-solving abilities. Translation: Must be able to understand pained quizzical eyebrow movements from PhD's Ability to effectively prioritize and execute tasks in a high-pressure environment. Translation: Oi, Gevald, my Outlook has deleted ALL MY MAIL JUST LIKE I TOLD IT! GET IT BACK NOW Good written, oral, and interpersonal communication skills. Translation: Good written, oral, labia-lingua-cloacal, and interpersonal communication skills Team-oriented and skilled in working within a collaborative environment. Translation: Able to withstand constant backbiting and angle-gnawing = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: [job offer] SysAdmin
Hi Gil, On Dec 18, 2007 11:19 PM, Gil Freund [EMAIL PROTECTED] wrote: EPIX Pharmaceutical is looking for a system administrator for the computational chemistry department. Send CV to [EMAIL PROTECTED] Oh, you do need someone which has some miminal DNS configuration skills.. Check this out: Go to: http://epixpharma.com Now go to: http://www.epixpharma.com Clearly someone at Epix needs to resolve this issue, for the sake of company public image. Thanks, Hetz -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Extra Features of MP3 Drivers
Hi Eli, haven't seen you here for quite a long time Most of the cheap MP3 players here in Israel, their software is in ROM without being able to upgrade or program them to something different. The more expensive ones (iRiver, Archos, SanDisk, Apple, etc) do have the ability to be upgraded and replace their firmware, so basically you CAN program their buttons and display, but not through their drivers. See for example rockbox and ipod-Linux for iPod variants. I'll warn you though - lots and lots of reverse engineering is required and good knowledge of ARM assembly is required (unless it's IPod and if it is, then you can ask either the rock-box or the ipod-linux guys about your desire). Be prepared for lots of fun with firmware images (upload, remove, etc...), and no JTAG here.. :) Thanks, Hetz On Dec 18, 2007 3:08 PM, Eli Marmor [EMAIL PROTECTED] wrote: Hi, I'm not sure if it's off-topic or not, so please excuse me if it is: Although MP3 players comply with standards, so they can work with PCs without installing any driver, most of them come with drivers: Some for Windows, some for Linux, some for both, and some without drivers. My question: Are there MP3 players which allow their drivers to access their display and/or buttons? (so programs running on the PC can write directly to the LCD of the MP3, and/or read which buttons are pressed). There are more than 1000 MP3 models in Israel (according to ZAP), so even if only 1% of them support this feature, it gives enough choices... Thanks, -- Eli Marmor [EMAIL PROTECTED] Netmask (El-Mar) Internet Technologies Ltd. __ Tel.: +972-9-766-1020 8 Yad-Harutzim St. Fax.: +972-9-766-1314 P.O.B. 7004 Mobile: +972-50-5237338 Kfar-Saba 44641, Israel = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED] -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
64-bit linux and 32-bit applications
Hi list, What is the prevailing opinion about installing and running 32-bit applications and shared libraries on 64-bit Linux operating systems? Do Red Hat based 64-bit operating systems support 32-bit applications and shared libraries, but Debian based 64-bit operating systems do not? -- Moshe Gorohovsky A6 CC A7 E1 C2 BD 8C 1B 30 8E A4 C3 4C 09 88 47 Tk Open Systems Ltd. --- - [EMAIL PROTECTED] - tel: +972.2.679.5364, http://www.tkos.co.il - = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
