Which projects are most important?
Hello, I am aware of several important problems that I can help with: 1. Removing of store_table global and moving non-shared memory cache into its own class. It would be nice to do that before we add support for shared caching of large objects (#2) so that the new code can be isolated to shared caching areas only. However, other than new bugs, this project alone will not result in changes immediately visible to users. 2. Support for shared caching of large objects in memory and Rock store. Currently, the shared memory caching limit is hard-coded to 32KB and Rock Store maximum is configurable. 3. Triage bug reports related to slow loading of Rock store at startup and fix the problem, if any. Rock store loads cache index in disker, not worker processes, so the performance impact should be minimal, but perhaps something still needs fixing or optimizing. 4. Research and possibly optimize IPC exchanges between Squid kids. I suspect busy (but not overloaded) Squids suffer from overheads related to creating new UDS sockets. If that suspicion is confirmed, we may be able to noticeably improve performance by keeping UDS sockets persistent. This project may also have a positive impact on how Squid kids locate each other during startup and kids restarts. 5. Work with Kinkie to complete the performance regression investigation and finalize performance regression testing procedures going forward. Kinkie made great progress, but several critical (and hardest to get) data points are still missing. Doing general optimizations such as StringNG or IPv6 without a good performance testing framework would be rather foolish. 6. Fix libipc/libmgr linking problems. 7. StringNG. 8. IP::Address optimization/polishing to address a known performance regression added by IPv6 support. In your opinion, what are the two or three projects I should focus on first? Please feel free to add new items if I missed something important. I will try to pick one from your prioritized list. Thank you, Alex.
Re: Which projects are most important?
Hi Alex, I'd go for the quickest wins, to remove blockers for other work. In that way, I'd work on 5. (non-coding task, time consuming) and 6, then 1. Thanks for doing this (sorry for bottom-quoting but it seems the most appropriate way to quote in this case) Kinkie On Wed, Feb 15, 2012 at 6:42 PM, Alex Rousskov rouss...@measurement-factory.com wrote: Hello, I am aware of several important problems that I can help with: 1. Removing of store_table global and moving non-shared memory cache into its own class. It would be nice to do that before we add support for shared caching of large objects (#2) so that the new code can be isolated to shared caching areas only. However, other than new bugs, this project alone will not result in changes immediately visible to users. 2. Support for shared caching of large objects in memory and Rock store. Currently, the shared memory caching limit is hard-coded to 32KB and Rock Store maximum is configurable. 3. Triage bug reports related to slow loading of Rock store at startup and fix the problem, if any. Rock store loads cache index in disker, not worker processes, so the performance impact should be minimal, but perhaps something still needs fixing or optimizing. 4. Research and possibly optimize IPC exchanges between Squid kids. I suspect busy (but not overloaded) Squids suffer from overheads related to creating new UDS sockets. If that suspicion is confirmed, we may be able to noticeably improve performance by keeping UDS sockets persistent. This project may also have a positive impact on how Squid kids locate each other during startup and kids restarts. 5. Work with Kinkie to complete the performance regression investigation and finalize performance regression testing procedures going forward. Kinkie made great progress, but several critical (and hardest to get) data points are still missing. Doing general optimizations such as StringNG or IPv6 without a good performance testing framework would be rather foolish. 6. Fix libipc/libmgr linking problems. 7. StringNG. 8. IP::Address optimization/polishing to address a known performance regression added by IPv6 support. In your opinion, what are the two or three projects I should focus on first? Please feel free to add new items if I missed something important. I will try to pick one from your prioritized list. Thank you, Alex.
Re: Which projects are most important?
Performance is a hot topic at the moment; I would love to see more time going into that through any of the performance related items you listed (or other ones you may come up with). -Rob
Re: Which projects are most important?
On 16.02.2012 06:42, Alex Rousskov wrote: Hello, I am aware of several important problems that I can help with: 1. Removing of store_table global and moving non-shared memory cache into its own class. It would be nice to do that before we add support for shared caching of large objects (#2) so that the new code can be isolated to shared caching areas only. However, other than new bugs, this project alone will not result in changes immediately visible to users. I realize its probably a big work though. So don't let it threaten or suck time away from 3.2 stability. 2. Support for shared caching of large objects in memory and Rock store. Currently, the shared memory caching limit is hard-coded to 32KB and Rock Store maximum is configurable. 3. Triage bug reports related to slow loading of Rock store at startup and fix the problem, if any. Rock store loads cache index in disker, not worker processes, so the performance impact should be minimal, but perhaps something still needs fixing or optimizing. Please take a look at the bugzilla list with rating major+. Everything, including 2.x bugs are on the chopping block now. There may be some low-hanging bugs you can add to this list. ++ You could do is followup on our earlier discussion about converting the shutdown procedure into a async call series. We ended up with a good plan IMHO, but I have not had time to implement it and you can probably implement as a async faster anyway. This will have a user visible effect on faster restart/shutdown and less crash bugs. 4. Research and possibly optimize IPC exchanges between Squid kids. I suspect busy (but not overloaded) Squids suffer from overheads related to creating new UDS sockets. If that suspicion is confirmed, we may be able to noticeably improve performance by keeping UDS sockets persistent. This project may also have a positive impact on how Squid kids locate each other during startup and kids restarts. 5. Work with Kinkie to complete the performance regression investigation and finalize performance regression testing procedures going forward. Kinkie made great progress, but several critical (and hardest to get) data points are still missing. Doing general optimizations such as StringNG or IPv6 without a good performance testing framework would be rather foolish. 6. Fix libipc/libmgr linking problems. 7. StringNG. 8. IP::Address optimization/polishing to address a known performance regression added by IPv6 support. With the 3.2 focus on release, this all kind of falls under improvement of new features. Good, but really can be done after stable 1 goes out. The new parallel-DNS slow bug need properly isolating before it can be determined whether its worth being on the blockers list (looks like a config related bug right now). In your opinion, what are the two or three projects I should focus on first? Please feel free to add new items if I missed something important. I will try to pick one from your prioritized list. Bug fixing aimed at stability top priority please. The next round of major distro LTE releases start in April. So if at all possible to be properly stable by then it would be great for Squid. This seems like a reasonably ordered by priority already. I would just place (3) above (1) as it seems to be more of a low-hanging fruit. Amos
