Re: [webkit-dev] strategy for evaluating performance issues related to memory.
On Jun 21, 2010, at 11:59 AM, Mike Marchywka wrote: > > I was hardly worried about who does anything as much as what would make sense > to do. I have interest, motivation, > and multiple copies of the code but not a lot of time to waste of bad > approaches. There was a prior discussion > about coding conventions that should be applicable even to those > contemplating a contribution of just browsing > the code, I fail to see how this discussion is less relevant to current and > possible future development concerns. > > If there was some piece of this or a related effort that could be aided by > certain code features that > would seem to be of interest to everyone and it isn't clear which people > would have important thoughts > to contribute ( or I would take it some other place). > > So I take it that now you just have factories and smart pointers and just > make stuff and have it > allocated wherever without further thought? I guess I could do some > profiling my self and empirically > find problems and just assume that no one has specific comments on suspects > or things they have observed > as possible problems. In my experience with performance work, and specifically in the context of WebKit, I believe the following are useful approaches to reducing memory use: 1) Find and fix memory leaks. There are good tools for this, and memory leaks contribute considerably to memory growth over a long browsing session. Long-term memory growth is a bigger concern than one-time costs or per-page memory that is properly returned to the system. 2) Run memory profiling tools under a significant and realistic workload, such as Mozilla's "membuster" test. We have had great success with this and in particular you can find some good recent memory use improvements from Sam Weinig and Anders Carlsson, among others, if you look at the ChangeLog. 3) Track memory benchmarks regularly, and identify and fix regressions. 4) Run long automated page loads to verify that memory growth stabilizes eventually, rather than continuing to grow without bound. 5) Investigate memory held by caches, and figure out ways to get the same speed benefits with less overall memory use, for example by discarding redundant data or better tuning the cache to hold the items most likely to be reused. 6) Find reproducible cases of non-leak repeatable memory growth, and determine where the extra memory is going. If you are interested in improving WebKit's memory use, I encourage you to consider one or more of the above approaches. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] strategy for evaluating performance issues related to memory.
On Jun 21, 2010, at 6:21 AM, Mike Marchywka wrote: > (and yes my disk light still comes on for minutes at a time locking me out of > doing antyhing with iceweasel or firefox) Two things: 1. What hardware platform and O/S are you running a WebKit browser on that still uses a disk light? (Do PC cases still have disk lights? I guess I haven't looked recently.) 2. Why do you keep mentioning Iceweasel and Firefox? Those browsers are based on Mozilla's Gecko engine, not WebKit. Dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Directory upload experimental feature
FYI, I've posted a patch for my proposed version of the directory upload feature at https://bugs.webkit.org/show_bug.cgi?id=40872. Most of it is behind a ENABLE_DIRECTORY_UPLOAD flag, except for some glue code I needed to integrate with the recent Blob changes. One question was on the directory attribute. I didn't notice any precedent for attributes named with the webkit-foo style, but since this is an experimental thing I would think "webkit-directory" might make sense. Thanks, -John On Thu, Jun 3, 2010 at 7:19 AM, David Kilzer wrote: > Also, I was simply pointing out existing behavior, not arguing for/against > the zip file format. > > Dave > > > -- > *From:* Sam Weinig > *To:* David Kilzer > *Cc:* John Gregg ; webkit-dev@lists.webkit.org; Adele > Peterson > *Sent:* Wed, June 2, 2010 11:28:11 AM > > *Subject:* Re: [webkit-dev] Directory upload experimental feature > > I think this is only true for Mac OS X style bundles, not all folders. > > On Wed, Jun 2, 2010 at 3:44 AM, David Kilzer wrote: > >> > Other alternatives? >> >> I believe Safari will zip a folder and send it as a single file for you if >> you attach a folder to a file upload element instead of an individual file. >> >> Dave >> >> >> -- >> *From:* John Gregg >> *To:* Sam Weinig >> *Cc:* webkit-dev@lists.webkit.org >> *Sent:* Tue, June 1, 2010 3:09:00 PM >> *Subject:* Re: [webkit-dev] Directory upload experimental feature >> >> My proposal for that is that all the files would be listed in the form >> submission the same way as if it were a , but in >> the Content-Disposition header, the filename component would contain the >> path information. >> >> One alternative idea would be add a "path" component to the >> Content-Disposition header alongside the filename which remains unchanged, >> but I think that would be a much more difficult approach. Other >> alternatives? >> >> Example follows. >> >> -John >> >> If you are have these files >> /home/John/photos/vacation/1.jpeg >> /home/John/photos/vacation/2.jpeg >> /home/John/photos/conference/1.jpeg >> >> and choose "photos" from the directory picker, you'd end up with >> input.files[0].name = "1.jpeg" >> input.files[0].path = "photos/vacation/1.jpeg" >> input.files[1].name = "2.jpeg" >> input.files[1].path = "photos/vacation/2.jpeg" >> input.files[2].name = "1.jpeg" >> input.files[2].path = "photos/conference/1.jpeg" >> >> Your POST would look like >> Content-type: multipart/form-data; boundary=WebKitFormBoundaryFoo >> >> WebKitFormBoundaryFoo >> Content-Disposition: form-data; name="input"; >> filename="photos/vacation/1.jpeg" >> Content-Type: image/jpeg >> >> >> >> WebKitFormBoundaryFoo >> Content-Disposition: form-data; name="input"; >> filename="photos/vacation/2.jpeg" >> Content-Type: image/jpeg >> >> >> >> WebKitFormBoundaryFoo >> Content-Disposition: form-data; name="input"; >> filename="photos/conference/1.jpeg" >> Content-Type: image/jpeg >> >> >> >> >> On Sat, May 29, 2010 at 10:22 AM, Sam Weinig wrote: >> >>> How will the directory structure and all the files therein be represented >>> in the form submission? >>> >>> -Sam >>> >>> On Fri, May 28, 2010 at 3:17 PM, John Gregg wrote: >>> Hi WebKit, I recently proposed adding directory upload support to HTML via a new attribute to whatwg@, and the discussion arrived at "try it out". Having written some code I think I have something that works pretty well, and I'd like to land it on an experimental basis in WebKit, but want to reach out early before trying to put any code in the tree. The plan that comes to mind is a new ENABLE_DIRECTORY_UPLOAD flag, but I'm completely open to other options. Background (cf. the whatwg thread http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-April/025764.html ): - The use case for this is a photo album or file manager web application, which wants the user to easily choose an entire directory to recursively upload, while preserving the sub-directory structure. - The reason for the new attribute is to signal the UA to show a native folder-picker rather than a file-picker, which on most OSs are two distinct dialogs. The approach I'm using has 2 parts and is a small amount of WebCore code (about 200 lines). - Extend HTMLInputElement to support the directory attribute, which is passed up via FileChooser allowing the UA to display a folder-picker. UA enumerates all the files and returns them in the normal way. - Extend File to have a File.path property, which contains the path information starting from the chosen directory as the root. HTMLInputElement is responsible for generating these values from the list of files when the directory attribute is set. Thoughts? Thanks, -John ___
Re: [webkit-dev] strategy for evaluating performance issues related to memory.
On Jun 21, 2010, at 11:59 AM, Mike Marchywka wrote: > I guess I could do some profiling my self and empirically > find problems and just assume that no one has specific comments on suspects > or things they have observed > as possible problems. I spent a couple of months last year looking at memory usage of WebKit in Chromium, using tools like Shark and vmmap. I was able to tweak a couple of things for some measurable improvements, like shaving bytes off of each String object, but people have been optimizing this code for years, and believe me, there isn’t any low-hanging fruit left in there. Further optimization, IMHO, would need to be done by someone who’s a real expert at some of the high-level data structures involved (like the render tree, say) who can figure out a way to represent the same information in more compact or lazier form. Don’t assume that you can just walk into the codebase for the first time and start optimizing. —Jens ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] strategy for evaluating performance issues related to memory.
> From: jor...@chromium.org > Date: Mon, 21 Jun 2010 19:41:28 +0100 > Subject: Re: [webkit-dev] strategy for evaluating performance issues related > to memory. > To: marchy...@hotmail.com > CC: webkit-dev@lists.webkit.org > > On Mon, Jun 21, 2010 at 7:08 PM, Mike Marchywka> wrote: > > > >> Unless you're actively working on this problem within WebKit, these emails >> seem out of scope for webkit-dev. > > > > The topic addresses this doesn't it? I would think that outlining a > development strategy would be "actively working." > > I don't expect to dig into the code right now beyond what I have already done > but if I could figure out > > what to do I might be able to make more specific contributions later. > > If you're not contributing code and you don't have people interested in > following your lead, then no I don't think this is the applicable list. I'm > not aware of any WebKit contributor that's twiddling their thumbs trying to > find something to work on. > > Maybe this topic will rise to the top of a contributors priority queue at > some point, in which case I could see a discussion being on topic and useful. I was hardly worried about who does anything as much as what would make sense to do. I have interest, motivation, and multiple copies of the code but not a lot of time to waste of bad approaches. There was a prior discussion about coding conventions that should be applicable even to those contemplating a contribution of just browsing the code, I fail to see how this discussion is less relevant to current and possible future development concerns. If there was some piece of this or a related effort that could be aided by certain code features that would seem to be of interest to everyone and it isn't clear which people would have important thoughts to contribute ( or I would take it some other place). So I take it that now you just have factories and smart pointers and just make stuff and have it allocated wherever without further thought? I guess I could do some profiling my self and empirically find problems and just assume that no one has specific comments on suspects or things they have observed as possible problems. > > J _ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] strategy for evaluating performance issues related to memory.
On Mon, Jun 21, 2010 at 7:08 PM, Mike Marchywka wrote: > > > Unless you're actively working on this problem within WebKit, these > emails seem out of scope for webkit-dev. > > The topic addresses this doesn't it? I would think that outlining a > development strategy would be "actively working." > I don't expect to dig into the code right now beyond what I have already > done but if I could figure out > what to do I might be able to make more specific contributions later. > If you're not contributing code and you don't have people interested in following your lead, then no I don't think this is the applicable list. I'm not aware of any WebKit contributor that's twiddling their thumbs trying to find something to work on. Maybe this topic will rise to the top of a contributors priority queue at some point, in which case I could see a discussion being on topic and useful. J ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] strategy for evaluating performance issues related to memory.
> From: jor...@chromium.org > Date: Mon, 21 Jun 2010 18:22:19 +0100 > Subject: Re: [webkit-dev] strategy for evaluating performance issues related > to memory. > To: marchy...@hotmail.com > CC: webkit-dev@lists.webkit.org > > On Mon, Jun 21, 2010 at 2:21 PM, Mike Marchywka> wrote: > > > > > > > So where does this stand right now? I hope to actually contribute at some > point. > > but right now I'm just trying to determine approaches to even finding the > problems. > > ( and yes my disk light still comes on for minutes at a time locking me out > of doing antyhing > > with iceweasel or firefox) > > Yes, you've mentioned your disk light blinking several times now. If you have > specific bugs to file against specific projects, I suggest you do that. > Repeatedly posting about it on webkit-dev (until now, in unrelated threads) > doesn't seem particularly useful. I honestly think this has become a very big problem in many cases and no one seems to appreciate it. It is simply meant to motivate the rest of the post. > > > > > I guess at a gross level it would be nice to know when and why page faults > occur > > but a more pro-active approach would be to start understanding the memory > > needs during design or code examination for other reasons. If there is nothing > > better, perhaps people could put in formatted comments about memory needs > > and access patterns and various parameters or issues or assumptions that > > in the code. I haven't entirely thought this through (duh) and hope to > stimulate > > a discussion ( actually I'd be happy if someone had a one line answer or link > > to an answer but am not that opitmistic). > > > > I would approach this as an addition to finding memory leaks but maybe the > > worst offenders will go away if there is less clutter. > > This is all fairly hand-wavy. If you have specific problems you've seen, > please file bugs. If you want to spend some time investigating these issues, > great. But I don't understand why you keep bringing this up. Especially > since, as far as I can tell, it only involves situations where WebKit memory > is spilling to virtual memory which, AFAIK, is not something anyone has spent > time optimizing. This list has discussed many topics relating to bugs, coding conventions, performance, features, etc. These are not independent issues- presumably coding relates to ease of debug and optimization etc. I'm trying to put together a strategy for digging into the memory issues which everyone seems to agree have been largely ignored. Long before VM becomes a problem, usually you have cache problems so I wouldn't say that is the only issue but certainly the most noticable when it happens. Given that no one seems to have looked at it, it is reasonable for me to be concerned that there won't be many obvious clues in the code. If people are more aware of the possible problems perhaps they could leave comments related to memory usage in various places, as just one hand-waving thought. If I find a bunch of stuff like "new Widget[x]" what am I suppoed to think about it without knowing where "x" comes from ? > > > > Unless you're actively working on this problem within WebKit, these emails > seem out of scope for webkit-dev. The topic addresses this doesn't it? I would think that outlining a development strategy would be "actively working." I don't expect to dig into the code right now beyond what I have already done but if I could figure out what to do I might be able to make more specific contributions later. > > > > > Thanks. > > > > > > - - - - - - > > > > Mike Marchywka | V.P. Technology > > > > 415-264-8477 > > marchy...@phluant.com > > > > Online Advertising and Analytics for Mobile > > http://www.phluant.com > > > > > > > > > > > > note new address > > Mike Marchywka > > 1975 Village Round > > Marietta GA 30064 > > 415-264-8477 (w)<- use this > > 404-788-1216 (C)<- leave message > > 989-348-4796 (P)<- emergency only > > marchy...@hotmail.com > > Note: If I am asking for free stuff, I normally use for hobby/non-profit > > information but may use in investment forums, public and private. > > Please indicate any concerns if applicable. > > > > > > > > > > _ > > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. > > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 > > > > ___ > > webkit-dev mailing list > > webkit-dev@lists.webkit.org > > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:04201
Re: [webkit-dev] strategy for evaluating performance issues related to memory.
On Mon, Jun 21, 2010 at 2:21 PM, Mike Marchywka wrote: > > > So where does this stand right now? I hope to actually contribute at some > point. > but right now I'm just trying to determine approaches to even finding the > problems. > ( and yes my disk light still comes on for minutes at a time locking me out > of doing antyhing > with iceweasel or firefox) > Yes, you've mentioned your disk light blinking several times now. If you have specific bugs to file against specific projects, I suggest you do that. Repeatedly posting about it on webkit-dev (until now, in unrelated threads) doesn't seem particularly useful. I guess at a gross level it would be nice to know when and why page faults > occur > but a more pro-active approach would be to start understanding the memory > needs during design or code examination for other reasons. If there is > nothing > better, perhaps people could put in formatted comments about memory needs > and access patterns and various parameters or issues or assumptions that > in the code. I haven't entirely thought this through (duh) and hope to > stimulate > a discussion ( actually I'd be happy if someone had a one line answer or > link > to an answer but am not that opitmistic). > > I would approach this as an addition to finding memory leaks but maybe the > worst offenders will go away if there is less clutter. > This is all fairly hand-wavy. If you have specific problems you've seen, please file bugs. If you want to spend some time investigating these issues, great. But I don't understand why you keep bringing this up. Especially since, as far as I can tell, it only involves situations where WebKit memory is spilling to virtual memory which, AFAIK, is not something anyone has spent time optimizing. Unless you're actively working on this problem within WebKit, these emails seem out of scope for webkit-dev. Thanks. > > > - - - - - - > > Mike Marchywka | V.P. Technology > > 415-264-8477 > marchy...@phluant.com > > Online Advertising and Analytics for Mobile > http://www.phluant.com > > > > > > note new address > Mike Marchywka > 1975 Village Round > Marietta GA 30064 > 415-264-8477 (w)<- use this > 404-788-1216 (C)<- leave message > 989-348-4796 (P)<- emergency only > marchy...@hotmail.com > Note: If I am asking for free stuff, I normally use for hobby/non-profit > information but may use in investment forums, public and private. > Please indicate any concerns if applicable. > > > > > _ > Hotmail has tools for the New Busy. Search, chat and e-mail from your > inbox. > > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] strategy for evaluating performance issues related to memory.
So where does this stand right now? I hope to actually contribute at some point. but right now I'm just trying to determine approaches to even finding the problems. ( and yes my disk light still comes on for minutes at a time locking me out of doing antyhing with iceweasel or firefox) I guess at a gross level it would be nice to know when and why page faults occur but a more pro-active approach would be to start understanding the memory needs during design or code examination for other reasons. If there is nothing better, perhaps people could put in formatted comments about memory needs and access patterns and various parameters or issues or assumptions that in the code. I haven't entirely thought this through (duh) and hope to stimulate a discussion ( actually I'd be happy if someone had a one line answer or link to an answer but am not that opitmistic). I would approach this as an addition to finding memory leaks but maybe the worst offenders will go away if there is less clutter. Thanks. - - - - - - Mike Marchywka | V.P. Technology 415-264-8477 marchy...@phluant.com Online Advertising and Analytics for Mobile http://www.phluant.com note new address Mike Marchywka 1975 Village Round Marietta GA 30064 415-264-8477 (w)<- use this 404-788-1216 (C)<- leave message 989-348-4796 (P)<- emergency only marchy...@hotmail.com Note: If I am asking for free stuff, I normally use for hobby/non-profit information but may use in investment forums, public and private. Please indicate any concerns if applicable. _ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
