Re: [chromium-dev] Re: [WebGL] Recommending --no-sandbox
[from right address] On Fri, Dec 11, 2009 at 11:37 AM, Steve VanDeBogart vand...@google.comwrote: On Fri, Dec 11, 2009 at 11:23 AM, Scott Hess sh...@chromium.org wrote: [[And now I'm waiting for someone to suggest the --no-really-no-sandbox-i-like-being-insecure flag to suppress the dialog :-).]] Instead of telling people to use --no-sandbox on the blog, we could tell them to use a new flag, --disable-sandbox-until MM/DD/. It could limit the maximum amount of time the sandbox was disabled, to say two weeks. After that, the sandbox would automatically be reenabled. There should still be some kind of notification that the user is running with the sandbox disabled, but people that ignore it or forget about it will get converted back to a secure configuration in a reasonable amount of time. -- Steve -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Windows build problem: base_nacl_win64.lib LNK1112: module machine type 'x64' conflicts with target machine type 'X86'
My windows build has been broken for at least a day with the following error: base_nacl_win64.lib(string_util.obj) : fatal error LNK1112: module machine type 'x64' conflicts with target machine type 'X86' I've tried disabling nacl, gclient sync --force, clobbering the output directory, removing all non-repository files, etc with no luck. Any ideas? -- Steve -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Large commit - update your .gclient files to avoid
This afternoon I will update DEPS to pull in 170MB of profile data for memory_test. Unless you run memory_test, you probably want to add the following line to the custom_deps section of your .gclient file. src/data/memory_test/membuster: None, -- Steve --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Large commit - update your .gclient files to avoid
Apologies, this will only apply to committers from Google. -- Steve On Thu, Nov 5, 2009 at 11:16 AM, Steve VanDeBogart vand...@chromium.orgwrote: This afternoon I will update DEPS to pull in 170MB of profile data for memory_test. Unless you run memory_test, you probably want to add the following line to the custom_deps section of your .gclient file. src/data/memory_test/membuster: None, -- Steve --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Spellchecker and memory-mapped dicts
If you plan to read the entire file, mmap()ing it, then faulting it in will be slower than read()ing it, at least in some Linux versions. I never pinned down exactly why, but I think the kernel read-ahead mechanism works slightly differently. -- Steve On Thu, Oct 22, 2009 at 2:02 PM, Chris Evans cev...@chromium.org wrote: There's also option 3) Pre-fault the mmap()ed region in the file thread upon dictionary initialization. On Linux at least, that may give you better behaviour than malloc() + read() in the event of memory pressure. Cheers Chris On Thu, Oct 22, 2009 at 1:39 PM, Evan Stade est...@chromium.org wrote: Hi all, At its last meeting the jank task force discussed improving responsiveness of the spellchecker but we didn't come to a solid conclusion so I thought I'd bring it up here to see if anyone else has opinions. The main concern is that we don't block the IO thread on file access. To this end, I recently moved initialization of the spellchecker from the IO thread to the file thread. However, instead of reading in the spellchecker dictionary in one solid chunk, we memory-map it. Then later we check individual words on the IO thread, which will be slow since the dictionary starts off effectively completely paged out. The proposal is that we read in the dictionary at spellchecker intialization instead of memory mapping it. Memory mapping pros: - possibly uses less overall memory, depending on the structure of the dictionary and the usage pattern of the user. - strikeloading the dictionary doesn't block for a long time/strike this one no longer occurs either way due to my recent refactoring Reading it all at once pros: - costly disk accesses are kept to the file thread (excepting future memory paging) - overall disk access time is probably lower (since we can read in the dict in one chunk) For reference, the English dictionary is about 500K, and most dictionaries are under 2 megs, some (such as Hungarian) are much higher, but no dictionary is over 10 megs. Opinions? -- Evan Stade --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Spellchecker and memory-mapped dicts
It's been awhile since I looked at this, but the email I was able to dig up suggests that madvise is no faster than faulting in the mmap()ed region by hand. However, using posix_fadvise should give the same speeds as read()ing it into memory. IIRC though, posix_fadvise will only read so much in a single request and will let readahead handle the rest after that. -- Steve On Thu, Oct 22, 2009 at 2:27 PM, Scott Hess sh...@chromium.org wrote: On Linux what about mmap() and then madvise() with MADV_WILLNEED? [or posix_fadvise() with POSIX_FADV_WILLNEED on the file descriptor). -scott On Thu, Oct 22, 2009 at 2:06 PM, Steve Vandebogart vand...@chromium.org wrote: If you plan to read the entire file, mmap()ing it, then faulting it in will be slower than read()ing it, at least in some Linux versions. I never pinned down exactly why, but I think the kernel read-ahead mechanism works slightly differently. -- Steve On Thu, Oct 22, 2009 at 2:02 PM, Chris Evans cev...@chromium.org wrote: There's also option 3) Pre-fault the mmap()ed region in the file thread upon dictionary initialization. On Linux at least, that may give you better behaviour than malloc() + read() in the event of memory pressure. Cheers Chris On Thu, Oct 22, 2009 at 1:39 PM, Evan Stade est...@chromium.org wrote: Hi all, At its last meeting the jank task force discussed improving responsiveness of the spellchecker but we didn't come to a solid conclusion so I thought I'd bring it up here to see if anyone else has opinions. The main concern is that we don't block the IO thread on file access. To this end, I recently moved initialization of the spellchecker from the IO thread to the file thread. However, instead of reading in the spellchecker dictionary in one solid chunk, we memory-map it. Then later we check individual words on the IO thread, which will be slow since the dictionary starts off effectively completely paged out. The proposal is that we read in the dictionary at spellchecker intialization instead of memory mapping it. Memory mapping pros: - possibly uses less overall memory, depending on the structure of the dictionary and the usage pattern of the user. - strikeloading the dictionary doesn't block for a long time/strike this one no longer occurs either way due to my recent refactoring Reading it all at once pros: - costly disk accesses are kept to the file thread (excepting future memory paging) - overall disk access time is probably lower (since we can read in the dict in one chunk) For reference, the English dictionary is about 500K, and most dictionaries are under 2 megs, some (such as Hungarian) are much higher, but no dictionary is over 10 megs. Opinions? -- Evan Stade --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Spellchecker and memory-mapped dicts
Probably a bit off topic at this point, but but your response confuses me - MADV_WILLNEED and POSIX_FADV_WILLNEED will bring the pages into ram, just like faulting in mmap()'ed pages by hand, or read()ing it into memory. In my experience, read() and fadvise() are faster than mmap()+faulting everything in, or madvise(). Of course, read()ing it in means it has to be swapped out and can't just be dropped. If you want to suck the entire file in at some point, probably the best way is to fadvise() it in, then mmap() it and use it from there. -- Steve On Thu, Oct 22, 2009 at 2:52 PM, Scott Hess sh...@chromium.org wrote: Faulting it in by hand is only helpful if we're right! If we're wrong, it could evict other stuff from memory to support a feature which a user may not use until the memory is faulted back out anyhow. [From the rest of the thread, though, it sounds like maybe we should just fix hunspell to be more efficient for our usage.] -scott On Thu, Oct 22, 2009 at 2:42 PM, Steve Vandebogart vand...@chromium.org wrote: It's been awhile since I looked at this, but the email I was able to dig up suggests that madvise is no faster than faulting in the mmap()ed region by hand. However, using posix_fadvise should give the same speeds as read()ing it into memory. IIRC though, posix_fadvise will only read so much in a single request and will let readahead handle the rest after that. -- Steve On Thu, Oct 22, 2009 at 2:27 PM, Scott Hess sh...@chromium.org wrote: On Linux what about mmap() and then madvise() with MADV_WILLNEED? [or posix_fadvise() with POSIX_FADV_WILLNEED on the file descriptor). -scott On Thu, Oct 22, 2009 at 2:06 PM, Steve Vandebogart vand...@chromium.org wrote: If you plan to read the entire file, mmap()ing it, then faulting it in will be slower than read()ing it, at least in some Linux versions. I never pinned down exactly why, but I think the kernel read-ahead mechanism works slightly differently. -- Steve On Thu, Oct 22, 2009 at 2:02 PM, Chris Evans cev...@chromium.org wrote: There's also option 3) Pre-fault the mmap()ed region in the file thread upon dictionary initialization. On Linux at least, that may give you better behaviour than malloc() + read() in the event of memory pressure. Cheers Chris On Thu, Oct 22, 2009 at 1:39 PM, Evan Stade est...@chromium.org wrote: Hi all, At its last meeting the jank task force discussed improving responsiveness of the spellchecker but we didn't come to a solid conclusion so I thought I'd bring it up here to see if anyone else has opinions. The main concern is that we don't block the IO thread on file access. To this end, I recently moved initialization of the spellchecker from the IO thread to the file thread. However, instead of reading in the spellchecker dictionary in one solid chunk, we memory-map it. Then later we check individual words on the IO thread, which will be slow since the dictionary starts off effectively completely paged out. The proposal is that we read in the dictionary at spellchecker intialization instead of memory mapping it. Memory mapping pros: - possibly uses less overall memory, depending on the structure of the dictionary and the usage pattern of the user. - strikeloading the dictionary doesn't block for a long time/strike this one no longer occurs either way due to my recent refactoring Reading it all at once pros: - costly disk accesses are kept to the file thread (excepting future memory paging) - overall disk access time is probably lower (since we can read in the dict in one chunk) For reference, the English dictionary is about 500K, and most dictionaries are under 2 megs, some (such as Hungarian) are much higher, but no dictionary is over 10 megs. Opinions? -- Evan Stade --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Spellchecker and memory-mapped dicts
That is the intention of the interface yes, but all Linux implementations I've seen actually go and read what ever you say you will need. Of course with a few exceptions like actually being out of memory. -- Steve On Thu, Oct 22, 2009 at 3:06 PM, Scott Hess sh...@chromium.org wrote: WILLNEED says Hey, OS, I think I'm going to look at these pages soon, get yourself ready, but the OS could implement them as a nop, and can do it async. If memory is under pressure the system can do less, if memory is clear it can do more. Actually reading the data into memory blocks and actually reads them into memory. -scott On Thu, Oct 22, 2009 at 3:01 PM, Steve Vandebogart vand...@chromium.org wrote: Probably a bit off topic at this point, but but your response confuses me - MADV_WILLNEED and POSIX_FADV_WILLNEED will bring the pages into ram, just like faulting in mmap()'ed pages by hand, or read()ing it into memory. In my experience, read() and fadvise() are faster than mmap()+faulting everything in, or madvise(). Of course, read()ing it in means it has to be swapped out and can't just be dropped. If you want to suck the entire file in at some point, probably the best way is to fadvise() it in, then mmap() it and use it from there. -- Steve On Thu, Oct 22, 2009 at 2:52 PM, Scott Hess sh...@chromium.org wrote: Faulting it in by hand is only helpful if we're right! If we're wrong, it could evict other stuff from memory to support a feature which a user may not use until the memory is faulted back out anyhow. [From the rest of the thread, though, it sounds like maybe we should just fix hunspell to be more efficient for our usage.] -scott On Thu, Oct 22, 2009 at 2:42 PM, Steve Vandebogart vand...@chromium.org wrote: It's been awhile since I looked at this, but the email I was able to dig up suggests that madvise is no faster than faulting in the mmap()ed region by hand. However, using posix_fadvise should give the same speeds as read()ing it into memory. IIRC though, posix_fadvise will only read so much in a single request and will let readahead handle the rest after that. -- Steve On Thu, Oct 22, 2009 at 2:27 PM, Scott Hess sh...@chromium.org wrote: On Linux what about mmap() and then madvise() with MADV_WILLNEED? [or posix_fadvise() with POSIX_FADV_WILLNEED on the file descriptor). -scott On Thu, Oct 22, 2009 at 2:06 PM, Steve Vandebogart vand...@chromium.org wrote: If you plan to read the entire file, mmap()ing it, then faulting it in will be slower than read()ing it, at least in some Linux versions. I never pinned down exactly why, but I think the kernel read-ahead mechanism works slightly differently. -- Steve On Thu, Oct 22, 2009 at 2:02 PM, Chris Evans cev...@chromium.org wrote: There's also option 3) Pre-fault the mmap()ed region in the file thread upon dictionary initialization. On Linux at least, that may give you better behaviour than malloc() + read() in the event of memory pressure. Cheers Chris On Thu, Oct 22, 2009 at 1:39 PM, Evan Stade est...@chromium.org wrote: Hi all, At its last meeting the jank task force discussed improving responsiveness of the spellchecker but we didn't come to a solid conclusion so I thought I'd bring it up here to see if anyone else has opinions. The main concern is that we don't block the IO thread on file access. To this end, I recently moved initialization of the spellchecker from the IO thread to the file thread. However, instead of reading in the spellchecker dictionary in one solid chunk, we memory-map it. Then later we check individual words on the IO thread, which will be slow since the dictionary starts off effectively completely paged out. The proposal is that we read in the dictionary at spellchecker intialization instead of memory mapping it. Memory mapping pros: - possibly uses less overall memory, depending on the structure of the dictionary and the usage pattern of the user. - strikeloading the dictionary doesn't block for a long time/strike this one no longer occurs either way due to my recent refactoring Reading it all at once pros: - costly disk accesses are kept to the file thread (excepting future memory paging) - overall disk access time is probably lower (since we can read in the dict in one chunk) For reference, the English dictionary is about 500K, and most dictionaries are under 2 megs, some (such as Hungarian) are much higher, but no dictionary is over 10 megs. Opinions? -- Evan Stade --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev
[chromium-dev] Re: Why SOCK_SEQPACKET?
On Fri, Oct 2, 2009 at 4:20 PM, Dan Kegel d...@kegel.com wrote: (I think PIPE_BUF is 64K on modern Linux, but might be smaller elsewhere.) I seem to recall that pipe buf is a page and /usr/include/linux/limits.h says #define PIPE_BUF4096 -- Steve --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: WebKit Hygiene: Please be kind. Test your patches. Warn about two-sided patches.
On Thu, Sep 24, 2009 at 1:49 PM, Yaar Schnitman y...@chromium.org wrote: Yes. With git:git try -b bot --webkit webkit branch chromium branch With gcl: Manually create a patch by concatenating two (chromium + webkit) patches. Make sure that the webkit patches have the right prefixes by using --src-prefix=src/third_party/WebKit --dst-prefix=src/third_party/WebKit on your diff command. On Thu, Sep 24, 2009 at 1:43 PM, Jeremy Orlow jor...@chromium.org wrote: On Tue, Sep 22, 2009 at 6:35 PM, Adam Barth aba...@chromium.org wrote: On Tue, Sep 22, 2009 at 6:25 PM, John Abd-El-Malek j...@chromium.org wrote: Is this even possible? i.e. I had uploaded a WebKit patch on codereview but none of the patchsets got run on the try server http://codereview.chromium.org/178030/show It is possible: aba...@zenque:~/svn/kr/src/third_party/WebKit$ gcl try scriptcontext --use_svn --svn_repo=svn://svn.chromium.org/chrome-try/try --bot layout_win,layout_mac,layout_linux --root src/third_party What about 2 sided patches. Is it possible to test those? For gcl, you can also gcl try change -f wekitfile1,webkitfile2,etc. You can also synthesize a change with webkit files by creating a file in .svn/gcl_info/changes and then gcl try chrome_change,webkit_change. -- Steve --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Is there any good tool on Linux to browser and cross reference chromium source code?
I don't know if you've already looked at Doxygen, but to help bootstrap me into the code, I ran it on the codebase minus third_party/webkit. It picked up the doxygen specific formatting for V8, but the classes and files sections are there: http://chrome.nerdbox.net (r21988) I have also in the past used LXR to explore larger codebases. I have an older version of LXR indexing SQLite code here: http://read.cs.ucla.edu/~vandebo/sqlite/ -- Steve On Wed, Sep 16, 2009 at 10:41 AM, Evan Martin e...@chromium.org wrote: I'd like to set up a web-based code browser on chromium.org, but I played around with a couple and wasn't too happy with any of them. Suggestions are welcome. On Tue, Sep 15, 2009 at 9:19 PM, James Su su...@chromium.org wrote: Hi, I'm currently using cscope + vim, but it's so slow and hard to use. Is there any better choice? Regards James Su --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
