[chromium-dev] Re: Extracting Views, creating app/
If people need things moved to base, you can file a bug against me. During this refactor, it would be nice to also have the opposite, a clean dependencies on what uses views. For Linux we would like to build without views. Right now there are dependencies from outside of views/ to views/. There is a bug filed to try to untangle it, but I suppose it makes sense to wait until you pull views/ out completely. On Tue, Apr 28, 2009 at 1:00 AM, Scott Violet s...@chromium.org wrote: chrome/common/scoped_vector.h This file really wants to be in base. Same thing for chrome/common/stl_util-inl.h . -Scott --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: chromium resources always rebuilding
The deps for all the files generated is a problem, Mark and I have talked about it a few times, but haven't come up with something complete for it yet, hopefully it will pop back up on our queues shortly so we can figure out something more complete. TVL On Mon, Apr 27, 2009 at 6:14 PM, Evan Martin e...@chromium.org wrote: PS: here's a hack I did in make.gyp to output a rule that manually touches the output file to work around this. Perhaps you could do something in your Xcode generator to work around it for now. # Deep inside the rules conversions code path: if name == 'resources_grit': # HACK: This is ugly. Grit intentionally doesn't touch the # timestamp of its output file when the file doesn't change, # which is fine in hash-based dependency systems like scons # and forge, but not kosher in the make world. After some # discussion, hacking around it here seems like the least # amount of pain. fp.write('\...@touch --no-create $...@\n') On Mon, Apr 27, 2009 at 3:12 PM, Evan Martin e...@chromium.org wrote: The build system has a generation dependency path like this: 1) final output depends on 2) grit header and cc, which depends on 3) grit input .grd file, which depends on 4) theme resources This means whenever you tweak theme resources, you cause a recompile, even when the .cc and .h in (2) don't change. So Glen requested Tony change grit to not touch the outputs if their contents don't change, and that's what's causing the problem. This works fine for scons (because it uses file content hashes to compute whether dependencies need rebuilding), but breaks systems (including make and Xcode) that rely on file timestamps. Why? Because you end up with the timestamp of 3 being newer than the timestamp of 2, so the dependency checker thinks you always need to re-run grit to convert 3=2. The symptom you get is that grit runs every time you build. I think the correct fix is to make grit just modify the output files, like every other program does. Sorry, Glen. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: chromium resources always rebuilding
You really should take a look ASAP because yesterday, the mac try slaves were like 35+ jobs being. That makes mac testing inexistent and will just cause more mac breakage. I assume today, tomorrow, etc will be as bad. You can be our sheriff in the meantime if you want. :) M-A On Tue, Apr 28, 2009 at 9:02 AM, Thomas Van Lenten thoma...@chromium.org wrote: The deps for all the files generated is a problem, Mark and I have talked about it a few times, but haven't come up with something complete for it yet, hopefully it will pop back up on our queues shortly so we can figure out something more complete. TVL On Mon, Apr 27, 2009 at 6:14 PM, Evan Martin e...@chromium.org wrote: PS: here's a hack I did in make.gyp to output a rule that manually touches the output file to work around this. Perhaps you could do something in your Xcode generator to work around it for now. # Deep inside the rules conversions code path: if name == 'resources_grit': # HACK: This is ugly. Grit intentionally doesn't touch the # timestamp of its output file when the file doesn't change, # which is fine in hash-based dependency systems like scons # and forge, but not kosher in the make world. After some # discussion, hacking around it here seems like the least # amount of pain. fp.write('\...@touch --no-create $...@\n') On Mon, Apr 27, 2009 at 3:12 PM, Evan Martin e...@chromium.org wrote: The build system has a generation dependency path like this: 1) final output depends on 2) grit header and cc, which depends on 3) grit input .grd file, which depends on 4) theme resources This means whenever you tweak theme resources, you cause a recompile, even when the .cc and .h in (2) don't change. So Glen requested Tony change grit to not touch the outputs if their contents don't change, and that's what's causing the problem. This works fine for scons (because it uses file content hashes to compute whether dependencies need rebuilding), but breaks systems (including make and Xcode) that rely on file timestamps. Why? Because you end up with the timestamp of 3 being newer than the timestamp of 2, so the dependency checker thinks you always need to re-run grit to convert 3=2. The symptom you get is that grit runs every time you build. I think the correct fix is to make grit just modify the output files, like every other program does. Sorry, Glen. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: chromium resources always rebuilding
Yes, this is certainly a direct cause of making a null build on mac take far, far longer than it should. Can we just back out Tony's change that was made in the rules to go back to the way things were in the short term? On Tue, Apr 28, 2009 at 11:31 AM, Marc-Antoine Ruel mar...@chromium.org wrote: You really should take a look ASAP because yesterday, the mac try slaves were like 35+ jobs being. That makes mac testing inexistent and will just cause more mac breakage. I assume today, tomorrow, etc will be as bad. -- Mike Pinkerton Mac Weenie pinker...@google.com --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: How do I not sync the layout tests in an existing client?
On Mon, Apr 27, 2009 at 6:47 PM, Glen Murphy g...@chromium.org wrote: I have zero need for the layout tests, yet I've never been able to set up a client without them - even with the suggested custom_deps line, I still get them. I've tried deleting my webkit dir, a carpet-bomb custom deps like, and gclient sync --force, and I still get the damned things. layout_tests/platform don't seem covered by the suggested custom_deps line, but I don't really know how these things work. custom_deps : { src/webkit/data: None, src/webkit/data/layout_tests/: None, src/webkit/data/layout_tests/LayoutTests: None, src/webkit/data/layout_tests/platform: None, src/webkit/data/layout_tests/platform/chromium-linux/LayoutTests: None, } The only one that works is: src/webkit/data/layout_tests/LayoutTests: None Because this is fetched with the DEPS files. The other ones are in the checkout directly (under src/ in the repository), and they can't be bypassed by the custom_deps. Your svn-ignore might work, but make sure you don't submit this! Nicolas --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: chromium resources always rebuilding
I'm not sure this is related to the mac try servers being slow. This only causes GRIT to re-run (maybe 10s to run on all files?), but prevents .cc files from being recompiled. Mike is right that it causes null builds to be slower. I'm happy to rollback, it doesn't matter either way for me, but if we're trying to speed up the mac try slaves, this probably isn't going to help (this change has been in for almost a month). tony On Tue, Apr 28, 2009 at 8:46 AM, Mike Pinkerton pinker...@chromium.org wrote: Yes, this is certainly a direct cause of making a null build on mac take far, far longer than it should. Can we just back out Tony's change that was made in the rules to go back to the way things were in the short term? On Tue, Apr 28, 2009 at 11:31 AM, Marc-Antoine Ruel mar...@chromium.org wrote: You really should take a look ASAP because yesterday, the mac try slaves were like 35+ jobs being. That makes mac testing inexistent and will just cause more mac breakage. I assume today, tomorrow, etc will be as bad. -- Mike Pinkerton Mac Weenie pinker...@google.com --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: chromium resources always rebuilding
Tony Chang wrote: I'm not sure this is related to the mac try servers being slow. This only causes GRIT to re-run (maybe 10s to run on all files?), but prevents .cc files from being recompiled. If there was a change that (intentionally or otherwise) caused GRIT to run on each build, the effects could easily add up to more than 10s on all files even during a null build when you start to factor in additional linking time, etc., as Mike points out. Mark --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: chromium resources always rebuilding
On Tue, Apr 28, 2009 at 10:26 AM, Mark Mentovai m...@chromium.org wrote: Tony Chang wrote: I'm not sure this is related to the mac try servers being slow. This only causes GRIT to re-run (maybe 10s to run on all files?), but prevents .cc files from being recompiled. If there was a change that (intentionally or otherwise) caused GRIT to run on each build, the effects could easily add up to more than 10s on all files even during a null build when you start to factor in additional linking time, etc., as Mike points out. Tony's point is that GRIT will run, decide that none of its outputs are different, and then touch no output files. Nothing else should run based on that (?). --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] adding libraries to gyp
In http://codereview.chromium.org/99132 , I split one library into two. This worked fine up until the Windows build, which only uses gyp halfway, got the change to use the split library but didn't get the change to fix consumers of the library. :( Is there any reasonable way to work through this? It appears I need to do chrome.sln surgery to make this work, and that involves adding a ton to the project, etc. Should I just wait for gyp to go through Windows before doing this? (Once we're using gyp everywhere, can we just make up new guids for the msvs_guid field in the gyp file?) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: adding libraries to gyp
Hi Evan, I'll take a look. Once we switched completely the msvs_guid's can go away (it will generate stable but random ones). -BradN On Tue, Apr 28, 2009 at 12:31 PM, Evan Martin e...@chromium.org wrote: In http://codereview.chromium.org/99132 , I split one library into two. This worked fine up until the Windows build, which only uses gyp halfway, got the change to use the split library but didn't get the change to fix consumers of the library. :( Is there any reasonable way to work through this? It appears I need to do chrome.sln surgery to make this work, and that involves adding a ton to the project, etc. Should I just wait for gyp to go through Windows before doing this? (Once we're using gyp everywhere, can we just make up new guids for the msvs_guid field in the gyp file?) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: adding libraries to gyp
For reference, here was the failure: http://build.chromium.org/buildbot/waterfall/builders/Chromium%20XP/builds/4079/steps/compile/logs/stdio_html It needed to depend on this new main lib from gtest. On Tue, Apr 28, 2009 at 12:34 PM, Bradley Nelson bradnel...@google.com wrote: Hi Evan, I'll take a look. Once we switched completely the msvs_guid's can go away (it will generate stable but random ones). -BradN On Tue, Apr 28, 2009 at 12:31 PM, Evan Martin e...@chromium.org wrote: In http://codereview.chromium.org/99132 , I split one library into two. This worked fine up until the Windows build, which only uses gyp halfway, got the change to use the split library but didn't get the change to fix consumers of the library. :( Is there any reasonable way to work through this? It appears I need to do chrome.sln surgery to make this work, and that involves adding a ton to the project, etc. Should I just wait for gyp to go through Windows before doing this? (Once we're using gyp everywhere, can we just make up new guids for the msvs_guid field in the gyp file?) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: adding libraries to gyp
It's not urgent at all. On Tue, Apr 28, 2009 at 1:06 PM, Bradley Nelson bradnel...@google.com wrote: Hi Evan, I'm doing a similar change now to migrate v8.gyp from temp_gyp to src/v8. Once I get that working I'll see if I can get this change working. If it isn't urgent, it might be easier to wait until after at least webkit is switched (hopefully tonight at 6). -BradN On Tue, Apr 28, 2009 at 12:36 PM, Evan Martin e...@chromium.org wrote: For reference, here was the failure: http://build.chromium.org/buildbot/waterfall/builders/Chromium%20XP/builds/4079/steps/compile/logs/stdio_html It needed to depend on this new main lib from gtest. On Tue, Apr 28, 2009 at 12:34 PM, Bradley Nelson bradnel...@google.com wrote: Hi Evan, I'll take a look. Once we switched completely the msvs_guid's can go away (it will generate stable but random ones). -BradN On Tue, Apr 28, 2009 at 12:31 PM, Evan Martin e...@chromium.org wrote: In http://codereview.chromium.org/99132 , I split one library into two. This worked fine up until the Windows build, which only uses gyp halfway, got the change to use the split library but didn't get the change to fix consumers of the library. :( Is there any reasonable way to work through this? It appears I need to do chrome.sln surgery to make this work, and that involves adding a ton to the project, etc. Should I just wait for gyp to go through Windows before doing this? (Once we're using gyp everywhere, can we just make up new guids for the msvs_guid field in the gyp file?) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Changes to FilePath?
Hi Chromium Developers, I'm working on Google's O3D (http://code.google.com/p/o3d), and we (naturally) share some of Chrome's base classes for our code, including the very useful class FilePath. However, in using FilePath in the last few months, I've seen that it needs some refinement. I'd like to augment the FilePath class with some things that would make it more generally useful -- it's very nicely set up, but it's missing a few things that make it harder to work with than it needs to be: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). 2) I'd like to make it possible to instantiate a POSIX FilePath object on Windows and a Windows FilePath on POSIX platforms. This is because some libraries (e.g. the zip library, or tar files), use POSIX semantics for their paths even on Windows (I haven't seen a use case for Windows paths on POSIX yet, actually). This would make it possible to use the nice API that FilePath has to manipulate paths appropriately for these other libraries. This could be easily accomplished by having POSIX and Windows versions of FilePath, and then typedef'ing FilePath differently on different platforms to one of these versions. 3) It would be helpful to have real path normalization for each of the platforms (although I know what a testing nightmare that can be). I might try and tackle this if people think it would be beneficial. 4) Make sure we handle case sensitivity vs case preservation correctly. It's unclear to me that FilePath does this correctly on the Mac -- Mac file names are case preserving, but case insensitive, Unix filenames are both (and windows filenames are neither :-). So, is there any resistance to any of the above? Do you have other suggestions that I might take into account? Am I violating any design assumptions of FilePath? For #2, is speed/size enough of a concern to avoid a virtual base class (I wouldn't think so, but you never know..)? -Greg. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 4:39 PM, Greg Spencer gspen...@google.com wrote: Hi Chromium Developers, I'm working on Google's O3D (http://code.google.com/p/o3d), and we (naturally) share some of Chrome's base classes for our code, including the very useful class FilePath. However, in using FilePath in the last few months, I've seen that it needs some refinement. I'd like to augment the FilePath class with some things that would make it more generally useful -- it's very nicely set up, but it's missing a few things that make it harder to work with than it needs to be: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). 2) I'd like to make it possible to instantiate a POSIX FilePath object on Windows and a Windows FilePath on POSIX platforms. This is because some libraries (e.g. the zip library, or tar files), use POSIX semantics for their paths even on Windows (I haven't seen a use case for Windows paths on POSIX yet, actually). This would make it possible to use the nice API that FilePath has to manipulate paths appropriately for these other libraries. This could be easily accomplished by having POSIX and Windows versions of FilePath, and then typedef'ing FilePath differently on different platforms to one of these versions. 3) It would be helpful to have real path normalization for each of the platforms (although I know what a testing nightmare that can be). I might try and tackle this if people think it would be beneficial. 4) Make sure we handle case sensitivity vs case preservation correctly. It's unclear to me that FilePath does this correctly on the Mac -- Mac file names are case preserving, but case insensitive, Unix filenames are both (and windows filenames are neither :-). FYI - it's a drive format time option on the Mac, so they can be case preserving and case sensitive. TVL So, is there any resistance to any of the above? Do you have other suggestions that I might take into account? Am I violating any design assumptions of FilePath? For #2, is speed/size enough of a concern to avoid a virtual base class (I wouldn't think so, but you never know..)? -Greg. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Omnibox Experiments
- Having used this for a while, my main feedback is that that icons are subtle enough that I don't notice them. It's possible they're playing into my subconscious, but it doesn't feel that way. - It's a little strange that starred entries in the result list have the *empty* star as their icon. (Contrast callofduty.com's star with the star in the toolbar above.) Sorry, I'm not a designer, so I can only offer criticism not improvements. :\ On Mon, Apr 27, 2009 at 2:41 PM, Nicholas Jitkoff j...@gmail.com wrote: Hey all, We're experimenting with improvements to the presentation of results in the omnibox. I've attached our current design. You'll start to see this appear in some of the builds. Please feel free to send me any comments. Nicholas --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 1:39 PM, Greg Spencer gspen...@google.com wrote: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). Can you give some examples of where this is needed? We've historically fought against this pretty hard, and as soon as accessors are available users will get lazy about it. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 1:57 PM, Thomas Van Lenten thoma...@chromium.orgwrote: On Tue, Apr 28, 2009 at 4:39 PM, Greg Spencer gspen...@google.com wrote: 4) Make sure we handle case sensitivity vs case preservation correctly. It's unclear to me that FilePath does this correctly on the Mac -- Mac file names are case preserving, but case insensitive, Unix filenames are both (and windows filenames are neither :-). FYI - it's a drive format time option on the Mac, so they can be case preserving and case sensitive. Thanks for pointing that out. In fact, NTFS is actually case sensitive, where FAT32 is not (see http://support.microsoft.com/kb/100625). So we have issues there as well. The real issue would be dealing with relative paths that don't exist yet -- there would be no way to inspect the file location to find out what mode it was in. I think I would just punt and go with the widely-used defaults (the ones I mentioned above), since most apps seem to assume those limitations. An alternative would be to have an API to specify the desired mode, and default to the common case on each platform. -Greg. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 1:39 PM, Greg Spencer gspen...@google.com wrote: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). I'm pretty strongly against this for the same reasons as Evan. I think consumers who need to convert should be doing the conversion using their own routines (e.g. Chrome uses ones in our base/ module). PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 4:39 PM, Greg Spencer gspen...@google.com wrote: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). One thing many of us have found, across multiple projects, is that wchar_t is fraught with complication as soon as more than one platform is involved. wchar_t == UTF16 is a Windowsism (gcc defaults to 4 bytes, for example, and Lmumble gets stored in UCS-4, not UTF-16). Chrome started with more or less what you are suggesting, and we moved off of it after much pain. --Amanda --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 2:31 PM, Peter Kasting pkast...@google.com wrote: On Tue, Apr 28, 2009 at 1:39 PM, Greg Spencer gspen...@google.com wrote: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). I'm pretty strongly against this for the same reasons as Evan. I think consumers who need to convert should be doing the conversion using their own routines (e.g. Chrome uses ones in our base/ module). So, I was unable to find the conversion utilities in base that do the conversion to/from UTF8. What are they called? If I missed them (and I looked for a while before I gave up), then maybe they need to be more prominent? What is the danger here of being lazy? Is it that developers will unwittingly do expensive conversions? If so, I would expect that a member function called ToUTF8 would be just as much of a performance warning as a helper function called FilePathToUTF8, but be a heck of a lot more convenient (since it would not require the developer to create a local variable for use as a return value from the helper, and can be used as an argument to another library's functions). I can see the argument for not having a casting constructor that isn't from the platform native form, but in that case, a factory method called CreateFromUTF8 should be a sufficient warning to the developer that it might be expensive. -Greg. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 2:48 PM, Greg Spencer gspen...@google.com wrote: So, I was unable to find the conversion utilities in base that do the conversion to/from UTF8. What are they called? If I missed them (and I looked for a while before I gave up), then maybe they need to be more prominent? See base/string_util.h, UTF8ToUTF16() etc. What is the danger here of being lazy? Is it that developers will unwittingly do expensive conversions? Yes, partly because including dedicated helpers like this makes it sound as if the class is somehow special-cased or fastpathed to deal better with these than a generic converter would be. The other argument is simply that converting utf8 to utf16 is a generic sort of functionality that belongs in base/ or another similar general-purpose location, rather than specifically in FilePath. PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
(resend - arg) On Tue, Apr 28, 2009 at 2:47 PM, Greg Spencer gspen...@google.com wrote: On Tue, Apr 28, 2009 at 2:41 PM, Amanda Walker ama...@chromium.orgwrote: On Tue, Apr 28, 2009 at 4:39 PM, Greg Spencer gspen...@google.com wrote: 1) I'd like to add some explicit routines for converting to/from UTF8 and UTF16. While it's nice (and important) that FilePath uses the platform's native string, we've found that many third party libraries have made other assumptions, where they always expect UTF8 (char) or UTF16 (wchar_t) paths regardless of platform, and converting a FilePath to and from those forms is a platform-dependent exercise which should be centralized into the class (i.e. adding ToUTF8 and ToWide functions to the class, and explicit constructors that take each type). One thing many of us have found, across multiple projects, is that wchar_t is fraught with complication as soon as more than one platform is involved. wchar_t == UTF16 is a Windowsism (gcc defaults to 4 bytes, for example, and Lmumble gets stored in UCS-4, not UTF-16). Chrome started with more or less what you are suggesting, and we moved off of it after much pain. I understand those issues quite well (but I probably should call the conversion method ToUTF16, now that you mention it). And char* isn't necessarily UTF8 on all platforms either. OK, so what's the currently recommended path for converting to UTF16 or UTF8 from a FilePath? The biggest problem with this change is that it's not possible to do this conversion on Linux in a safe way. In Linux, there is no charset defined by the filesystem. Each filename is just a blob of bytes. Apps are supposed to respect an environment variable, but since this environment variable could change over time and be different from user to user, there's no reliable way to know what the charset is, so you can't convert from a FilePath on Linux to UTF8 or UTF16 unless you were the one who created the path to begin with. Erik --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 3:19 PM, Greg Spencer gspen...@google.com wrote: On Tue, Apr 28, 2009 at 3:11 PM, Erik Kay erik...@google.com wrote: The biggest problem with this change is that it's not possible to do this conversion on Linux in a safe way. And besides -- this problem isn't introduced by this change: it exists already because currently there's no safe way to convert, regardless of the API (since a consumer of a FilePath doesn't know what encoding it contains). -Greg. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 3:19 PM, Greg Spencer gspen...@google.com wrote: On Tue, Apr 28, 2009 at 3:11 PM, Erik Kay erik...@google.com wrote: The biggest problem with this change is that it's not possible to do this conversion on Linux in a safe way. In Linux, there is no charset defined by the filesystem. Each filename is just a blob of bytes. Apps are supposed to respect an environment variable, but since this environment variable could change over time and be different from user to user, there's no reliable way to know what the charset is, so you can't convert from a FilePath on Linux to UTF8 or UTF16 unless you were the one who created the path to begin with. But that's exactly the point. FilePath is the class that created the path to begin with. So it can know what the LC_*/LANG variables were was when it was created, and do the right conversion when you ask the FilePath to convert to UTF16. Also, if the developer calls something called FilePath::CreateFromUTF8, then it can know it was supposed to be UTF8 and remember that. If you created it yourself, that's fine. FilePaths aren't always created manually by users. They often are populated from system APIs where you can't know. See file_util* for some examples. So the problem is that if you add this API, people will mistakenly use the conversion functions when they can't be safe. I agree it sucks. I just don't know of a reasonable solution. Erik --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Changes to FilePath?
On Tue, Apr 28, 2009 at 3:26 PM, Erik Kay erik...@chromium.org wrote: On Tue, Apr 28, 2009 at 3:19 PM, Greg Spencer gspen...@google.com wrote: But that's exactly the point. FilePath is the class that created the path to begin with. So it can know what the LC_*/LANG variables were was when it was created, and do the right conversion when you ask the FilePath to convert to UTF16. Also, if the developer calls something called FilePath::CreateFromUTF8, then it can know it was supposed to be UTF8 and remember that. If you created it yourself, that's fine. FilePaths aren't always created manually by users. They often are populated from system APIs where you can't know. See file_util* for some examples. So the problem is that if you add this API, people will mistakenly use the conversion functions when they can't be safe. I agree it sucks. I just don't know of a reasonable solution. So there's currently no right way to do the conversion, but I still think that the FilePath constructor is probably in the best position to inspect LC_ALL, etc. and do as close to the right thing as possible. I doubt most Linux developers even think about this, and so the chances that they will implement anything other than assuming that it's ASCII are slim -- this would allow us to at least implement a baseline for them. Or would that just screw things up worse? Doesn't this mean that it's possible that the path manipulation routines fail for sufficiently odd encodings? (jis or something where an encoded char might include a /?) -Greg. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Need to run parts of WebCore in either the browser process or some browser helper process
Yes, yes, I know this is a horrible idea, but please hear me out :-) Last week, a couple of us (Darin F, Michael N, Jeremy M, and I) had lunch at Apple to talk to talk about sharing more code. HTML 5 brings with it a lot of APIs that reach outside of the top level browsing context boundary (i.e. the render process boundary in Chromium). We talked specifically about localStorage and appCache. Although I believe the following generalizes well for any such API, I recognize that there are some unique constraints for stuff like databases...so I'm not even going to talk about them right now. Anyhow... For a while now, I've looked at a bunch of ways to make localStorage multi-process aware, but really none of them have any hope except one: splitting localStorage into a frontend and backend. The frontend would be the portion in each renderer process that connects into the JS bindings. A single backend would store all the data and be shared by the frontends. Originally, my plan was to do this split and then write my own back end in the browser process, but there are several problems with this. From a technical standpoint, it's unclear how testing would work since our test_shell would be testing a different backend from what's in Chromium. It also means we have more code to maintain, and that code is completely off of WebKit's radar. It also makes Apple mad and Dimitri sad. So really, this doesn't seem like a good solution. Assuming the only viable solution is having several frontends talking to one backend (I'm confident this is true) and assuming having our own backend is not viable (this also seems true), then the only choice is for us to use the WebCore backend. We can't run this in any renderer process since the response times for browser-renderer communication are unbounded. So that leaves either the browser process or some browser helper process. Creating a helper process for the browser seems like a pretty interesting idea to me since there's already a lot of somewhat dangerous stuff running in the browser process. (The only thing I can remember right now is v8 for parsing .pac files, but I know there's more.) Basically, the helper process would be a sandboxed process where anything dangerous but not bound to a single renderer process would run. Ideally it would store little to no state so that the browser could easily restart it and resend any pending messages if it crashed. For localStorage, the backend (which is part of WebCore) would run there and all localStorage messages would be proxied through the browser process. The VFS could be used to retrieve file handles. The other option is to simply run part of WebCore's localStorage within the browser process. LocalStorage only depends on WTF, so this really isn't _that_ terrible of an idea. Thoughts? Anyhow, the WebKit guys we talked to like the idea of a split frontend/backend, especially if it means we'll continue sharing code. I believe Michael is going to be doing something similar for AppCache. J --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: webkit gyp conversion planned for tomorrow (Tuesday) evening
Hi All, Due to some last minute integration issues this will get pushed to 8pmPDT. Wish me luck! -BradN On Mon, Apr 27, 2009 at 4:51 PM, Steven Knight s...@chromium.org wrote: Brad Nelson is planning to close the tree some time tomorrow night (Tuesday 28 April) to land the conversion of webkit to gyp-based build. To get out from under playing constant catch-up with the rapidly-moving code base, it's possible that we'll leave the tree converted even if there are a small number--two or three?--of unit test failures. (By comparison, the last attempt to convert webkit had about five unit test failures plus a V8 performance regression -- since fixed -- that tipped the scales in favor of backing out the change.) If there are any outstanding problems, we'll send out an announcement--and could really use help making sure we weed out any known (and unkown) problems as quickly as possible. After we let the webkit conversion soak a bit, we'll schedule the conversion of the chrome/ subdirectory itself, based on how things have gone with webkit. WHILE WE'RE STILL IN TRANSITION: please continue to add new webkit/ and chrome/ files to *both* Visual Studio and the {webkit,chrome}.gyp files. We have just recently caught up both of these components with the last few weeks of added and deleted files. It helps immensely if we don't have to keep tracking these down by hand. If the planned timing is bad for you, let me or Brad know. Thanks, --SK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Tue, Apr 28, 2009 at 5:27 PM, Michael Nordman micha...@google.comwrote: In some sense we do have separate process in which to run sandboxed 'backend' code relevant to multiple renders if the need arises... the worker process. The way you stated this is a bit odd, but on the surface I guess this could be solved via special shared workers that ran WebKit code instead of javascript. That said, this means a lot of extra processes (for now, shared workers are each in their own process), this blocks localStorage on that, and actually makes the design more complicated. This might be worth exploring at some point, but not now. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
I have always felt like running the WebCore backend in the browser process with an IPC channel between it and the frontend was a really elegant design for this problem. And I never really understood why we weren't allowed to depend on WebCore (even indirectly) in the browser process. Is the idea that someday the browser and renderer processes might be separate binaries? - a On Tue, Apr 28, 2009 at 5:05 PM, Jeremy Orlow jor...@chromium.org wrote: Yes, yes, I know this is a horrible idea, but please hear me out :-) Last week, a couple of us (Darin F, Michael N, Jeremy M, and I) had lunch at Apple to talk to talk about sharing more code. HTML 5 brings with it a lot of APIs that reach outside of the top level browsing context boundary (i.e. the render process boundary in Chromium). We talked specifically about localStorage and appCache. Although I believe the following generalizes well for any such API, I recognize that there are some unique constraints for stuff like databases...so I'm not even going to talk about them right now. Anyhow... For a while now, I've looked at a bunch of ways to make localStorage multi-process aware, but really none of them have any hope except one: splitting localStorage into a frontend and backend. The frontend would be the portion in each renderer process that connects into the JS bindings. A single backend would store all the data and be shared by the frontends. Originally, my plan was to do this split and then write my own back end in the browser process, but there are several problems with this. From a technical standpoint, it's unclear how testing would work since our test_shell would be testing a different backend from what's in Chromium. It also means we have more code to maintain, and that code is completely off of WebKit's radar. It also makes Apple mad and Dimitri sad. So really, this doesn't seem like a good solution. Assuming the only viable solution is having several frontends talking to one backend (I'm confident this is true) and assuming having our own backend is not viable (this also seems true), then the only choice is for us to use the WebCore backend. We can't run this in any renderer process since the response times for browser-renderer communication are unbounded. So that leaves either the browser process or some browser helper process. Creating a helper process for the browser seems like a pretty interesting idea to me since there's already a lot of somewhat dangerous stuff running in the browser process. (The only thing I can remember right now is v8 for parsing .pac files, but I know there's more.) Basically, the helper process would be a sandboxed process where anything dangerous but not bound to a single renderer process would run. Ideally it would store little to no state so that the browser could easily restart it and resend any pending messages if it crashed. For localStorage, the backend (which is part of WebCore) would run there and all localStorage messages would be proxied through the browser process. The VFS could be used to retrieve file handles. The other option is to simply run part of WebCore's localStorage within the browser process. LocalStorage only depends on WTF, so this really isn't _that_ terrible of an idea. Thoughts? Anyhow, the WebKit guys we talked to like the idea of a split frontend/backend, especially if it means we'll continue sharing code. I believe Michael is going to be doing something similar for AppCache. J --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
Well, the question comes down to this: Can we trust running small amounts of WebCore in the browser process. Historically, the answer to running WebCore in the browser has been a resounding no. Can you please explain what you think has changed since such decisions were made (or why it's time to revisit such decisions)? My _feeling_ is that it would be OK with a security review, and my feeling is that a security review is possible since the code is fairly isolated, but that's just my opinion. I don't have anything terribly solid to back it up. J On Tue, Apr 28, 2009 at 5:46 PM, Michael Nordman micha...@chromium.orgwrote: On Tue, Apr 28, 2009 at 5:38 PM, Jeremy Orlow jor...@chromium.org wrote: On Tue, Apr 28, 2009 at 5:27 PM, Michael Nordman micha...@google.com wrote: In some sense we do have separate process in which to run sandboxed 'backend' code relevant to multiple renders if the need arises... the worker process. The way you stated this is a bit odd, but on the surface I guess this could be solved via special shared workers that ran WebKit code instead of javascript. That said, this means a lot of extra processes (for now, shared workers are each in their own process), this blocks localStorage on that, and actually makes the design more complicated. This might be worth exploring at some point, but not now. Let me re-iterate my main point... i dont think we need to sandbox the localstorage or appcache backend code, so we should be able to run that directly in the browser process. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
+ chromium-dev Can you please explain what you think has changed since such decisions were made (or why it's time to revisit such decisions)? I don't think there was code in webcore suitable for this purpose before... html parsing, javascript,sql interpretting... all dangerous from a security point of view (acting in very complex ways on untrusted web content). The backend logic for these new features aren't like that. Its not so much that its webcore code is untrusted, as much as the data it operates on is untrusted. I have always felt like running the WebCore backend in the browser was elegant Yea, but we need a webcore backend to run :) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Tue, Apr 28, 2009 at 6:26 PM, Evan Martin e...@chromium.org wrote: On Tue, Apr 28, 2009 at 5:59 PM, Aaron Boodman a...@chromium.org wrote: Is the idea that someday the browser and renderer processes might be separate binaries? Though this shouldn't drive your decision, about 50% of our code weight (at least by one metric) is WebKit. You could imagine that the browser process doesn't need a copy of that, and that the renderer process doesn't need a copy of all the browser process stuff (network stack, UI goop). On Linux, startup time is heavily affected by the cost of the dynamic linker pulling in GTK, which we only would need from the browser process if we had separate binaries. However, OS-level page sharing may mean having separate binaries doesn't actually help too much here. This is definitely something to keep in mind, but shouldn't we be able to just compile in the bits of webkit we care about? Yeah, we'll have 2 copies of WTF, but the rest should be pretty separate. On Tue, Apr 28, 2009 at 6:22 PM, Aaron Boodman a...@chromium.org wrote: I fear that this is really a question for Darin, who is on vacation. Darin was there on that lunch and was actually the one who first suggested running parts of WebCore in the browser to me during a 1:1. :-) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: depricating CppBindingClass
Yup. I am not saying we need to get rid of all of it immediately; just put some comments in the header so that we don't use it *more*. Mike On Tue, Apr 28, 2009 at 6:38 PM, Feng Qian fq...@google.com wrote: CppBindingClass was started for test_shell if I remember correctly, and I think the functionality can be replaced by using NPAPI instead. I had an implementation of replacing CppBindingClass by NPAPI in test_shell (to avoid two implementations for JSC and V8), I don't remember where the code went. On Tue, Apr 28, 2009 at 6:15 PM, Mike Belshe mbel...@google.com wrote: I've felt this way for a while, and I figured I'd throw it out to see if we all agree. I'm hoping we can deprecate CppBindingClass. At this point, most extensions are better served (I think!) via the v8::extension facility than the CppBindingObject. If there are cases where this isn't true, I'd happily work on v8::extension to make it so it could be. If nobody objects, I hope we can agree to stop adding any new CppBindingClass based bindings, and then over time get rid of CppBindingObject altogether. Anyone feel strongly in opposition? Mike --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Tree is closed - v8 bustage.
The tree is closed. Several people are seeing the following error when compiling on Windows: Error 1 fatal error C1083: Cannot open source file: '..\..\..\chrome\Debug\obj\global_intermediate\libraries.cc': No such file or directory c1xx Error 2 fatal error LNK1181: cannot open input file '..\..\..\chrome\debug\lib\v8_nosnapshot.lib' mksnapshot Error 3 Error result 127 returned from 'C:\Windows\SysWow64\cmd.exe'. Project It is not clear to me when this started happening. It seems to have shown up mysteriously for a while on the XP release buildbot at r14784 but then mysteriously disappeared again at 14787. Reverting various changes around this range does not help. The tree should not be reopened until this is resolved. -Ben --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Tree is closed - v8 bustage.
It looks like the missing dependency did the trick: both clobber of Chromium builder and clean local build work now. Can haz open tree? :DG On Tue, Apr 28, 2009 at 9:04 PM, Dimitri Glazkov dglaz...@chromium.org wrote: Testing the fix ... :DG On Tue, Apr 28, 2009 at 8:36 PM, Bradley Nelson bradnel...@google.com wrote: Looking into a fix, this may be a missing dependency from v8_snapshot on js2c. It would non-deterministically pass with incredibuild. -BradN On Tue, Apr 28, 2009 at 8:32 PM, Feng Qian f...@chromium.org wrote: libraries.cc is a generated file, did you try a clean build? On Tue, Apr 28, 2009 at 7:56 PM, Ben Goodger (Google) b...@chromium.org wrote: The tree is closed. Several people are seeing the following error when compiling on Windows: Error 1 fatal error C1083: Cannot open source file: '..\..\..\chrome\Debug\obj\global_intermediate\libraries.cc': No such file or directory c1xx Error 2 fatal error LNK1181: cannot open input file '..\..\..\chrome\debug\lib\v8_nosnapshot.lib' mksnapshot Error 3 Error result 127 returned from 'C:\Windows\SysWow64\cmd.exe'. Project It is not clear to me when this started happening. It seems to have shown up mysteriously for a while on the XP release buildbot at r14784 but then mysteriously disappeared again at 14787. Reverting various changes around this range does not help. The tree should not be reopened until this is resolved. -Ben --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Tree is closed - v8 bustage.
Yes I think so.-BradN On Tue, Apr 28, 2009 at 9:20 PM, Dimitri Glazkov dglaz...@chromium.orgwrote: It looks like the missing dependency did the trick: both clobber of Chromium builder and clean local build work now. Can haz open tree? :DG On Tue, Apr 28, 2009 at 9:04 PM, Dimitri Glazkov dglaz...@chromium.org wrote: Testing the fix ... :DG On Tue, Apr 28, 2009 at 8:36 PM, Bradley Nelson bradnel...@google.com wrote: Looking into a fix, this may be a missing dependency from v8_snapshot on js2c. It would non-deterministically pass with incredibuild. -BradN On Tue, Apr 28, 2009 at 8:32 PM, Feng Qian f...@chromium.org wrote: libraries.cc is a generated file, did you try a clean build? On Tue, Apr 28, 2009 at 7:56 PM, Ben Goodger (Google) b...@chromium.org wrote: The tree is closed. Several people are seeing the following error when compiling on Windows: Error 1 fatal error C1083: Cannot open source file: '..\..\..\chrome\Debug\obj\global_intermediate\libraries.cc': No such file or directory c1xx Error 2 fatal error LNK1181: cannot open input file '..\..\..\chrome\debug\lib\v8_nosnapshot.lib' mksnapshot Error 3 Error result 127 returned from 'C:\Windows\SysWow64\cmd.exe'. Project It is not clear to me when this started happening. It seems to have shown up mysteriously for a while on the XP release buildbot at r14784 but then mysteriously disappeared again at 14787. Reverting various changes around this range does not help. The tree should not be reopened until this is resolved. -Ben --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---