[chromium-dev] Re: Passing around NativeViews between processes.

[Dean McNamee]

Hey Darin,

The long story is pretty long, but I can explain it if you're
interested in all of the details.  The medium sized version:

Gtk implements a non-trivial amount of code for handling the X event
forwarding and management of the xembed protocol, which is how modern
plugin embedding on Linux works.  In order to take advantage of this
code, we need a GtkWidget (in this case a GtkSocket) in the Gtk
hierarchy.  If we were to do things like move the backing X window
directly (without asking Gtk to move it), Gtk is just going to "move
it back" on the next layout, since Gtk is managing its position.

Evan had originally implemented a custom container widget that prevent
Gtk from laying out the plugin windows.  However, it turned out layout
is an important step, as the GtkSocket uses these events to send an
expose message to the plugin.  There is a balance here, the more we
start trying to sidestep how Gtk wants things to be done, the more
we're skipping over important code thats required and expected by the
embedded plugins.

I have a strategy that I think will work for plugins.  It will mean
that the window can be create and managed from only the same process.
If the renderer needs to tell the browser to move a window, we need to
move the GtkWidget, and not the X window.  We can't work in X window
IDs here, we need the GtkWidget* so we can move it within its parent
container.  We could put the X id in a map, and pull out the
GtkWidget*, but then the id is arbitrary.  We could also iterate over
all of the plugin windows, asking for their X window id until it
matches, that's an O(n) search which would probably be ok, but it's
not very elegant.

I believe the situation is similar on the Mac.  I'm not sure the
current design of passing HWNDs between processes conceptually extends
beyond win32.

My impression so far from a handful of conversions is there isn't a
big willingness to change here.  If that's the case, we will just have
to come up with some hacks that allows the win32 design work on the
other platforms.

Thanks
-- dean

On Fri, Feb 13, 2009 at 4:08 AM, Darin Fisher  wrote:
> I had assumed that NativeWindowID should just be the X window ID in our
> Linux build.  That is what NPAPI uses (see NPWindow), and so it is what we
> will need to pass around.
> I think you can initialize a GdkWindow from a X window ID.  I don't think we
> should worry about the case where GDK is not running on top of X :-)
> It would probably take some considerable work to stop passing HWNDs / X
> window IDs through the renderer on behalf of plugins.
> -Darin
>
>
> On Wed, Feb 11, 2009 at 6:51 AM, Dean McNamee  wrote:
>>
>> On Windows, pretty much everything deals in HWNDs, which is an integer
>> index into a kernel handle table.  This means HWNDs, which are just
>> integer window ids, are good across processes.  The fact that all of
>> the Windows APIs work in HWNDs, means that it is the primitive for
>> NativeWindow and NativeView on Windows.
>>
>> On Linux, we have a similar concept, the X window id.  However, our
>> NativeWindow is currently a GtkWindow, and NativeView is a GtkWidget.
>> Both a GtkWindow (this is the top level application window) and a
>> GtkWidget can be backed by a GdkWindow, which encapsulates the X
>> window id.  Gtk is a cross-platform toolkit, which is one of the
>> reasons you don't deal w/ the window ids directly.
>>
>> A recent example of this I've encountered is
>> DidMove(WebPluginGeometry), which has a NativeView handle.  This is
>> not going to be IPC-able, as this would be a pointer to a GtkWidget
>> instance, which obviously won't be good across processes.  In order to
>> have something that would be, we would need to do GtkWidget ->
>> GdkWindow -> x window id.  Then on the other side we'd have to create
>> a new GdkWindow around the x window id.
>>
>> It seems we might need some new abstractions, since on Linux both
>> NativeView and NativeWindow are pointers, not integers that are valid
>> between processes.  It seems like we need some NativeWindowId
>> abstraction.  Hey, I just looked at the code, I noticed someone added
>> a NativeViewId type and conversion methods, so we already have the
>> abstraction.  We just need to start using them.
>>
>> I'm not sure this is enough security-wise though.  It is scary to
>> allow the to renderer send messages to the browser, telling it to do
>> operations on arbitrary windows.  I think this is where Adam mentioned
>> something like HMAC, or we could do some sort of handle type thing to
>> implement security.  This might be a bit tricky, depending where the
>> window is created an used (we have 3 processes involved, plugin,
>> browser, and renderer).  It would be nice if we could also enforce IDs
>> per-renderer process, and not just HMAC'd to any renderer.
>>
>> In some of these cases, we are probably just using the HWND out of
>> convenience so we don't have to keep state.  It seems like it's
>> probably a mess to try to duplicate the state in

[chromium-dev] Re: Passing around NativeViews between processes.

[Amanda Walker]

On Fri, Feb 13, 2009 at 1:34 AM, Dean McNamee  wrote:
> I believe the situation is similar on the Mac.  I'm not sure the
> current design of passing HWNDs between processes conceptually extends
> beyond win32.

The situation on the Mac is similar.  There's an internal window
server ID for each window; this is unique per user session (roughly,
per "active desktop", of which each tree of processes can talk to only
one).  This can be sent around between processes, but it's not what
any of the UI APIs take.  There's enough state in the per-process
window structures that it's difficult to just pass window IDs around
as a reference to a window (for example, the public API only support
using this to read window properties, not actually for manipulating
windows or their contents).  While we may decide it's worth using SPIs
for performance reasons in the long run, it would be nice to start
with an above-board implementation.  In addition, things like iframes
and NPAPI plugins don't actually get their own windows in the HWND
sense on the Mac, even in the single-process case (such as Safari), so
the win32 design is already a bit of a mismatch.  An ID that can be
treated as an HWND under Windows, and an index into a table or map on
the Mac and/or Linux would probably work fine.  It's just that a lot
of the windows code currently doesn't distinguish between the various
uses of HWNDs, and other platforms may need to.

--Amanda

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Now we can call it Chrome!

[eduyanez]

Interesting... I just can't wait to use it as my main browser on the
Mac too!!!

:)


On Feb 12, 10:42 pm, Avi Drissman  wrote:
> For those of you who aren't at Google, let me try again...
>
> http://homepage.mac.com/avidrissman/extimgs/st.png
>
> Avi
>
> 2009/2/12 Avi Drissman 
>
>
>
> >http://www.corp.google.com/~avi/randompics/st.png
>
> > Avi
--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Passing around NativeViews between processes.

[Darin Fisher]

On Fri, Feb 13, 2009 at 7:40 AM, Amanda Walker  wrote:

> On Fri, Feb 13, 2009 at 1:34 AM, Dean McNamee  wrote:
> > I believe the situation is similar on the Mac.  I'm not sure the
> > current design of passing HWNDs between processes conceptually extends
> > beyond win32.
>
> The situation on the Mac is similar.  There's an internal window
> server ID for each window; this is unique per user session (roughly,
> per "active desktop", of which each tree of processes can talk to only
> one).  This can be sent around between processes, but it's not what
> any of the UI APIs take.  There's enough state in the per-process
> window structures that it's difficult to just pass window IDs around
> as a reference to a window (for example, the public API only support
> using this to read window properties, not actually for manipulating
> windows or their contents).  While we may decide it's worth using SPIs
> for performance reasons in the long run, it would be nice to start
> with an above-board implementation.  In addition, things like iframes
> and NPAPI plugins don't actually get their own windows in the HWND
> sense on the Mac, even in the single-process case (such as Safari), so
> the win32 design is already a bit of a mismatch.  An ID that can be
> treated as an HWND under Windows, and an index into a table or map on
> the Mac and/or Linux would probably work fine.  It's just that a lot
> of the windows code currently doesn't distinguish between the various
> uses of HWNDs, and other platforms may need to.
>
> --Amanda
>


I thought on the mac, we were just going to support windowless mode.  This
discussion of passing HWNDs or XwindowIDs only pertains to windowed mode
plugins.

Is there a windowed mode for mac that we need to worry about?

-Darin

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Passing around NativeViews between processes.

[Darin Fisher]

Ah, very interesting.  I had not paid attention to gtk2xtbin.c when blizzard
added that back in '02.  Originally, people just managed all of the X level
stuff manually for plugins.  I agree that we should do like Mozilla and use
GtkSocket.

-Darin



On Fri, Feb 13, 2009 at 1:34 AM, Dean McNamee  wrote:

> Hey Darin,
>
> The long story is pretty long, but I can explain it if you're
> interested in all of the details.  The medium sized version:
>
> Gtk implements a non-trivial amount of code for handling the X event
> forwarding and management of the xembed protocol, which is how modern
> plugin embedding on Linux works.  In order to take advantage of this
> code, we need a GtkWidget (in this case a GtkSocket) in the Gtk
> hierarchy.  If we were to do things like move the backing X window
> directly (without asking Gtk to move it), Gtk is just going to "move
> it back" on the next layout, since Gtk is managing its position.
>
> Evan had originally implemented a custom container widget that prevent
> Gtk from laying out the plugin windows.  However, it turned out layout
> is an important step, as the GtkSocket uses these events to send an
> expose message to the plugin.  There is a balance here, the more we
> start trying to sidestep how Gtk wants things to be done, the more
> we're skipping over important code thats required and expected by the
> embedded plugins.
>
> I have a strategy that I think will work for plugins.  It will mean
> that the window can be create and managed from only the same process.
> If the renderer needs to tell the browser to move a window, we need to
> move the GtkWidget, and not the X window.  We can't work in X window
> IDs here, we need the GtkWidget* so we can move it within its parent
> container.  We could put the X id in a map, and pull out the
> GtkWidget*, but then the id is arbitrary.  We could also iterate over
> all of the plugin windows, asking for their X window id until it
> matches, that's an O(n) search which would probably be ok, but it's
> not very elegant.
>
> I believe the situation is similar on the Mac.  I'm not sure the
> current design of passing HWNDs between processes conceptually extends
> beyond win32.
>
> My impression so far from a handful of conversions is there isn't a
> big willingness to change here.  If that's the case, we will just have
> to come up with some hacks that allows the win32 design work on the
> other platforms.
>
> Thanks
> -- dean
>
> On Fri, Feb 13, 2009 at 4:08 AM, Darin Fisher  wrote:
> > I had assumed that NativeWindowID should just be the X window ID in our
> > Linux build.  That is what NPAPI uses (see NPWindow), and so it is what
> we
> > will need to pass around.
> > I think you can initialize a GdkWindow from a X window ID.  I don't think
> we
> > should worry about the case where GDK is not running on top of X :-)
> > It would probably take some considerable work to stop passing HWNDs / X
> > window IDs through the renderer on behalf of plugins.
> > -Darin
> >
> >
> > On Wed, Feb 11, 2009 at 6:51 AM, Dean McNamee 
> wrote:
> >>
> >> On Windows, pretty much everything deals in HWNDs, which is an integer
> >> index into a kernel handle table.  This means HWNDs, which are just
> >> integer window ids, are good across processes.  The fact that all of
> >> the Windows APIs work in HWNDs, means that it is the primitive for
> >> NativeWindow and NativeView on Windows.
> >>
> >> On Linux, we have a similar concept, the X window id.  However, our
> >> NativeWindow is currently a GtkWindow, and NativeView is a GtkWidget.
> >> Both a GtkWindow (this is the top level application window) and a
> >> GtkWidget can be backed by a GdkWindow, which encapsulates the X
> >> window id.  Gtk is a cross-platform toolkit, which is one of the
> >> reasons you don't deal w/ the window ids directly.
> >>
> >> A recent example of this I've encountered is
> >> DidMove(WebPluginGeometry), which has a NativeView handle.  This is
> >> not going to be IPC-able, as this would be a pointer to a GtkWidget
> >> instance, which obviously won't be good across processes.  In order to
> >> have something that would be, we would need to do GtkWidget ->
> >> GdkWindow -> x window id.  Then on the other side we'd have to create
> >> a new GdkWindow around the x window id.
> >>
> >> It seems we might need some new abstractions, since on Linux both
> >> NativeView and NativeWindow are pointers, not integers that are valid
> >> between processes.  It seems like we need some NativeWindowId
> >> abstraction.  Hey, I just looked at the code, I noticed someone added
> >> a NativeViewId type and conversion methods, so we already have the
> >> abstraction.  We just need to start using them.
> >>
> >> I'm not sure this is enough security-wise though.  It is scary to
> >> allow the to renderer send messages to the browser, telling it to do
> >> operations on arbitrary windows.  I think this is where Adam mentioned
> >> something like HMAC, or we could do some sort of handle 

[chromium-dev] Re: Passing around NativeViews between processes.

[Amanda Walker]

On Fri, Feb 13, 2009 at 9:27 AM, Darin Fisher  wrote:
> I thought on the mac, we were just going to support windowless mode.  This
> discussion of passing HWNDs or XwindowIDs only pertains to windowed mode
> plugins.
> Is there a windowed mode for mac that we need to worry about?

There are a few of independent issues:
- The current plugin code makes some assumptions about being able to
pass HWNDs around.  So far, working around those on the Mac hasn't
been too difficult, but having a "native view ID" as Dean proposes
looks like it would be handy, though not critical.

- Mac NPAPI only supports "windowless" mode in the window-ownership
sense, but still supports two variations (which differ in how layering
is handled, in order to emulate "windowed" mode from a layering/event
dispatching standpoint), and technically requires a native window
reference for both windowed and windowless mode.  So far, flash
doesn't seem to mind if that's NULL as long as I force it into
windowless mode by overriding the "wmode" plugin parameter, but other
plugins may, so it's on my radar.  Plugins (such as flash) that want
to switch to full screen mode will also need native windows, though
that's a can of worms I'm not even looking at until normal windowless
mode is working.

- for video, 3D, or anything else that needs/wants an accelerated
drawing surface, or if we need/want to avoid bouncing everything
through a shared memory buffer, we'll need to be able to create
overlay windows that are locked to the browser window.  This gives the
effect of a subwindow, but using public APIs it can't be accessed by a
different process.  The Earth plugin would gain a 2X increase in frame
rate (according to the Earth engineers last time I talked to them) if
we can do this, and Client3D currently requires it to work at all.  As
with the previous item, I'm not actively worrying about it yet, but
we'll need to solve it eventually.

--Amanda

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Passing around NativeViews between processes.

[Dean McNamee]

On Fri, Feb 13, 2009 at 6:50 PM, Amanda Walker  wrote:
> On Fri, Feb 13, 2009 at 9:27 AM, Darin Fisher  wrote:
>> I thought on the mac, we were just going to support windowless mode.  This
>> discussion of passing HWNDs or XwindowIDs only pertains to windowed mode
>> plugins.
>> Is there a windowed mode for mac that we need to worry about?
>
> There are a few of independent issues:
> - The current plugin code makes some assumptions about being able to
> pass HWNDs around.  So far, working around those on the Mac hasn't
> been too difficult, but having a "native view ID" as Dean proposes
> looks like it would be handy, though not critical.
>
> - Mac NPAPI only supports "windowless" mode in the window-ownership
> sense, but still supports two variations (which differ in how layering
> is handled, in order to emulate "windowed" mode from a layering/event
> dispatching standpoint), and technically requires a native window
> reference for both windowed and windowless mode.  So far, flash
> doesn't seem to mind if that's NULL as long as I force it into
> windowless mode by overriding the "wmode" plugin parameter, but other
> plugins may, so it's on my radar.  Plugins (such as flash) that want
> to switch to full screen mode will also need native windows, though
> that's a can of worms I'm not even looking at until normal windowless
> mode is working.
>
> - for video, 3D, or anything else that needs/wants an accelerated
> drawing surface, or if we need/want to avoid bouncing everything
> through a shared memory buffer, we'll need to be able to create
> overlay windows that are locked to the browser window.  This gives the
> effect of a subwindow, but using public APIs it can't be accessed by a
> different process.  The Earth plugin would gain a 2X increase in frame
> rate (according to the Earth engineers last time I talked to them) if
> we can do this, and Client3D currently requires it to work at all.  As

You must mean canvas3d, I don't think that's a big deal yet as it's
still just a Mozilla prototype.  It would be nice to try to make
things like that possible in our design though, anything with opengl
is probably going to require a window?

I wasn't so much trying to use Mac as an argument for how things
should be on this specific instance, but just so try to get everyone
informed about how the other platforms work, and see if it's possible
to abstraction something out that works for everyone.

> with the previous item, I'm not actively worrying about it yet, but
> we'll need to solve it eventually.
>
> --Amanda
>

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Passing around NativeViews between processes.

[Amanda Walker]

On Fri, Feb 13, 2009 at 10:14 AM, Dean McNamee  wrote:
> You must mean canvas3d, I don't think that's a big deal yet as it's
> still just a Mozilla prototype.  It would be nice to try to make
> things like that possible in our design though, anything with opengl
> is probably going to require a window?

Right, that's what I'm thinking about in the long term.  Anything that
needs to use the GPU (3D drawing, accelerated video decoding, etc.) on
a Mac will need access to a window.  Now, a plugin can always create
an offscreen window, do whatever it wants to do with the GPU there,
pull the bits down from the video card, and then send them via shared
memory up to the renderer.  I believe this is what the Google Earth
plugin for the Mac currently does, but as I noted in my last message,
this imposes a noticeable frame rate penalty.  More generally, this
seems wasteful of RAM and power.

Of course, if Apple eventually exposes an API that lets, for example,
two processes share an OpenGL drawable, that would solve all sorts of
problems :-).

> I wasn't so much trying to use Mac as an argument for how things
> should be on this specific instance, but just so try to get everyone
> informed about how the other platforms work, and see if it's possible
> to abstraction something out that works for everyone.

Fair enough.  I didn't mean to derail that line of discussion.

--Amanda

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---