Re: [g-a-devel] GtkSocket/GtkPlug accessibility

[Samuel Thibault]

Samuel Thibault, le jeu. 22 août 2019 18:06:52 +0200, a ecrit:
> Samuel Thibault, le jeu. 22 août 2019 13:16:44 +0200, a ecrit:
> > Now, thinking more about this, maybe there is a way to make all of this
> > transparent to applications using GtkSocket/GtkPlug:
> > 
> > - introduce the at-spi objects I mentioned in my Aug 14th mail
> > - make GtkPlug expose the at-spi path as a _NET_WM property.
> > - make gtk_socket_add_window try to read it, and if it exists, call
> >   atk_socket_embed on it
> > 
> > That should be working both for the mate-panel implementation mentioned
> > above, and for tray icons. And for anything that uses GtkPlug/Socket.
> 
> Here is what is looks like, it looks quite fine in the end, what do
> people think?
> 
> Arguably, the AtkPlugExt part could be stuffed into AtkPlug:

That part got committed to atk. I eventually submitted it to:

https://gitlab.gnome.org/GNOME/gtk/merge_requests/1147

Samuel
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: [g-a-devel] GtkSocket/GtkPlug accessibility

[Samuel Thibault]

Samuel Thibault, le jeu. 22 août 2019 13:16:44 +0200, a ecrit:
> Now, thinking more about this, maybe there is a way to make all of this
> transparent to applications using GtkSocket/GtkPlug:
> 
> - introduce the at-spi objects I mentioned in my Aug 14th mail
> - make GtkPlug expose the at-spi path as a _NET_WM property.
> - make gtk_socket_add_window try to read it, and if it exists, call
>   atk_socket_embed on it
> 
> That should be working both for the mate-panel implementation mentioned
> above, and for tray icons. And for anything that uses GtkPlug/Socket.

Here is what is looks like, it looks quite fine in the end, what do
people think?

Arguably, the AtkPlugExt part could be stuffed into AtkPlug: currently
it does not actually implement the ref_child/get_n_children interfaces
so subclasses have to implement it anyway. So we could as well have
a default implementation which returns 0 children by default, and 1
child if it was built by using a new dedicated atk_plug_new_child(child)
allocator which takes the child to be used as parameter. AIUI doing this
can't break existing usage, and allows to have it shared by all toolkits
needing it instead of putting it in gtk only.

Samuel
Index: gtk3/gtk/a11y/Makefile.inc
===
--- gtk3.orig/gtk/a11y/Makefile.inc
+++ gtk3/gtk/a11y/Makefile.inc
@@ -30,6 +30,7 @@ a11y_h_sources =  \
a11y/gtknotebookaccessible.h\
a11y/gtknotebookpageaccessible.h\
a11y/gtkpanedaccessible.h   \
+   a11y/gtkplugaccessible.h\
a11y/gtkpopoveraccessible.h \
a11y/gtkprogressbaraccessible.h \
a11y/gtkradiobuttonaccessible.h \
@@ -39,6 +40,7 @@ a11y_h_sources =  \
a11y/gtkscaleaccessible.h   \
a11y/gtkscalebuttonaccessible.h \
a11y/gtkscrolledwindowaccessible.h  \
+   a11y/gtksocketaccessible.h  \
a11y/gtkspinbuttonaccessible.h  \
a11y/gtkspinneraccessible.h \
a11y/gtkstatusbaraccessible.h   \
@@ -104,6 +106,7 @@ a11y_c_sources =\
a11y/gtknotebookaccessible.c\
a11y/gtknotebookpageaccessible.c\
a11y/gtkpanedaccessible.c   \
+   a11y/gtkplugaccessible.c\
a11y/gtkpopoveraccessible.c \
a11y/gtkprogressbaraccessible.c \
a11y/gtkradiobuttonaccessible.c \
@@ -113,6 +116,7 @@ a11y_c_sources =\
a11y/gtkscaleaccessible.c   \
a11y/gtkscalebuttonaccessible.c \
a11y/gtkscrolledwindowaccessible.c  \
+   a11y/gtksocketaccessible.c  \
a11y/gtkspinbuttonaccessible.c  \
a11y/gtkspinneraccessible.c \
a11y/gtkstatusbaraccessible.c   \
Index: gtk3/gtk/gtk-a11y.h
===
--- gtk3.orig/gtk/gtk-a11y.h
+++ gtk3/gtk/gtk-a11y.h
@@ -57,6 +57,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -67,6 +68,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
Index: gtk3/gtk/a11y/gtkplugaccessible.c
===
--- /dev/null
+++ gtk3/gtk/a11y/gtkplugaccessible.c
@@ -0,0 +1,172 @@
+/* GTK+ - accessibility implementations
+ * Copyright 2019 Samuel Thibault 
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library. If not, see .
+ */
+
+#include "config.h"
+
+#include 
+#include "gtkplugaccessible.h"
+
+/* We can not make GtkPlugAccessible inherit both from GtkContainerAccessible
+ * and GtkPlug, so we make it the atk child of an AtkPlug */
+
+#ifndef HAVE_ATK_PLUG_NEW_CHILD
+
+// This could be stuffed into AtkPlug?
+
+#define ATK_TYPE_PLUG_EXT atk_plug_ext_get_type ()
+#define ATK_PLUG_EXT(obj) (G_TYPE_CHECK_INSTANCE_CAST ((obj), 
ATK_TYPE_PLUG_EXT_PLUG, AtkPlugExt))
+
+GType atk_plug_ext_get_type (void);
+
+typedef struct _AtkPlugExt AtkPlugExt;
+typedef struct _AtkPlugExtClass AtkPlugExtClass;
+
+AtkObject *atk_plug_ext_new (AtkObject *child);
+
+
+
+
+struct _AtkPlugExt
+{
+  AtkPlug parent;
+
+  /

Re: [g-a-devel] GtkSocket/GtkPlug accessibility

[Samuel Thibault]

Hello,

Samuel Thibault, le mer. 14 août 2019 17:44:43 +0200, a ecrit:
> I have add AtkSocket/AtkPlug support to mate-panel here:
> 
> https://salsa.debian.org/hypra-team/mate-panel/blob/stretch-applets/debian/patches/1001_atk-socket-plug

I'm now looking at the support for tray icons, and the situation is more
tricky and makes me think we really need to stuff the support inside
GtkPlug.

Tray icon docking seems to be requested through a
_NET_SYSTEM_TRAY_OPCODE XClientMessage event, that transmits the plug
Window id for the socket to embed it. The event is too small to also
embed the at-spi path, but we could set a property on the Window to
contain it.

Each and every applet creating tray icons could add that support,
but we'd rather have a shared implementation. Applets happen to use
GtkStatusIcon which contains a GtkTrayIcon, which inherits from GtkPlug.
So, again it would make sense to integrate the addition of an accessible
at the GtkPlug level, and advertise it with a _NET_WM property there.

Now, thinking more about this, maybe there is a way to make all of this
transparent to applications using GtkSocket/GtkPlug:

- introduce the at-spi objects I mentioned in my Aug 14th mail
- make GtkPlug expose the at-spi path as a _NET_WM property.
- make gtk_socket_add_window try to read it, and if it exists, call
  atk_socket_embed on it

That should be working both for the mate-panel implementation mentioned
above, and for tray icons. And for anything that uses GtkPlug/Socket.

Is there anything I might be missing?

Samuel
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: [g-a-devel] GtkSocket/GtkPlug accessibility

[Samuel Thibault]

Samuel Thibault, le mer. 14 août 2019 17:44:43 +0200, a ecrit:
> What I had to do is:
> 
> * on the socket side:
[...]
> 
>   - create PanelAtkSocket, a subclass of AtkSocket that makes its
> get_extents method call the parent's get_extents method.

FTR, this part was integrated into AtkSocket with

https://gitlab.gnome.org/GNOME/atk/commit/4b5ed8630c5ae9ef37884c4199da8463b7e5a127

Samuel
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: [g-a-devel] GtkSocket/GtkPlug accessibility

[Samuel Thibault]

Samuel Thibault, le mer. 14 août 2019 17:44:43 +0200, a ecrit:
> Maybe another way would be to make GtkSocket/GtkPlug directly create
> AtkSocket/AtkPlug, and override the get_accessible method to return the
> AtkSocket/AtkPlug directly?  I didn't see such approach used in gtk
> so I'm unsure if that's supposed to work?

I just gave it a small try with GtkPlug, and the problem is that it
doesn't allow to inherit the underlying methods:

When atk_object_get_n_accessible_children calls get_n_children, we'd
want it to directly call gtk_container_accessible_get_n_children and
let that take the information from the GtkContainer behind GtkPlug. But
gtk_container_accessible_get_n_children assumes that the parameter is a
GtkContainerAccessible, not an AtkPlug (multi-heritage issue again).  So
we'd have to reimplement the get_n_children method anyway to reach the
proper GtkWidget.

So AIUI it's actually simpler to just indeed introduce the intermediate
accessible objects?  We could probably stuff that into some
GtkSocketAccessible/GtkPlugAccessible?  The application would
then only have to use some gtk_plug_accessible_get_id() and
gtk_socket_accessible_embed().

Samuel
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: [g-a-devel] GtkSocket/GtkPlug accessibility

[Samuel Thibault]

Hello,

I have add AtkSocket/AtkPlug support to mate-panel here:

https://salsa.debian.org/hypra-team/mate-panel/blob/stretch-applets/debian/patches/1001_atk-socket-plug

I am however wondering whether the complexity is needed and/or whether
some of it could already be implemented in gtk.

What I had to do is:

* on the socket side:

  - create PanelSocket, a subclass of GtkSocket that creates an
AtkSocket (actually a PanelAtkSocket, see below) and makes itself
its accessible parent.

  - create PanelSocketAccessible, the corresponding subclass of
GetContainerAccessible that overrides get_n_children and ref_child
to return the AtkSocket as child of the accessible of the
PanelSocket.

  - create PanelAtkSocket, a subclass of AtkSocket that makes its
get_extents method call the parent's get_extents method.

* on the plug side:

  - create PanelAPlug, a subclass of GtkPlug that creates an AtkPlug
(actually a PanelAtkPlug, see below) and set it as accessible
parent.

  - create PanelAtkPlug, a subclass of AtkPlug that makes it parent of
the root of the applet, and calls ref_state_set from the root of the
applet to implement the ref_state_set method.


All of this is not specific to mate-panel so might be done by
gtk itself, and the complexity seems useless: normally AtkSocket
should directly be the accessible of GtkSocket, and ditto for
AtkPlug/GtkPlug. But AFAICT:

- To make AtkSocket the accessible of GtkSocket one would make GtkSocket
  call gtk_widget_class_set_accessible_type(GTK_TYPE_SOCKET_ACCESSIBLE).

- But gtk_widget_class_set_accessible_type checks that
  GTK_TYPE_SOCKET_ACCESSIBLE is "a subtype of the type used for
  accessibles of the parent class".
  
- Since the parent class of GtkSocket is GtkContainer,
  GtkSocketAccessible would have to be a parent of
  GtkContainerAccessible, and eventually of GtkAccessible.

- But GtkAccessible contains an AtkObject, while AtkSocket extends
  AtkObject (to store the plug id).

- Put another way, we'd want multi-heritage for GtkSocketAccessible:
  both GtkContainerAccessible and AtkSocket ; and AFAIK this is not
  supported by gobject (which makes sense: the AtkObject structure
  within GtkAccessible can't magically be extended to contain
  embedded_plug_id).

ditto for AtkPlug/GtkPlug.


Maybe another way would be to make GtkSocket/GtkPlug directly create
AtkSocket/AtkPlug, and override the get_accessible method to return the
AtkSocket/AtkPlug directly?  I didn't see such approach used in gtk
so I'm unsure if that's supposed to work?  Or perhaps there might be
possibly only a couple of bugs to fix to get it to work?

Samuel
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: GtkSocket/GtkPlug accessibility

[Colomban Wendling]

Le 27/01/2017 à 15:32, Mario Sanchez Prada a écrit :
> […]
> 
> I know I arrive tremendously late to this thread, and I'm not even sure if
> this is going to be helpful for your situation, but just in case this is a
> small test application I wrote long ago to understand how the whole thing of
> AtkSocket & AtkPlug works:
> 
> https://github.com/mariospr/atksocket-atkplug-example

I did found your test app when searching info about that, looked helpful
if I happened to need implementing something :)

> Also, in case it also helps, this is a (long) rant I wrote a while ago too
> explaining how we implemented accessibility support in the multiprocess
> architecture of WebKit2, also using AtkSocket and AtkPlug:
> 
> http://mariospr.org/2013/02/03/accessibility-in-webkitgtk
> 
> Now, feel free to dismiss all this if it's totally irrelevant :-)

No, that looks interesting, esp. as you have included some info about
Socket/Plug.  Maybe not technical enough, but the example you linked
above covers that :)

Thanks!
Colomban
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: GtkSocket/GtkPlug accessibility

[Colomban Wendling]

Hi,

Thanks for those answers.

Le 29/12/2016 à 11:51, Alejandro Piñeiro a écrit :
>>> But fact is things like MATE-Panel make heavy use of those, so it'd be
>>> nice to have it work properly.
> 
> Then perhaps it is about using AtkSocket/AtkPlug on MATE-Panel. But that
> is an advice given without looking at all that code.

Okay, I'll see then.  If it's not something that should be done at the
GtkSocket/Plug level and would profit everyone, I might investigate a
different hackish solution that might be good enough for the MATE-Panel
specific problem.  Or see to use AtkSocket/Plug in MATE-Panel's side if
needed, we'll see.

Regards,
Colomban
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: GtkSocket/GtkPlug accessibility

[Alejandro Piñeiro]

On 28/12/16 18:33, Emmanuele Bassi wrote:
> Hi;
>
> On 28 December 2016 at 17:29, Colomban Wendling  wrote:
>> Hi,
>>
>> It seems that GtkSocket and GtkPlug aren't tied together at the
>> accessibility level: e.g. the ATSPI tree from Accerciser shows them
>> separately, and atspi_accessible_get_application() returns the embedded
>> application rather than the embedding one.
> AFAIR, there's really no mechanism to bridge two separate processes;

Not sure if you are talking specifically about GtkSocket/GtkPlug, but
just in case that is a general statement:
AtkSocket/AtkPlug can be used to bridge two processes of one application
from the a11y POV. They were initially created to support some Mozilla
plugins, where the plugin was running on different processes, but it was
preferred to expose them as a single application from the AT-SPI pov. It
is also used on WebKitGTK, where each tab can be a different process.
Probably there are some more cases where they are used.

> there's not even a discovery mechanism that allows you to know if the
> embedded window has any accessibility support, let alone something
> ATSPI can consume. 
Yes thats true. All the current usages of AtkSocket/AtkPlug are based on
assuming that the embedded windows are controlled by the embedder, and
that they have accessibility support.

> Additionally, it's even possible to embed a
> sub-tree of a separate process, within different contexts of execution
> — e.g. it's entirely possible that the embedder is the window
> manager/compositor, and the embeddee is a part of an application
>
> We'd need a separate interface for ATK and ATSPI to negotiate
> capabilities and act as a proxy — and we're already coming up short
> with regards to other windowing systems like Wayland.

Perhaps it's too early in the morning, but I don't follow this last
paragraph. Could you provide a specific example?

>> So I'm wondering what should be done here.  Should GtkSocket and GtkPlug
>> have accessible implementations making use of AtkSocket and AtkPlug, and
>> this just hasn't been done yet, or is something else required?  Would
>> that solve the issue?
> It's likely that we'd need something more than that…

Last time I checked, GtkSocket/GtkPlug were not really good candidates
for providing a default AtkSocket/AtkPlug implementation. As Emmanuele
is mentioning, it is likely that something else would be needed.

>> I know some people would like to forget about GtkSocket and GtkPlug :)
> … but, really, the actual solution is to revise the accessibility
> stack in a way that's usable under Wayland.

I agree that there are several things on the accessibility stack that
needs improvement/reimplementation under Wayland. But still not sure if
it is the embedder/embedded window mechanism for applications with more
that one process is one of it. But as I mentioned before, probably Im
forgetting a specific case.

>
> And, yes: GtkSocket and GtkPlug can die in a fire, as far as I'm concerned.
>
>> But fact is things like MATE-Panel make heavy use of those, so it'd be
>> nice to have it work properly.

Then perhaps it is about using AtkSocket/AtkPlug on MATE-Panel. But that
is an advice given without looking at all that code.

BR

___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

Re: GtkSocket/GtkPlug accessibility

[Emmanuele Bassi]

Hi;

On 28 December 2016 at 17:29, Colomban Wendling  wrote:
> Hi,
>
> It seems that GtkSocket and GtkPlug aren't tied together at the
> accessibility level: e.g. the ATSPI tree from Accerciser shows them
> separately, and atspi_accessible_get_application() returns the embedded
> application rather than the embedding one.

AFAIR, there's really no mechanism to bridge two separate processes;
there's not even a discovery mechanism that allows you to know if the
embedded window has any accessibility support, let alone something
ATSPI can consume. Additionally, it's even possible to embed a
sub-tree of a separate process, within different contexts of execution
— e.g. it's entirely possible that the embedder is the window
manager/compositor, and the embeddee is a part of an application

We'd need a separate interface for ATK and ATSPI to negotiate
capabilities and act as a proxy — and we're already coming up short
with regards to other windowing systems like Wayland.

> So I'm wondering what should be done here.  Should GtkSocket and GtkPlug
> have accessible implementations making use of AtkSocket and AtkPlug, and
> this just hasn't been done yet, or is something else required?  Would
> that solve the issue?

It's likely that we'd need something more than that…

> I know some people would like to forget about GtkSocket and GtkPlug :)

… but, really, the actual solution is to revise the accessibility
stack in a way that's usable under Wayland.

And, yes: GtkSocket and GtkPlug can die in a fire, as far as I'm concerned.

> But fact is things like MATE-Panel make heavy use of those, so it'd be
> nice to have it work properly.

As long as you want to keep MATE X11-only, accessibility is probably
the least of your worries.

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

GtkSocket/GtkPlug accessibility

[Colomban Wendling]

Hi,

It seems that GtkSocket and GtkPlug aren't tied together at the
accessibility level: e.g. the ATSPI tree from Accerciser shows them
separately, and atspi_accessible_get_application() returns the embedded
application rather than the embedding one.

This leads to e.g. Orca finding them being part of separate applications
and generally announcing some extra stuff.  And it technically makes
sense as nothing seems to tie them up at the ATSPI level.
Maybe Orca could hack something up to work around it, but I would think
it's not the proper place to change this.

So I'm wondering what should be done here.  Should GtkSocket and GtkPlug
have accessible implementations making use of AtkSocket and AtkPlug, and
this just hasn't been done yet, or is something else required?  Would
that solve the issue?

I know some people would like to forget about GtkSocket and GtkPlug :)
But fact is things like MATE-Panel make heavy use of those, so it'd be
nice to have it work properly.  I'd be willing to work on that, but I'm
first trying to see if I'm heading in the right direction or if I'm
totally off track.
Also, if it's the right thing to do, I guess the question of how to
communicate the socket ID arises, so that it could potentially work
outside GTK itself for other plugged windows to work.

Regards,
Colomban
___
gnome-accessibility-list mailing list
gnome-accessibility-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnome-accessibility-list