From: "Owen W. Taylor" <otay...@fishsoup.net> To support an application marking the beginning and end of the frame, add support for an extended frame synchronization protocol that extends the protocol of _NET_WM_SYNC_REQUEST to allow an application to provide a counter that it updates to an odd value to begin a frame and to an even value to end the frame.
On top of that are built: * A message _NET_WM_FRAME_DRAWN that is sent when the compositor has drawn the frame. * A message _NET_WM_FRAME_TIMINGS that is sent including extra frame timing information. * A property _NET_WM_SYNC_FENCES that can be combined with the counter value to find an appropriate fence to wait on. --- wm-spec/wm-spec.xml | 500 ++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 465 insertions(+), 35 deletions(-) diff --git a/wm-spec/wm-spec.xml b/wm-spec/wm-spec.xml index 8d83a94..debaea4 100644 --- a/wm-spec/wm-spec.xml +++ b/wm-spec/wm-spec.xml @@ -1773,28 +1773,327 @@ window when all updates are done. The application should not be generating many frames of content when only one of them is drawn to the output device. </para> <para> -Limited synchronization of window manager and client drawing during resizing -is possible without a compositing manager, and is briefly described below, -but the remaining protocols only make sense when there is a compositing -manager. These protocols are additionally designed with the assumption that -the window manager and the compositing manager are the same process, and are -not applicable to the case of a stand-alone compositing manager. +Limited synchronization of window manager and client drawing during +resizing is possible without a compositing manager, and described +below under <link linkend="basic_synchronization">Basic +Synchronization</link>, but the majority of protocols in this section +only make sense when there is a compositing manager. These protocols +are additionally designed with the assumption that the window manager +and the compositing manager are the same process, and are not +applicable to the case of a stand-alone compositing manager. </para> - <sect2 id="NET_WM_SYNC_REQUEST"> - <title>_NET_WM_SYNC_REQUEST</title> + <para> +Both forms of synchronization identify "frames" of drawing using +XSync extension <citation><link linkend="XSync">XSync</link></citation> +counters. The XSync extension allows creating Counter objects that +hold a 64-bit value. Applications can select to get events when +the counter object changes or reaches a particular value. + </para> + <para> +In the synchronization protocols, drawing is timed with reference to +the output device's <firstterm>refresh cycle</firstterm>. An output +device, such as a monitor, will typically read data from the frame +buffer sequentially starting at the top of the displayed area, pause +for a period known as the <firstterm>vertical blanking +period</firstterm> (or vertical blanking interval) and then +repeat. The vertical blanking period provides an opportunity to +atomically change the screen contents without the risk of tearing, and +is when a compositor using double-buffered drawing will swap +buffers. The entire length of the process is called the +<firstterm>refresh interval</firstterm>. For example, a monitor +updating at 60 frames per second has a refresh interval of 1/60th of a +second. + </para> + <para> +A compositor may be managing multiple output devices with different +refresh cycles, and windows may overlap multiple devices. This means +when an application draws a frame, the point in time where that frame +of drawing actually is displayed to the user may be different on +different output devices. For each application frame, the compositor +MUST identify a single output device that the frame is +<emphasis>primarily</emphasis> displayed upon, and report timing +information for the display of the frame with respect to that +device. For frame updates that don't include the entire window, the +chosen device MAY depend on the particular updated area, and the +chosen device MAY change from frame to frame for this or other +reasons. + </para> + <sect2 id="basic_synchronization"> + <title>Basic Synchronization</title> + <para> +The goal of basic synchronization is limited to coordinating redraws +during interactive resizing. A client indicates that it is willing to +participate in basic synchronization by listing _NET_WM_SYNC_REQUEST +in the WM_PROTOCOLS property of the client window and storing the XID +of a XSync counter in the property _NET_WM_SYNC_REQUEST_COUNTER. +This counter is known as the <firstterm>basic frame counter</firstterm>. + </para> + <para> +Before resizing a window, the window manager sends a +_NET_WM_SYNC_REQUEST message to the client window containing a value +that the application stores in the basic frame counter when it is done +handling the ConfigureNotify event resulting from the resize. This +allows the window manager to know that it can move on to the next step +of resizing without getting ahead of the client. If the window manager +is a compositing manager it may also choose to freeze redrawing of the +window until it sees the change in counter value, so that the resized +window frame and window are drawn as a single unit. + </para> + </sect2> + <sect2 id="extended_synchronization"> + <title>Extended Synchronization</title> + <para> +Extended synchronization provides a more general framework for redraw +coordination, including spontaneous application updates as well as +resizing. A client indicates its willingness to participate in the +extended form of frame synchronization by listing _NET_WM_SYNC_REQUEST +in WM_PROTOCOLS and including <emphasis>two</emphasis> XSync counters +in the property _NET_WM_SYNC_REQUEST_COUNTER, the basic frame counter +and an <firstterm>extended frame counter</firstterm>. A window manager +indicates that it will participate by listing _NET_WM_FRAME_DRAWN in +_NET_SUPPPORTED. + </para> + <para> +In extended synchronization, the basic frame counter is unused. The +extended frame counter is updated in response to _NET_WM_SYNC_REQUEST +messages, but can can additionally be updated spontaneously by the +client to indicate the beginning or end of a frame of drawing. The +beginning of a frame of drawing is indicated by an increment to an odd +number, and the end of a frame of drawing is indicated by an increment +to an even value. For each frame marked this way, a client will +receive _NET_WM_FRAME_DRAWN and _NET_WM_FRAME_TIMINGS messages from +the compositor. + </para> + <para> +Rationale: using the same counter for basic and extended +synchronization would cause problems when switching from a window +manager that supports only the basic synchronization to one supporting +extended synchronization - since in one case the window manager +determines the counter values and in the other case the client +determines the counter values, there would be race conditions during +the switch. Packing both counters into a single property increases +efficiency when initially managing a window. + </para> + </sect2> + <sect2 id="frame_timing_algorithm"> + <title>Compositor frame timing algorithm</title> + <para> +When an application is displaying changing content, such as animated +transitions, videos, or user interface elements that respond to the +user's mouse input, there are three primary measures of quality: +<firstterm>Frame rate</firstterm> - the number of frames that are +drawn per second. <firstterm>Latency</firstterm> - the time between +when a frame is generated and when it is displayed to the user. Moderate +latency will cause dragged objects to lag behind the cursor, higher +latency will cause apparent discrepencies to the user between when an +action is taken and when it is displayed. In some cases, such as the +synchronization of audio and video, a known latency can be compensated +for. <firstterm>Jitter</firstterm> - the difference in latency across +frames. Jitter causes the appearance of stuttering, uneven motion. + </para> + <para> +The choice of algorithm for scheduling redraws in the compositor will +involve tradeoffs between these quality measures. In particular +there is a tradeoff between latency and jitter. If a compositor draws +as soon as it receives a new frame from any application, this will +result in minimum latency with large amounts of jitter - a frame may +be processed immediately and drawn with minimal delay, but if +the compositor is busy processing a redraw triggered +by another client, there will be additional latency. In order to +present consistent latency to applications, an alternate approach is +recommended: after receiving a new frame, the compositor waits for a +fixed point in the refresh cycle before starting the redraw. By making +sure that each frame of drawing is finished with some time to spare before this +point, an application achieves a latency that is both known and +consistent. + </para> + <para> +The approach of waiting to a fixed point in the refresh cycle can +backfire if an application is unable to achieve its target frame rate: +the delay before the compositor draws will waste time that could have +been spent drawing and cause unnecessary synchronization to a +sub-multiple of the system's refresh rate. For this reason, in the +protocols described below, frames can be marked as +<firstterm>urgent</firstterm>. In response to an urgent frame, a +compositor SHOULD draw as quickly as possible and send a +_NET_WM_FRAME_DRAWN message so the application can draw the next +frame. One approach to deciding which frames to mark urgent is for the +application to note whether it sleeps after receiving +_NET_WM_FRAME_DRAWN before starting the next frame. If not, the next +frame is marked urgent. + </para> + <para> +The following algorithm is <emphasis>recommended</emphasis> for +compositors. It is however, not <emphasis>required</emphasis>. + </para> +<itemizedlist> + <listitem> + <para>When the compositor receives a XDamageNotify event:</para> + <itemizedlist> + <listitem><para>If it is part of an urgent frame, schedule an immediate redraw</para></listitem> + <listitem><para>Otherwise, schedule a redraw for the next redraw point. Redraw points + occur <replaceable>frame_delay</replaceable> milliseconds after the start of the vertical blanking period.</para></listitem> + </itemizedlist> + </listitem> + <listitem><para> + If a redraw is scheduled for time T, and and the swap has not yet completed at time T, redraw immediately when the swap completes + </para></listitem> +</itemizedlist> + <para> +The choice of <replaceable>frame_delay</replaceable> is up to the +compositor. The minimum event-handling latency that an application can +achieve is <replaceable>application_frame_draw_time</replaceable> + +<replaceable>refresh_interval</replaceable> - +<replaceable>frame_delay</replaceable>. A larger value will reduce +latency slightly, but increases the chance that the compositor won't +be done drawing the frame by the vertical blanking period, and +it will be displayed a refresh cycle late. Using +0 for <replaceable>frame_delay</replaceable> for delay is not +recommended, since that will produce pessimized latency for legacy GL +clients that swap buffers during the vertical blanking period. +Instead a small value such as 2 milliseconds is recommended. + </para> + </sect2> + <sect2> + <title>High precision timestamps</title> + <para> +When exchanging timing information about drawing, it is useful for +clients to be able to have a higher-precision than the millisecond +precision of X server timestamps. To allow for greater precision, +intervals and timestamps for protocols in this section are represented +in microseconds. For timestamps, the timestamp has the form +(<replaceable>x_server_timestamp</replaceable>) * 1000 + <replaceable>microsecond_value</replaceable>. + </para> + <para> +The window manager MAY produce these <firstterm>high precision +timestamps</firstterm> by periodically determining an offset between +the server time and a local system time. The server time can be +determined by the standard method of changing a dummy property and +observing the timestamp in the PropertyNotify event. For the system +time it is preferred to use a monotonic time, such as the time given +by clock_gettime(CLOCK_MONOTONIC, ...), rather than a real time, such +as that given by gettimeofday(). If this method of tracking an offset +is used, the accuracy of passing a time between two clients will be +limited by the precision of the X server time, and the round-trip time +for obtaining the X server time. + </para> + <para> +Alternatively, a client MAY observe that X server times correspond to +a specific method of obtaining the local time, such as using +gettimeofday() or clock_gettime(). In this case, high precision times +can be generated directly, and two clients such will be able to +exchange timestamps with sub-millisecond accuracy. (This technique is +only applicable to clients running on the same machine as the X +server.) + </para> + </sect2> + <sect2> + <title>_NET_WM_SYNC_REQUEST_COUNTER</title> + <programlisting><![CDATA[ +_NET_WM_SYNC_REQUEST_COUNTER, COUNTER[]/32 +]]></programlisting> + <para> +The _NET_WM_SYNC_REQUEST_COUNTER property is set by the application on a toplevel +window. It contains either one or two XSync counter IDs, depending on whether +the client supports only basic synchronization or both basic and extended +synchronization. + </para> + <para> +If the client supports only basic synchronization, then it puts only the +basic frame counter ID in the _NET_WM_SYNC_REQUEST_COUNTER property. The initial +value of basic frame counter is not defined by the specification. The window +manager MAY set the value of the basic frame counter at any time, and, +if it chooses t use basic synchronization, MUST do so when it first manages +a new window. + </para> + <para> +If the client supports extended synchronization, then the client +creates both a basic frame counter and an extended frame counter, and +puts both IDs in _NET_WM_SYNC_REQUEST_COUNTER, with the first element +of the property being the basic frame counter ID and the second +element the extended frame counter ID. The client sets the extended +frame counter to an initial value before before changing the window +from the withdrawn state, and the client MAY update the value at any +time. + </para> + <para> +To indicate the beginning of a frame, the client increases the value +of the extended frame counter to an odd value. At this point, the +window is an a "frozen" state and the compositing manager SHOULD NOT +redraw the window in response to XDamageNotify events. If the +compositing manager needs to redraw for other reasons, such as a +change to a different window that overlaps the frozen window, it MAY +redraw the window. To indicate the end of the frame the client +increases the value of the extended frame counter to an even value. At +this point the window is no longer frozen, and the compositor SHOULD +redraw the window and send _NET_WM_FRAME_DRAWN and +_NET_WM_FRAME_TIMINGS messages. The window manager SHOULD redraw the +window and send the messages even if no damage events were received. + </para> + <para> +Rational: the reason for redrawing the window even if no damage events have +been received is so that there is consistent timing for each frame, +even if one frame happens to involve no changes to the window. It is recommended +to behave as if a damage event was received containing only a single pixel. + </para> + <para> +To distinguish urgent from non-urgent frames, two different patterns of +increases are used. (See <xref linkend="frame_timing_algorithm"/> for +the definition of urgent frames.) To mark an urgent frame the odd value +is chosen to have a value such that <replaceable>v</replaceable> % 4 = 3, then +the counter is increased by 1 to give the even value. To mark a +non-urgent frame the odd value is chosen to have a value such that +<replaceable>v</replaceable> % 4 = 1, then +the counter is increased by 3 to give the even value. + </para> + </sect2> + <sect2> + <title>_NET_WM_SYNC_FENCES</title> + <para> +On some systems with loose synchronization between different clients +using the graphics system, drawing immediately after XDamageNotify +events are received does not work properly, and a GL-based compositing +manager must also insert GL-level synchronization to ensure correct +drawing. Because there is no way of knowing whether that is the case +on any particular system, _NET_WM_SYNC_FENCES SHOULD always be +set by applications and compositors MUST synchronize drawing +either using _NET_WM_SYNC_FENCES or the alternate method described +below. + </para> + <para> +The _NET_WM_SYNC_FENCES property is set on a toplevel by an +application and contains a list of XSync fence objects. Before ending +a frame by setting the XSync counter to an even value N, the +application uses XSyncTriggerFence() to trigger the fence object +stored in the property at index (N / 4) % L, where L is the number of +counters in property. If an application waits for the +_NET_WM_SYNC_DRAWN message before starting a new frame, 2 is a +sufficiently large value for L. + </para> <para> -This protocol uses the XSync extension (see <ulink -url="http://freedesktop.org/cgi-bin/viewcvs.cgi/xorg/xc/doc/hardcopy/Xext/sync.PS.gz">the -protocol specification</ulink> and <ulink -url="http://freedesktop.org/cgi-bin/viewcvs.cgi/xorg/xc/doc/hardcopy/Xext/synclib.PS.gz"> -the library documentation</ulink>) to let client and window manager -synchronize the repaint of the window manager frame and the client -window. A client indicates that it is willing to participate in the -protocol by listing _NET_WM_SYNC_REQUEST in the WM_PROTOCOLS property -of the client window and storing the XID of an XSync counter in the -property _NET_WM_SYNC_REQUEST_COUNTER. The initial value of this -counter is not defined by this specification. +When the window is mapped, the GL-based compositor SHOULD import the +fences in _NET_WM_SYNC_FENCES as GL sync objects using the +ImportSyncEXT() procedure from <citation><link +linkend="x11_sync_object">EXT_x11_sync_object</link></citation>. +After receiving a frame update ending with counter value N, the next +time that the window is redrawn, the compositing manager calls +glWaitSync() on the fence at index (N / 4) % L. </para> + <sect3> + <title>Alternate method</title> + <para> +If an application window doesn't export _NET_WM_SYNC_FENCES, or draws +outside a frame update, the compositing manager still MUST ensure +correct synchronization. If the compositing manager has received +XDamageNotify events that are not synchronized by _NET_WM_SYNC_FENCES, then +before drawing the screen, the compositing manager MUST use +XSyncTriggerFence() to trigger an X fence it has created itself, and then +wait for that fence using glWaitSync(). + </para> + </sect3> + </sect2> + <sect2 id="NET_WM_SYNC_REQUEST"> + <title>_NET_WM_SYNC_REQUEST</title> <para> A window manager uses this protocol by preceding a ConfigureNotify event sent to a client by a client message as follows: @@ -1806,31 +2105,139 @@ message_type = WM_PROTOCOLS format = 32 data.l[0] = _NET_WM_SYNC_REQUEST data.l[1] = timestamp -data.l[2] = low 32 bits of the update request number -data.l[3] = high 32 bits of the update request number -other data.l[] elements = 0 +data.l[2] = low 32 bits of a frame counter value +data.l[3] = high 32 bits of a frame counter value +data.l[4] = 1 if the the extended frame counter should be updated, otherwise 0 ]]></programlisting> <para> -After receiving one or more such message/ConfigureNotify pairs, and -having handled all repainting associated with the ConfigureNotify -events, the client MUST set the _NET_WM_SYNC_REQUEST_COUNTER to the 64 -bit number indicated by the data.l[2] and data.l[3] fields of the last -client message received. +If data.l[4] is 0, then after receiving one or more such +message/ConfigureNotify pairs, and having handled all repainting +associated with the ConfigureNotify events, the client MUST set the +basic frame counter to the frame counter value from the last client +message received. + </para> + <para> +The frame counter value in the client message is determined by the +window manager subject to the restriction that it MUST NOT be 0. The +number is generally intended to be incremented by one for each message +sent. + </para> + <para> +If data.l[4] is 1, then after receiving one or more such +message/ConfigureNotify pairs, and having handled all repainting +associated with the ConfigureNotify events, the client MUST set the +extended frame counter to a value <emphasis>greater than</emphasis> +the frame counter value in the last client message received. This will +normally be done as part of indicating the end of a frame of drawing. +The exact value used is chosen so that the fences listed in +_NET_WM_SYNC_FENCES are used in rotation. + </para> + <para> +The frame counter value in the client message is determined by the +window manager based on the most recent value it has seen for the +extended frame counter. The window manager SHOULD retrieve +this value when managing the window, and monitor subsequent changes +to the value by creating an XSync Alarm object. The update request +number in the client message SHOULD be chosen as +<replaceable>last_seen_value</replaceable> + 240. + </para> + <para> +Rationale: if the client is continually redrawing, then the last +seen value may be out of date when the window manager sends the message. +Picking a number that is 240 later would allow for 1 second of frames +at 60fps. In the normal case client will draw only one more frame before +waiting for _NET_WM_FRAME_DRAWN. </para> <para> By using either the Alarm or the Await mechanisms of the XSync extension, the window manager can know when the client has finished -handling the ConfigureNotify events. The window manager SHOULD not +handling the ConfigureNotify events. The window manager SHOULD NOT resize the window faster than the client can keep up. </para> + </sect2> + <sect2> + <title>_NET_WM_FRAME_DRAWN</title> <para> -The update request number in the client message is determined by the -window manager subject to the restriction that it MUST NOT be 0. The -number is generally intended to be incremented by one for each message -sent. Since the initial value of the XSync counter is not defined by -this specification, the window manager MAY set the value of the XSync -counter at any time, and MUST do so when it first manages a new -window. +This client message allows a client to know when an update it has +created has been drawn to the screen by the compositing window +manager. If the compositing manager lists this property in +_NET_SUPPORTED, then the compositing manager MUST send a client that +supports extended synchronization a _NET_WM_FRAME_DRAWN client message +immediately after finishing the redrawing that results from an +application frame. If an application draws several frames without +waiting for _NET_WM_FRAME_DRAWN, the compositing manager MAY send only +one _NET_WM_FRAME_DRAWN for the last frame. The contents of the +client message are as follows: + </para> + <programlisting><![CDATA[ +type = ClientMessage +window = the respective client window +message_type = _NET_WM_FRAME_DRAWN +format = 32 +data.l[0] = low 32 bits of the extended frame counter value +data.l[1] = high 32 bits of the extended frame counter value +data.l[2] = low 32 bits of high precision timestamp +data.l[3] = high 32 bits of high precision timestamp +data.l[4] = 0 +]]></programlisting> + <para> +The timestamp indicates that time at which the compositing manager +finished submitting drawing for entire scene to the graphics system. +(If that time is not available, the compositing manager MAY approximate +it by the time at which the message is sent.) The actual completion +of drawing will occur at some time later than the indicated time. + </para> + <para> +In addition to sending one message at the end of each frame, the +compositing manager MUST send one _NET_WM_FRAME_DRAWN message for each +newly mapped window that supports extended synchronization. If the +initial value of the frame counter is odd, then the window starts in +the frozen state, and the message is sent as per normal after the +frame ends. If the initial frame value is even, the message is +sent after the window is first drawn. + </para> + </sect2> + <sect2> + <title>_NET_WM_FRAME_TIMINGS</title> + <para> +This message provides information about the timing of a previous frame; +it is sent subsequent to the _NET_WM_FRAME_DRAWN message for the frame +once the window manager has obtained all available timing information. + </para> + <programlisting><![CDATA[ +type = ClientMessage +window = the respective client window +message_type = _NET_WM_FRAME_TIMINGS +format = 32 +data.l[0] = low 32 bits of the extended frame counter value +data.l[1] = high 32 bits of the extended frame counter value +data.l[2] = presentation time offset +data.l[3] = refresh interval +data.l[4] = frame_delay +]]></programlisting> + <para> +The presentation time offset is a 32-bit signed offset in microseconds +from the timestamp in the _NET_WM_FRAME_DRAWN message. It may be 0 if +no presentation time is available. The presentation time indicates the +time of the end of the vertical blanking period after which the frame +contents will be scanned out to the output device. (If this exact time +is not available, the composite manager MAY provide a nearby time such +as the time at which buffers are swapped.) + </para> + <para> +The refresh interval a 32-bit unsigned number indicating the +refresh interval in microseconds. It may be 0 if the refresh interval is +not available. + </para> + <para> +The frame delay is a 32-bit unsigned number indicating the point in the +refresh interval where the compositor will start drawing if it previously +received damage, as a number of microseconds after the end of the +vertical blanking period. +See <xref linkend="frame_timing_algorithm"/> for details. +The flag value 0x80000000 indicates that the compositor uses a different +frame timing algorithm and no meaningful value is available. All other +values with the high-bit set are reserved. </para> </sect2> </sect1> @@ -2239,6 +2646,29 @@ int net_get_hostname (char *buf, size_t maxlen) </para> </listitem> </varlistentry> + <varlistentry> + <term>[XSync]</term> + <listitem> + <para id="XSync"> + Tim Glauert, Dave Carver, Jim Gettys, David P. Wiggins, and James Jones, + "X Synchronization Extension Protocol, X Consortium Standard" (Version 3.1) + <ulink url="http://www.x.org/releases/X11R7.7/doc/xextproto/sync.html"> + http://www.x.org/releases/X11R7.7/doc/xextproto/sync.html</ulink> + </para> + </listitem> + </varlistentry> + <varlistentry> + <term>[x11_sync_object]</term> + <listitem> + <para id="x11_sync_object"> + Piers Daniell, Pierre-Loup Griffais, James Jones, Aaron Plattner, + "EXT_x11_sync_object". + <ulink url="http://www.opengl.org/registry/specs/EXT/x11_sync_object.txt"> + http://www.opengl.org/registry/specs/EXT/x11_sync_object.txt + </ulink> + </para> + </listitem> + </varlistentry> </variablelist> </sect1> <sect1> -- 1.8.0.2 _______________________________________________ wm-spec-list mailing list wm-spec-list@gnome.org https://mail.gnome.org/mailman/listinfo/wm-spec-list