Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
On 2020-06-24 5:19 p.m., Keith Packard wrote: > Michel Dänzer writes: > >> I assume 2. refers to the case of a single late frame, where the >> next frame hitting the original absolute target would result in a >> second visible stutter. While writing >> https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/22#note_548234 >> >> , it occurred to me that although a relative target time may avoid >> that second stutter, the audio stream needs to adapt to the >> visual delay, likely producing audible artifacts instead? It's >> not obvious to me that would be an overall win. (The only other >> way I can think of is to re-synchronize later frames to the audio >> stream, but it's not clear that this is possible without either >> producing visible stutter again, or de-synced audio/video for a >> noticeable period of time) > > I think the idea is that the application would interpolate time > values in the video stream to bring it back in sync with the > expected time over a couple of frames. I think we could easily > construct a demo which shows the difference and see what we think. Such a demo could be very helpful! > I think we could ignore the audio stream as a 16ms lag between > audio and video shouldn't be a big deal; we handle that in real > life quite easily as that's the lag you get at a distance of about > 5m. I suspect it may not be quite that simple, e.g. the brain may compensate for the distance determined from vision. :) Anyway, it's plausible that this can be ignored. >> P.S. Have you more formally proposed a Vulkan extension in the >> meantime? If so, could you provide a reference to that here? > > No. If I had, the IP restrictions with Khronos would prevent me > from discussing it here in any technical detail. Ah, didn't mean to put you in a difficult position. -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer signature.asc Description: OpenPGP digital signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: > I assume 2. refers to the case of a single late frame, where the next > frame hitting the original absolute target would result in a second > visible stutter. While writing > https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/22#note_548234 > , it occurred to me that although a relative target time may avoid > that second stutter, the audio stream needs to adapt to the visual > delay, likely producing audible artifacts instead? It's not obvious to > me that would be an overall win. (The only other way I can think of is > to re-synchronize later frames to the audio stream, but it's not clear > that this is possible without either producing visible stutter again, > or de-synced audio/video for a noticeable period of time) I think the idea is that the application would interpolate time values in the video stream to bring it back in sync with the expected time over a couple of frames. I think we could easily construct a demo which shows the difference and see what we think. I think we could ignore the audio stream as a 16ms lag between audio and video shouldn't be a big deal; we handle that in real life quite easily as that's the lag you get at a distance of about 5m. > P.S. Have you more formally proposed a Vulkan extension in the > meantime? If so, could you provide a reference to that here? No. If I had, the IP restrictions with Khronos would prevent me from discussing it here in any technical detail. -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
On 2020-02-02 7:51 a.m., Keith Packard wrote: > > I spent some time over the last week experimenting with a different > way of doing frame timing. > > Instead of specifying *when* a particular frame should be > displayed, how about we specify how *long* a particular frame > should be made visible to the user? > > This has a couple of advantages over the approach taken in > GOOGLE_display_timing: > > 1. It provides information to the backend about frame timing a > frame earlier. > > 2. Missing a frame can generate fewer artifacts. I assume 2. refers to the case of a single late frame, where the next frame hitting the original absolute target would result in a second visible stutter. While writing https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/22#note_548234 , it occurred to me that although a relative target time may avoid that second stutter, the audio stream needs to adapt to the visual delay, likely producing audible artifacts instead? It's not obvious to me that would be an overall win. (The only other way I can think of is to re-synchronize later frames to the audio stream, but it's not clear that this is possible without either producing visible stutter again, or de-synced audio/video for a noticeable period of time) P.S. Have you more formally proposed a Vulkan extension in the meantime? If so, could you provide a reference to that here? -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer signature.asc Description: OpenPGP digital signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: > Should this extension specify how it interacts with the various > VK_PRESENT_MODE_* modes? Yes. It needs to be clear on how this extension interacts with all existing display stuff. Thanks for pointing out one pretty important interaction. > For one example: With VK_PRESENT_MODE_MAILBOX_KHR, does the period > specified by this extension correspond to: > > > a) The time between when the image is placed in the the queue of > pending presentation requests and when the next image is placed in the > queue > > b) The time between when the image is taken from the queue to be > actually presented and when the same thing happens for another image > (which happens to be in the queue at the time) > > c) Yet something else? > > If it's a), given the extension talks about rounding to the nearest > upcoming frame, does VK_PRESENT_MODE_MAILBOX_KHR effectively behave > the same as VK_PRESENT_MODE_FIFO(_RELAXED)_KHR with this extension? > > If it's b), there can be any number of images entering and leaving the > queue during the period, so it's not clear what purpose the period > would serve? Given that the period is defined as being relative to the time when the image was presented to the screen (not when the image is queued for presentation), and that the extension specifies that future images will be delayed by that period, I think the right definition will be that specifying non-zero present_period for a QueuePresent call will force images queued later to not replace the first image and be delayed for display until the specified present_period has passed. Which looks a lot like FIFO, but only for QueuePresent calls which specify a non-zero present_period. I think I've got enough to start writing a more 'formal' specification for the extension, which I'll do as a patch to the Vulkan specification. -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 2020-02-02 7:51 a.m., Keith Packard wrote:
>
> I spent some time over the last week experimenting with a different
> way of doing frame timing.
>
> Instead of specifying *when* a particular frame should be
> displayed, how about we specify how *long* a particular frame
> should be made visible to the user?
>
> This has a couple of advantages over the approach taken in
> GOOGLE_display_timing:
>
> 1. It provides information to the backend about frame timing a
> frame earlier.
>
> 2. Missing a frame can generate fewer artifacts.
>
> Here's what I'm thinking the extension would look like:
>
> An application uses VK_MESA_present_period by including a
> VkPresentPeriodMESA structure in the pNext chain in the
> VkPresentInfoKHR structure passed to the vkQueuePresentKHR call.
>
> typedef struct VkPresentPeriodMESA { VkStructureTypesType;
> const void*pNext; uint32_t swapchainCount; const
> int64_t* pPresentPeriods; } VkPresentPeriodMESA;
>
> The fields in this structure are:
>
> * sType. Set to VK_STRUCTURE_TYPE_PRESENT_PERIOD_MESA * pNext.
> Points to the next extension structure in the chain (if any). *
> swapchainCount. A copy of the swapchainCount field in the
> VkPresentInfoKHR structure. * pPresentPeriods. An array, length
> swapchainCount, of presentation periods for each image in the
> call.
>
> Positive presentation periods represent nanoseconds. Negative
> presentation periods represent frames. A zero value means the
> extension does not affect the associated presentation. Nanosecond
> values are rounded to the nearest upcoming frame so that a value of
> n * refresh_interval is the same as using a value of n frames.
>
> The presentation period causes *future* images to be delayed at
> least until the specified interval after this image has been
> presented. Specifying both a presentation period in a previous
> frame and using GOOGLE_display_timing is well defined -- the
> presentation will be delayed until the later of the two times.
Should this extension specify how it interacts with the various
VK_PRESENT_MODE_* modes?
For one example: With VK_PRESENT_MODE_MAILBOX_KHR, does the period
specified by this extension correspond to:
a) The time between when the image is placed in the the queue of
pending presentation requests and when the next image is placed in the
queue
b) The time between when the image is taken from the queue to be
actually presented and when the same thing happens for another image
(which happens to be in the queue at the time)
c) Yet something else?
If it's a), given the extension talks about rounding to the nearest
upcoming frame, does VK_PRESENT_MODE_MAILBOX_KHR effectively behave
the same as VK_PRESENT_MODE_FIFO(_RELAXED)_KHR with this extension?
If it's b), there can be any number of images entering and leaving the
queue during the period, so it's not clear what purpose the period
would serve?
- --
Earthling Michel Dänzer | https://redhat.com
Libre software enthusiast | Mesa and X developer
-BEGIN PGP SIGNATURE-
iF0EARECAB0WIQSwn681vpFFIZgJURRaga+OatuyAAUCXkrQ9gAKCRBaga+Oatuy
AB2FAJ4glgpyMYSy1WPmbpzbI2O5Rvv0/QCfSeQBF7vsooAtjEBicmW76NfWWws=
=MzZO
-END PGP SIGNATURE-
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: > I think at least the following needs to happen first: > > * At least a prototype of plumbing through this information to the > amdgpu kernel driver (or another one which may grow VRR support) and > making use of it for adjusting the refresh periods to allow hitting > the target as closely as possible. > > * At least a prototype of a game engine making use of it to control > the variable refresh rate. > > This will allow confirming that this approach actually works and > provides the sought benefit. Awesome. Thanks for the recommendation. I can work on this. -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2020-02-07 10:19 p.m., Keith Packard wrote: > Michel Dänzer writes: > >> With variable refresh rate, it could certainly be useful for the >> kernel to have this information as early as possible. It >> shouldn't make any difference with fixed refresh though, since >> the kernel should always be able to hit the next refresh in that >> case as long as the fences for the flip signal in time for >> vertical blank. > > Although, if the application is mixing present_period and > display_timing operations, things can still get mixed up -- a frame > with present_period followed by a frame with display_timing can > invalidate the computed next frame time. Applications doing that > may not get the best possible result... Right. Applications should normally stick to one or the other extension, not flip-flop between them. > Ok, it sounds like I should at least go clean up the implementation > and make it into something not quite so embarrassing to me. > > Going forward, how can we provide this to application developers > for experimentation? Would we consider including it in a release > once reviewed within the Mesa community? I think at least the following needs to happen first: * At least a prototype of plumbing through this information to the amdgpu kernel driver (or another one which may grow VRR support) and making use of it for adjusting the refresh periods to allow hitting the target as closely as possible. * At least a prototype of a game engine making use of it to control the variable refresh rate. This will allow confirming that this approach actually works and provides the sought benefit. - -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSwn681vpFFIZgJURRaga+OatuyAAUCXkEoYwAKCRBaga+Oatuy AEHbAJ42gnXFAJ1j+znHDt68kS1k+BrrUwCgolxHui7Mqux37yjbfGZMmCUlFGo= =ZVoG -END PGP SIGNATURE- ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: > With variable refresh rate, it could certainly be useful for the kernel > to have this information as early as possible. It shouldn't make any > difference with fixed refresh though, since the kernel should always be > able to hit the next refresh in that case as long as the fences for the > flip signal in time for vertical blank. Although, if the application is mixing present_period and display_timing operations, things can still get mixed up -- a frame with present_period followed by a frame with display_timing can invalidate the computed next frame time. Applications doing that may not get the best possible result... Ok, it sounds like I should at least go clean up the implementation and make it into something not quite so embarrassing to me. Going forward, how can we provide this to application developers for experimentation? Would we consider including it in a release once reviewed within the Mesa community? -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
On 2020-02-04 8:12 p.m., Keith Packard wrote: > Michel Dänzer writes: > >> Hmm, I didn't fully realize this in my reading of the draft, thanks >> Matias for pointing it out! >> >> That the timing of frame N+1 has to be specified when submitting frame >> N for presentation is odd to me TBH. While I can imagine this might be >> suitable for some apps such as video players, I'm skeptical about game >> engines. They would need to calculate frame time budget and choose >> simulation time for frame N+1 before submitting frame N for >> presentation. Is that really how game engines (want to) work? > > It's not asking the application to figure this out much earlier -- the > application needs to know the target time for the next frame before > starting any of the frame computations, and that happens right after > submitting the previous frame. > > The goal here is to provide the display system the timing information as > soon as the application knows it, in case that helps the backend work > better. With variable refresh rate, it could certainly be useful for the kernel to have this information as early as possible. It shouldn't make any difference with fixed refresh though, since the kernel should always be able to hit the next refresh in that case as long as the fences for the flip signal in time for vertical blank. -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: >> actual_period = n * r - ε > > Is that still an issue if the (minimal) length of a vertical blanking > period is subtracted from the computed period? What you're doing is defining a 'window' of times which match the desired frame -- any time within 'length of vertical blanking period' of the actual top of frame matches that frame. I'm simply making that window as large as possible by using 'half the frame time' instead of 'length of vertical blanking period'. > FWIW, one thing making "not before" semantics attractive is that > they're easy to achieve in the kernel: It can just make sure the page > flip isn't programmed to hardware before the target time. Yup, offering only 'not before' makes it easy to implement in the kernel, but very difficult for the application to get the right result. For fixed refresh rate monitors, converting from 'nearest to time T' to 'not before frame count F' conversion can be done by rounding T/rate to F and then using that to program the hardware using 'not before' semantics. For variable refresh rate monitors, the computation will be slightly more complicated as it may involve some multiple of a nominal frame rate plus some stretching of the final frame. > PresentOptionUST has never been functional, so there can't be any > clients relying on specific semantics (other than being a no-op :) for > it. Therefore, we could still change its semantics before making it > functional, FWIW. Oops. Someone should fix that :-) -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: > Hmm, I didn't fully realize this in my reading of the draft, thanks > Matias for pointing it out! > > That the timing of frame N+1 has to be specified when submitting frame > N for presentation is odd to me TBH. While I can imagine this might be > suitable for some apps such as video players, I'm skeptical about game > engines. They would need to calculate frame time budget and choose > simulation time for frame N+1 before submitting frame N for > presentation. Is that really how game engines (want to) work? It's not asking the application to figure this out much earlier -- the application needs to know the target time for the next frame before starting any of the frame computations, and that happens right after submitting the previous frame. The goal here is to provide the display system the timing information as soon as the application knows it, in case that helps the backend work better. > Instead, have you considered allowing the GOOGLE_display_timing > desiredPresentTime to be specified as a relative time, measured from > when the previous image of the swapchain was actually presented, > instead of as an absolute time? Could something like that also address > the motivation for VK_MESA_present_period? Yes, that would avoid the display problem. -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2020-02-03 8:48 p.m., Keith Packard wrote: > Michel Dänzer writes: > >> Instead of rounding to the nearest upcoming frame, should it >> round to the earliest frame after the specified period has >> passed? As written, it seems to contradict the next paragraph >> below: > > Sorry for the poor wording; let me describe what I want informally > here. > > For nanosecond periods to be easy to use on fixed-refresh-rate > monitors, I want the näive computation to "just work". For a given > refresh period, 'r', I want to select a period of 'n' frames > using: > > computed_period = n * r > > Because of clock skew and rounding problems, it's quite possible > that the period could easily end up being just slightly smaller > than that: > > actual_period = n * r - ε Is that still an issue if the (minimal) length of a vertical blanking period is subtracted from the computed period? >> (I'm not ruling out that rounding to nearest might be better, but >> there should be a rationale for it, which also justifies being >> inconsistent with GOOGLE_display_timing) > > Yes, this is intentionally inconsistent with > GOOGLE_display_timing, which I believe is hard to use correctly. > > By specifying not before semantics, GOOGLE_display_timing requires > applications compute a fake time guaranteed to be not after the > actual target frame time. This really messes things up when you > might have variable refresh rate monitors. I see. Same question as above. FWIW, one thing making "not before" semantics attractive is that they're easy to achieve in the kernel: It can just make sure the page flip isn't programmed to hardware before the target time. > I just went and read the time-based stuff from the X Present > extension. That uses 'nearest' semantics too, when supported by > the driver. When not supported, the server gets to do whatever it > likes (oops!). PresentOptionUST has never been functional, so there can't be any clients relying on specific semantics (other than being a no-op :) for it. Therefore, we could still change its semantics before making it functional, FWIW. >> P.S. It would be good to create a WIP merge request for this in >> the main Mesa project, to have a central place for gathering >> feedback and tracking progress. > > Done, thanks for the suggestion. Thanks. For the record, that's https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3682 . - -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSwn681vpFFIZgJURRaga+OatuyAAUCXjlKiwAKCRBaga+Oatuy ALsMAKCRxgfQa1zDXrLZ6iUnkl1+PtHqIQCgrFiNxQmRMye9B0t3RRXtr3dGjiY= =lLMw -END PGP SIGNATURE- ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2020-02-03 9:16 p.m., Keith Packard wrote: > "Matias N. Goldberg" writes: > >> I read your article. > > Thanks! > >> What I feel are missing are just minor pesky details: > > Yes, it's definitely a rough draft at best. Figuring out the right > words to make it do precisely what we want is hard, and I'm hoping > we can first figure out what we want, then figure out how to say > that precisely. > >> 1. Written as is, the frame being submitted is rounded up to >> display timing, delaying *future* frames.But there is no way to >> delay the *currently displaying frame* (i.e. the 'previous' >> frame). > > Correct. The period provided controls how long the specified frame > will be shown to the user (at a minimum). Future frames will be > delayed by at least that long. > >> Right now if frame N was submitted without VkPresentTimeMESA but >> frame N+1 is, then frame N+1 will be presented to screen ASAP. > > Correct. > >> What I'm saying is that if frame N was submitted without >> VkPresentTimeMESA, then at frame N+1 I should be able to tell >> 'keep frame N displayed for at least P nanoseconds since it was >> displayed, and *then* present frame N+1, which is the frame I am >> now submitting' > > That's not what this extension does; if you wanted frame 'N' to be > displayed for 'P' nanoseconds, then you would specify 'P' when > queuing frame 'N'. Hmm, I didn't fully realize this in my reading of the draft, thanks Matias for pointing it out! That the timing of frame N+1 has to be specified when submitting frame N for presentation is odd to me TBH. While I can imagine this might be suitable for some apps such as video players, I'm skeptical about game engines. They would need to calculate frame time budget and choose simulation time for frame N+1 before submitting frame N for presentation. Is that really how game engines (want to) work? Instead, have you considered allowing the GOOGLE_display_timing desiredPresentTime to be specified as a relative time, measured from when the previous image of the swapchain was actually presented, instead of as an absolute time? Could something like that also address the motivation for VK_MESA_present_period? - -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSwn681vpFFIZgJURRaga+OatuyAAUCXjlKBwAKCRBaga+Oatuy ADDxAJ49XUxG6tnAWuC/br12sORqQSBUyACgk7h1jf9fjjtMYvrkIfUnkcHwkqg= =m2sK -END PGP SIGNATURE- ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
"Matias N. Goldberg" writes: > I read your article. Thanks! > What I feel are missing are just minor pesky details: Yes, it's definitely a rough draft at best. Figuring out the right words to make it do precisely what we want is hard, and I'm hoping we can first figure out what we want, then figure out how to say that precisely. > 1. Written as is, the frame being submitted is rounded up to display > timing, delaying *future* frames.But there is no way to delay the > *currently displaying frame* (i.e. the 'previous' frame). Correct. The period provided controls how long the specified frame will be shown to the user (at a minimum). Future frames will be delayed by at least that long. > Right now if frame N was submitted without VkPresentTimeMESA but frame > N+1 is, then frame N+1 will be presented to screen ASAP. Correct. > What I'm saying is that if frame N was submitted without > VkPresentTimeMESA, then at frame N+1 I should be able to tell 'keep > frame N displayed for at least P nanoseconds since it was displayed, > and *then* present frame N+1, which is the frame I am now submitting' That's not what this extension does; if you wanted frame 'N' to be displayed for 'P' nanoseconds, then you would specify 'P' when queuing frame 'N'. > The API needs to be expanded further to explain Vulkan what a 'frame' > is.Is it the monitor's refresh rate? KHR_swapchain and EXT_display_surface_counter both mention 'vertical blanking periods'. GOOGLE_display_timing has vkGetRefreshCycleDurationGOOGLE which returns 'refreshDuration'. 'frame' is also used throughout Vulkan and seems to describe one of a sequence of images displayed to the user. Coming up with language which ties back into all of these would be helpful. > Or is it an arbitrary elapsed time defined in the form numerator and > denominator? (e.g. 60hz is numerator = 1, denominator = 60; 59.94hz is > numerator = 1001 denominator = 6000)By specifying arbitrary > definitions of a frame, it is possible to be compatible with variable > refresh rates e.g. for VRR monitors, applications may define > denominator = 240 or denominator = 120 When I talked about 'frames' in my informal description, I could have clarified that by referring to the GOOGLE_display_timing 'refreshDuration' value. I think that variable refresh rate monitors would be expected to set that to a useful value, possibly based on the nominal display period, but I don't know. I think that we'll need some more Vulkan extensions designed to expose the capabilities and limits of variable refresh rate monitors. Do we need to do that in conjunction with this extension, or can we define this extension in such a way that it should work with whatever we end up with in the future? -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Michel Dänzer writes: > Instead of rounding to the nearest upcoming frame, should it round to > the earliest frame after the specified period has passed? As written, it > seems to contradict the next paragraph below: Sorry for the poor wording; let me describe what I want informally here. For nanosecond periods to be easy to use on fixed-refresh-rate monitors, I want the näive computation to "just work". For a given refresh period, 'r', I want to select a period of 'n' frames using: computed_period = n * r Because of clock skew and rounding problems, it's quite possible that the period could easily end up being just slightly smaller than that: actual_period = n * r - ε When I said 'nearest', what I intended to describe was that the target frame would be as close as possible to the specified time. > (I'm not ruling out that rounding to nearest might be better, but there > should be a rationale for it, which also justifies being inconsistent > with GOOGLE_display_timing) Yes, this is intentionally inconsistent with GOOGLE_display_timing, which I believe is hard to use correctly. By specifying not before semantics, GOOGLE_display_timing requires applications compute a fake time guaranteed to be not after the actual target frame time. This really messes things up when you might have variable refresh rate monitors. I just went and read the time-based stuff from the X Present extension. That uses 'nearest' semantics too, when supported by the driver. When not supported, the server gets to do whatever it likes (oops!). > I like this extension in general. Thanks! I've been trying to get this posted for a few months now. > However, I think allowing the period to be specified in frames might be > a mistake, because it won't work well with variable refresh rate. But > it'll be tempting for application / toolkit developers not to bother > with proper time calculations but to just use frames instead. (It would > be good to seek feedback on this from AMD DC developers and the larger > DRM kernel driver community as well) Yeah, given that the application will need the refresh period to know what to display, it certainly doesn't provide much technical benefit here. I stuck it in to make the extension feel like GL's swap interval stuff (although it isn't the same), and because it seemed like a 'nice' thing to offer applications. > P.S. It would be good to create a WIP merge request for this in the main > Mesa project, to have a central place for gathering feedback and > tracking progress. Done, thanks for the suggestion. -- -keith signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
Hi! I read your article. What I feel are missing are just minor pesky details: 1. Written as is, the frame being submitted is rounded up to display timing, delaying *future* frames.But there is no way to delay the *currently displaying frame* (i.e. the 'previous' frame). Right now if frame N was submitted without VkPresentTimeMESA but frame N+1 is, then frame N+1 will be presented to screen ASAP. What I'm saying is that if frame N was submitted without VkPresentTimeMESA, then at frame N+1 I should be able to tell 'keep frame N displayed for at least P nanoseconds since it was displayed, and *then* present frame N+1, which is the frame I am now submitting' > However, I think allowing the period to be specified in frames might be> a > mistake, because it won't work well with variable refresh rate The API needs to be expanded further to explain Vulkan what a 'frame' is.Is it the monitor's refresh rate? Or is it an arbitrary elapsed time defined in the form numerator and denominator? (e.g. 60hz is numerator = 1, denominator = 60; 59.94hz is numerator = 1001 denominator = 6000)By specifying arbitrary definitions of a frame, it is possible to be compatible with variable refresh rates e.g. for VRR monitors, applications may define denominator = 240 or denominator = 120 It should also be possible to dynamically change how long a frame lasts, in case the GPU simply can't catch up (e.g. specifying a denominator = 240 and using frames when the GPU can only render at 30hz is almost the same as presenting ASAP i.e. the same as not using VK_MESA_present_period at all) Specifying denominator = 0 means using the monitor's native refresh rate. If such concept makes no sense (e.g. VRR?) then the behavior fallbacks to some unspecified low value (like denominator = 240) or some other vendor-defined behavior.The value is unspecified because vendors will likely want to adjust this to an optimal value (e.g. controlled via driver settings in the Control Panel, defined by the Monitor manufacturer, defined by the GPU vendor, etc). CheersMatias___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Playing with display timing -- VK_MESA_present_period
On 2020-02-02 7:51 a.m., Keith Packard wrote:
>
> I spent some time over the last week experimenting with a different way
> of doing frame timing.
>
> Instead of specifying *when* a particular frame should be displayed, how
> about we specify how *long* a particular frame should be made visible
> to the user?
>
> This has a couple of advantages over the approach taken in
> GOOGLE_display_timing:
>
> 1. It provides information to the backend about frame timing
> a frame earlier.
>
> 2. Missing a frame can generate fewer artifacts.
>
> Here's what I'm thinking the extension would look like:
>
> An application uses VK_MESA_present_period by including a
> VkPresentPeriodMESA structure in the pNext chain in the
> VkPresentInfoKHR structure passed to the vkQueuePresentKHR call.
>
> typedef struct VkPresentPeriodMESA {
> VkStructureTypesType;
> const void*pNext;
> uint32_t swapchainCount;
> const int64_t* pPresentPeriods;
> } VkPresentPeriodMESA;
>
> The fields in this structure are:
>
> * sType. Set to VK_STRUCTURE_TYPE_PRESENT_PERIOD_MESA
> * pNext. Points to the next extension structure in the chain (if any).
> * swapchainCount. A copy of the swapchainCount field in the
>VkPresentInfoKHR structure.
> * pPresentPeriods. An array, length swapchainCount, of presentation
>periods for each image in the call.
>
> Positive presentation periods represent nanoseconds. Negative
> presentation periods represent frames. A zero value means the
> extension does not affect the associated presentation.
> Nanosecond values are rounded to the nearest upcoming frame so that a
> value of n * refresh_interval is the same as using a value of n
> frames.
Instead of rounding to the nearest upcoming frame, should it round to
the earliest frame after the specified period has passed? As written, it
seems to contradict the next paragraph below:
(I'm not ruling out that rounding to nearest might be better, but there
should be a rationale for it, which also justifies being inconsistent
with GOOGLE_display_timing)
> The presentation period causes *future* images to be delayed at least
> until the specified interval after this image has been
> presented. Specifying both a presentation period in a previous frame
> and using GOOGLE_display_timing is well defined -- the presentation
> will be delayed until the later of the two times.
>
> I've got this kludged together to experiment with; I managed to make it
> work purely within Vulkan using the infrastructure built for
> GOOGLE_display_timing.
>
> https://gitlab.freedesktop.org/keithp/mesa/commits/present-period
>
> I wrote a longer article on my blog:
>
> https://keithp.com/blogs/present-period/
>
> I'm interested in hearing what people think about the general approach.
Repeating my comment on
https://gitlab.freedesktop.org/keithp/mesa/commit/c81c6ffe335032d7c88b68f83360597f7dfd5b60
(hadn't got to this post yet):
I like this extension in general.
However, I think allowing the period to be specified in frames might be
a mistake, because it won't work well with variable refresh rate. But
it'll be tempting for application / toolkit developers not to bother
with proper time calculations but to just use frames instead. (It would
be good to seek feedback on this from AMD DC developers and the larger
DRM kernel driver community as well)
P.S. It would be good to create a WIP merge request for this in the main
Mesa project, to have a central place for gathering feedback and
tracking progress.
--
Earthling Michel Dänzer | https://redhat.com
Libre software enthusiast | Mesa and X developer
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
[Mesa-dev] Playing with display timing -- VK_MESA_present_period
I spent some time over the last week experimenting with a different way
of doing frame timing.
Instead of specifying *when* a particular frame should be displayed, how
about we specify how *long* a particular frame should be made visible
to the user?
This has a couple of advantages over the approach taken in
GOOGLE_display_timing:
1. It provides information to the backend about frame timing
a frame earlier.
2. Missing a frame can generate fewer artifacts.
Here's what I'm thinking the extension would look like:
An application uses VK_MESA_present_period by including a
VkPresentPeriodMESA structure in the pNext chain in the
VkPresentInfoKHR structure passed to the vkQueuePresentKHR call.
typedef struct VkPresentPeriodMESA {
VkStructureTypesType;
const void*pNext;
uint32_t swapchainCount;
const int64_t* pPresentPeriods;
} VkPresentPeriodMESA;
The fields in this structure are:
* sType. Set to VK_STRUCTURE_TYPE_PRESENT_PERIOD_MESA
* pNext. Points to the next extension structure in the chain (if any).
* swapchainCount. A copy of the swapchainCount field in the
VkPresentInfoKHR structure.
* pPresentPeriods. An array, length swapchainCount, of presentation
periods for each image in the call.
Positive presentation periods represent nanoseconds. Negative
presentation periods represent frames. A zero value means the
extension does not affect the associated presentation.
Nanosecond values are rounded to the nearest upcoming frame so that a
value of n * refresh_interval is the same as using a value of n
frames.
The presentation period causes *future* images to be delayed at least
until the specified interval after this image has been
presented. Specifying both a presentation period in a previous frame
and using GOOGLE_display_timing is well defined -- the presentation
will be delayed until the later of the two times.
I've got this kludged together to experiment with; I managed to make it
work purely within Vulkan using the infrastructure built for
GOOGLE_display_timing.
https://gitlab.freedesktop.org/keithp/mesa/commits/present-period
I wrote a longer article on my blog:
https://keithp.com/blogs/present-period/
I'm interested in hearing what people think about the general approach.
--
-keith
signature.asc
Description: PGP signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
