> Won't that result in fixed velocity objects not moving smoothly though, since the objects won't necessarily be in the positions they should at the time of display? (I'm not getting high enough frame rates yet for this to be apparent, so nvidia on linux not supporting async reprojection is the much larger cause of nausia! *sigh*).
> I suppose in practice if you do the common sampling of time delta each frame it'd work out pretty steady, its only with flightgear's "perform so many 120Hz timesteps until we've caught up to the current time" that a 90Hz HMD refresh would result in jittery motion. I'll worry about it when I observe it! I read your note about flightgear too quickly and did not fully understand the problem properly, my apologies! My understanding is that the 120hz is a default value and can be changed to a multiple of the display refresh rate, which might be a better solution if jitter does become an issue, but i'm not very familiar with FG so i shouldn't comment too much. Either way, make sure you know it's a real issue before optimizing for it! > Does that work out fairly straightforward in the end though? I suppose it depends on nvidia, which perhaps is why the person who did the > Blender work talked about doing a final DirectX frame copy, which sounds more heavyweight than sharing swapchains between DX and GL. I've never had any big issues with WGL_NV_DX_interop2. I am forced to do a gpu-gpu copy, which is a fairly negligible cost, as the swapchains returned by WMR have attributes that prevent directly sharing these with OpenGL. Instead i share a second set of DirectX textures and then copy those back to the swapchains. If the blender guy means doing a gpu-cpu-gpu copy then that is certainly a lot more heavyweight. Mads. On Wednesday, June 23, 2021 at 12:10:45 AM UTC+2 James Hogan wrote: > Hi, > > On Tue, 15 Jun 2021 at 18:42, Mads Sandvei <sand...@gmail.com> wrote: > > I have some experience integrating OpenXR and OSG from my work on > OpenMW-VR. > > I'll share some of what i've learned > > Ooh, thanks, I'll have a peak at how you've gone about it. > > > > OSG already has a concept of stereo (which currently this code doesn't > interact with) > > OSG's multithreaded rendering works better with its own stereo method > than the slave camera method, so i would recommend integrating with this > instead. > > For example, if a user uses DrawThreadPerContext, the main thread can > continue to the update phase of the next frame immediately when the last of > slave cameras have begun its draw traversals. > > With two cameras you get two separate traversals and the main thread may > is held up until the first camera is done with its draw, costing > performance. > > Ah okay, thats very useful to know. I can see that resulting in > preferential treatment for the stereo view configuration (but that is > the case that matters most to me anyway...). > > > In my work this meant using a single doublewide framebuffer instead of > one framebuffer per eye. This is not a problem for OpenXR as you can create > a doublewide swapchain and use the subimage structure > > to control the regions rendered to each eye when composing layers. I > haven't looked to closely at whether OSG supports attaching different > framebuffers per eye so that might be a moot point. > > Makes sense. > > > It's worth noting that OSG is adding support for the GL_OVR_multiview2 > extension: https://groups.google.com/g/osg-users/c/__WujmMK5KE > > It would be worth integrating this in the future as this would easily be > the fastest stereo method, though I don't have any personal experience with > it. > > Thanks. Unfortunately its still wholly in a separate branch of OSG AFAICT? > > > > Performance is currently terrible. CPU usage and frame times don't > seem high, so its blocking excessively somewhere > > Comparing your code to mine the only notable performance issues, that > are under your control, is forcing single-threaded and the choice of stereo > method. > > The code that is blocking is the xrWaitFrame() method, which is by > design. See what i wrote below about nausea. It is okay to delay > xrWaitFrame until the first time you need the predictedDisplayTime, but not > any longer. > > > > Forcing single-threaded is undoubtably the biggest issue for performance. > > I see in your code a comment that the reason is so that no other thread > can use the GL context. > > I have never touched openvr, so it's possible to openvrviewer has a good > reason for this concern. With OpenXR i don't think there is any good reason > for this. > > agreed, its mostly a hack to avoid having to understand how OSG uses > multithreading straight away. > > > > https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_KHR_opengl_enable > > The OpenXR spec explicitly demands that the runtime does not use the > context you give it except for a specific subset of functions you would > only call from the rendering thread, which will have the context every time. > > Thats probably where my multithreaded OSG was going wrong :-) > > > Inspecting your code, the flow of openxr calls is very similar to my own > and i have no issues running the threading mode DrawThreadPerContext. But i > cannot speak for the other threading modes. > > > > > due to SteamVR changing GL context somewhere (a known bug, worked > around in the swapchain abstraction > > My understanding is that the openxr spec doesn't actually forbid this > behaviour. It only limits when when the runtime is allowed to use the > context you gave it, not whether it binds/unbinds that or other contexts. > > This doesn't sound like behaviour anyone would want, though. Maybe an > oversight in the openxr standard? > > Certainly annoying behaviour yes, It should be specified whether that > is permitted either way. > ftr: https://github.com/ValveSoftware/SteamVR-for-Linux/issues/421 > > > The runtime cost of verifying the OpenGL context after each of the > relevant functions is low since you're only doing it a handful of times per > frame, > > so it might be a good idea to just wrap all of the mentioned methods in > code that checks and restores opengl context. > > Of course, the best would be if all vendors adopted reasonable behaviour. > > Yes, thats sounds like the best we can do right now until it becomes > clearer whether the behaviour will be fixed. > > > > Advancement is ideally driven by the expected display times of > individual frames, i.e. the next frame should show the scene at exactly the > moment when it is expected to be displayed to the user to avoid jitter and > nausia. This may well be more of an app level concern (certainly is for > flightgear which AFAICT currently uses fixed 120Hz simulation steps), but a > general VR-specific viewer mainloop is probably needed in any case. > > This is the purpose of the xr[Wait,Begin,End]Frame loop, and why you're > passing the predictedDisplayTime returned by xrWaitFrame() on to > xrEndFrame(). > > > https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#frame-synchronization > > In short: you don't have to care, OpenXR is already doing this for you. > > Won't that result in fixed velocity objects not moving smoothly > though, since the objects won't necessarily be in the positions they > should at the time of display? (I'm not getting high enough frame > rates yet for this to be apparent, so nvidia on linux not supporting > async reprojection is the much larger cause of nausia! *sigh*). > > I suppose in practice if you do the common sampling of time delta each > frame it'd work out pretty steady, its only with flightgear's "perform > so many 120Hz timesteps until we've caught up to the current time" > that a 90Hz HMD refresh would result in jittery motion. I'll worry > about it when I observe it! > > > Perhaps this is the issue for the openvrviewer, that openvr doesn't have > this and so isn't automatically synchronized? > > My interpretation of xrBeginFrame is that it exists precisely so that > the next frame never begins rendering operations before the runtime is done > compositing the latest xrEndFrame. > > > > The only Nausea element you have to consider is when to locate an > XrSpace. > > When locating an XrSpace, what you get is a predicted pose for the time > you give it (usually the predictedDisplayTime you got from xrWaitFrame()). > The close you get to the predicted time, the better the prediction will be. > So it is encouraged to predict as close to draw as possible. > > By using the update slave callback, i believe you are accomplishing this > as well as can be. > > This is also the motivation for the xrWaitFrame call, it delays your > processing so that your poses will be predicted closer to the time they > will actually be displayed. > > > > For the same reason, that predictions change in quality over time, it is > encouraged to make all predictions at the same time and not spread out over > time. > > Action spaces (i.e. motion controllers) have their pose data updated > only when you sync actions, so sync these immediately before locating. I > deal with this by putting all pose actions in their own action set so they > don't get lumped together with other inputs. > > Okay, thanks for the tip. > > > > The OpenXR session is created using OpenGL graphics binding info > provided via GraphicsWindow::getXrGraphicsBinding() which is only > implemented for X11 > > Just a heads up. On windows you will find that some OpenXR runtimes, > such as WMR, do not support OpenGL. Not surprising, being microsoft's own > runtime. > > I worked around this by using the wgl extension WGL_NV_DX_interop2 to > share DirectX swapchains with OpenGL. I believe this would be the only way > to support such runtimes in OSG. > > Yeh, I spotted that mentioned for Blender's OpenXR support, but since > I don't run windows I'll let somebody else worry about implementing or > testing it :-) > Does that work out fairly straightforward in the end though? I suppose > it depends on nvidia, which perhaps is why the person who did the > Blender work talked about doing a final DirectX frame copy, which > sounds more heavyweight than sharing swapchains between DX and GL. > > > Hope this is of some help! > > Definitely! Thanks for the detailed feedback! > > Cheers > -- > James Hogan > -- You received this message because you are subscribed to the Google Groups "OpenSceneGraph Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to osg-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/38c79694-fae0-443f-90ae-f685aeef8e09n%40googlegroups.com.
