Hi, On Tue, 15 Jun 2021 at 18:42, Mads Sandvei <sandv...@gmail.com> wrote: > I have some experience integrating OpenXR and OSG from my work on OpenMW-VR. > I'll share some of what i've learned
Ooh, thanks, I'll have a peak at how you've gone about it. > > OSG already has a concept of stereo (which currently this code doesn't > interact with) > OSG's multithreaded rendering works better with its own stereo method than > the slave camera method, so i would recommend integrating with this instead. > For example, if a user uses DrawThreadPerContext, the main thread can > continue to the update phase of the next frame immediately when the last of > slave cameras have begun its draw traversals. > With two cameras you get two separate traversals and the main thread may is > held up until the first camera is done with its draw, costing performance. Ah okay, thats very useful to know. I can see that resulting in preferential treatment for the stereo view configuration (but that is the case that matters most to me anyway...). > In my work this meant using a single doublewide framebuffer instead of one > framebuffer per eye. This is not a problem for OpenXR as you can create a > doublewide swapchain and use the subimage structure > to control the regions rendered to each eye when composing layers. I haven't > looked to closely at whether OSG supports attaching different framebuffers > per eye so that might be a moot point. Makes sense. > It's worth noting that OSG is adding support for the GL_OVR_multiview2 > extension: https://groups.google.com/g/osg-users/c/__WujmMK5KE > It would be worth integrating this in the future as this would easily be the > fastest stereo method, though I don't have any personal experience with it. Thanks. Unfortunately its still wholly in a separate branch of OSG AFAICT? > > Performance is currently terrible. CPU usage and frame times don't seem > high, so its blocking excessively somewhere > Comparing your code to mine the only notable performance issues, that are > under your control, is forcing single-threaded and the choice of stereo > method. > The code that is blocking is the xrWaitFrame() method, which is by design. > See what i wrote below about nausea. It is okay to delay xrWaitFrame until > the first time you need the predictedDisplayTime, but not any longer. > > Forcing single-threaded is undoubtably the biggest issue for performance. > I see in your code a comment that the reason is so that no other thread can > use the GL context. > I have never touched openvr, so it's possible to openvrviewer has a good > reason for this concern. With OpenXR i don't think there is any good reason > for this. agreed, its mostly a hack to avoid having to understand how OSG uses multithreading straight away. > https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_KHR_opengl_enable > The OpenXR spec explicitly demands that the runtime does not use the context > you give it except for a specific subset of functions you would only call > from the rendering thread, which will have the context every time. Thats probably where my multithreaded OSG was going wrong :-) > Inspecting your code, the flow of openxr calls is very similar to my own and > i have no issues running the threading mode DrawThreadPerContext. But i > cannot speak for the other threading modes. > > > due to SteamVR changing GL context somewhere (a known bug, worked around in > > the swapchain abstraction > My understanding is that the openxr spec doesn't actually forbid this > behaviour. It only limits when when the runtime is allowed to use the context > you gave it, not whether it binds/unbinds that or other contexts. > This doesn't sound like behaviour anyone would want, though. Maybe an > oversight in the openxr standard? Certainly annoying behaviour yes, It should be specified whether that is permitted either way. ftr: https://github.com/ValveSoftware/SteamVR-for-Linux/issues/421 > The runtime cost of verifying the OpenGL context after each of the relevant > functions is low since you're only doing it a handful of times per frame, > so it might be a good idea to just wrap all of the mentioned methods in code > that checks and restores opengl context. > Of course, the best would be if all vendors adopted reasonable behaviour. Yes, thats sounds like the best we can do right now until it becomes clearer whether the behaviour will be fixed. > > Advancement is ideally driven by the expected display times of individual > frames, i.e. the next frame should show the scene at exactly the moment when > it is expected to be displayed to the user to avoid jitter and nausia. This > may well be more of an app level concern (certainly is for flightgear which > AFAICT currently uses fixed 120Hz simulation steps), but a general > VR-specific viewer mainloop is probably needed in any case. > This is the purpose of the xr[Wait,Begin,End]Frame loop, and why you're > passing the predictedDisplayTime returned by xrWaitFrame() on to xrEndFrame(). > https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#frame-synchronization > In short: you don't have to care, OpenXR is already doing this for you. Won't that result in fixed velocity objects not moving smoothly though, since the objects won't necessarily be in the positions they should at the time of display? (I'm not getting high enough frame rates yet for this to be apparent, so nvidia on linux not supporting async reprojection is the much larger cause of nausia! *sigh*). I suppose in practice if you do the common sampling of time delta each frame it'd work out pretty steady, its only with flightgear's "perform so many 120Hz timesteps until we've caught up to the current time" that a 90Hz HMD refresh would result in jittery motion. I'll worry about it when I observe it! > Perhaps this is the issue for the openvrviewer, that openvr doesn't have this > and so isn't automatically synchronized? > My interpretation of xrBeginFrame is that it exists precisely so that the > next frame never begins rendering operations before the runtime is done > compositing the latest xrEndFrame. > > The only Nausea element you have to consider is when to locate an XrSpace. > When locating an XrSpace, what you get is a predicted pose for the time you > give it (usually the predictedDisplayTime you got from xrWaitFrame()). The > close you get to the predicted time, the better the prediction will be. So it > is encouraged to predict as close to draw as possible. > By using the update slave callback, i believe you are accomplishing this as > well as can be. > This is also the motivation for the xrWaitFrame call, it delays your > processing so that your poses will be predicted closer to the time they will > actually be displayed. > > For the same reason, that predictions change in quality over time, it is > encouraged to make all predictions at the same time and not spread out over > time. > Action spaces (i.e. motion controllers) have their pose data updated only > when you sync actions, so sync these immediately before locating. I deal with > this by putting all pose actions in their own action set so they don't get > lumped together with other inputs. Okay, thanks for the tip. > > The OpenXR session is created using OpenGL graphics binding info provided > via GraphicsWindow::getXrGraphicsBinding() which is only implemented for X11 > Just a heads up. On windows you will find that some OpenXR runtimes, such as > WMR, do not support OpenGL. Not surprising, being microsoft's own runtime. > I worked around this by using the wgl extension WGL_NV_DX_interop2 to share > DirectX swapchains with OpenGL. I believe this would be the only way to > support such runtimes in OSG. Yeh, I spotted that mentioned for Blender's OpenXR support, but since I don't run windows I'll let somebody else worry about implementing or testing it :-) Does that work out fairly straightforward in the end though? I suppose it depends on nvidia, which perhaps is why the person who did the Blender work talked about doing a final DirectX frame copy, which sounds more heavyweight than sharing swapchains between DX and GL. > Hope this is of some help! Definitely! Thanks for the detailed feedback! Cheers -- James Hogan -- You received this message because you are subscribed to the Google Groups "OpenSceneGraph Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to osg-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/CAAG0J9-eu09Cr07BOTd6s7RJ6hcq--zECd%3DrGQGHnsnxbtkh%2BQ%40mail.gmail.com.
