Cedric's gonna hate me, all these long emails. On Sun, 3 Jan 2016 22:19:53 -0500 Christopher Michael <[email protected]> wrote:
> On 01/03/2016 08:11 PM, David Seikel wrote: > > On Sun, 3 Jan 2016 16:46:43 -0500 Christopher Michael > > <[email protected]> wrote: > > > >> On 01/03/2016 03:51 PM, Cedric BAIL wrote: > >>> On Sun, Jan 3, 2016 at 9:09 PM, David Seikel <[email protected]> > >>> wrote: > >>>> I've recently come out of a contract where I was adding Oculus > >>>> Rift support to an existing application. This is something I > >>>> always wanted to play with. Having had that experience, now I > >>>> want to add generic HMD (Head Mounted Display, like the Rift) > >>>> support to Evas_3D. It will be great for my major, years long, > >>>> virtual worlds project. > >>>> > >>>> There's a few different HMDs being released soonish, many > >>>> variations on Google Cardboard have been out for a while, Samsung > >>>> Gear VR was released last month, Oculus Rift is likely to be > >>>> released in the next month or three, HTC Vive probably released > >>>> later this year, and many more. > >>>> > >>>> http://www.hypergridbusiness.com/faq/best-virtual-reality-headsets/ > >>>> lists dozens of them (mostly Cardboard variations). > >>>> > >>>> I was using an Oculus Rift DK2 (Developer Kit 2) supplied by the > >>>> client for this contract. I have to hand that back in about a > >>>> week, but I'll be buying my own Oculus Rift when it gets > >>>> released. I'll probably get a Google Cardboard style device as > >>>> well. > >>>> > >>>> The contract used the Oculus SDK, but obviously that only handles > >>>> Oculus HMDs, and these days only on Windows. Oculus used to > >>>> support Linux and Mac OS X, but they dropped that to concentrate > >>>> on Windows for the release. Oculus claim the other OS support > >>>> will return. There's a couple of open source, cross platform, > >>>> generic HMD libraries that are trying to handle a bunch of HMDs, > >>>> and that's what I'd rather be using for EFL. It's still early > >>>> days for this tech, so no standards or anything yet. > >>>> > >>>> http://hg.sitedethib.com/libvr > >>>> > >>>> http://openhmd.net/ > >>>> > >>>> And a conversion of the Oculus SDK (horrid bit of C++ coding, not > >>>> all of it open surce) into open source C - > >>>> > >>>> https://github.com/ultranbrown/libovr_nsb > >>>> > >>>> What do people think of adding generic HMD support to Evas_3D? > >>> > >>> I am guessing that what you really need to add is support for > >>> stereoscopic output. > > > > That would be a part of it, yes. That was the starting point of my > > contract, support stereoscopic output first, then morph that into a > > Rift display. Stereoscopic display is an easy bit. > > > > You also need to distort the result, not just be stereoscopic. The > > distortion remaps the resulting image so that when it eventually > > hits your eyes, the distortion of the HMD lenses is compensated > > for. Not that tricky, shaders do it easily. Basically the lenses > > needed to make sure ordinary people can focus on the screen that is > > a few mere centimetres away from your eyes, introduce barrel > > distortion, so the reverse pin cushion distortion has to be pre > > applied, so it all comes out straight in the end. > > > > Then there's also dealing with rotation and position sensors on > > HMDs. They do their magic by detecting where your head is at and > > which direction it is pointing, so the software can adjust the view > > into the 3D world as you move your head around. This is the very > > crucial part, due to VR sickness. > > > > It's crucial that this head tracking / update display thing happens > > with absolutely bare minimum latency. Basically the more latency > > in a HMD, the more likely the wearer is to throw up, and when > > wearing a HMD, you can't see the real world, so you are likely to > > throw up all over your expensive computer, missing the conveniently > > placed bucket entirely. > > > > So called VR sickness is actually a very crucial issue. It's > > similar to motion sickness, only reversed. The world is moving for > > your eyes, but your inner ear disagrees, which triggers a > > "something is terribly wrong" response in your body, which tends to > > make you throw up. Your body thinks that maybe the world is > > screwed up coz of what ever you ate last, so tries to get rid of > > that as a first level emergency response. So you throw up. I'm no > > doctor, this is how it was explained to me. There's lots of > > ongoing research to reduce this problem. In general though, most > > people can "get their VR legs" as it's called, slowly getting used > > to it by short initial exposures, getting longer as you feel more > > comfy. > > > > Motion sickness is similar, with exactly the same response. Only > > the world is still for your eyes, but your inner ear thinks you are > > moving. > > > > I don't suffer from either problem myself, but it's something you > > have to be aware of. The amount of times I have thrown up in my > > five and a half decade life can be counted on one hand, with plenty > > of fingers left over. VR sickness a BIIIG topic of conversation > > for HMD developers. > > > > Either way, the likely hood of throwing up and feeling bad is high > > for some people. There's heaps of discussions about how to avoid > > these problems. There's a medical checklist I go through when > > introducing new people to HMDs. Rift in particular insists on > > displaying a health and safety screen when you start. > > > > Yes, HMD developers can be obsessive about preventing VR sickness. > > Coz if you are not, people get ill, people stop using your stuff, > > people sue you for making them ill, people give you a bad rep, ... > > it's not good, best to avoid it. > > > > These considerations exist for most, if not all, HMDs. > > > >>> This is definitively one of the reason why we > >>> have added Evas_3D. There is still some work to be done to reach > >>> the point where that would work out of the box. The step I see so > >>> far are : > >>> - Negotiate with the compositor over Wayland protocol if it can > >>> handle a 3d stereoscopic output > >>> - Add detection of 3d stereoscopic output to Enlightenment and > >>> ecore_drm backend (If I remember correctly Intel GPU support under > >>> Linux 3d output over HDMI somewhere in libdrm) > > > > Intel GPUs are not supported well for HMDs, they tend to not be > > grunty enough. It's early days, VR sickness is a thing, so > > generally really high end graphics cards are needed. In fact, Rift > > has trouble with nVidia Optimus graphics chips that are very common > > in laptops. Optimus puts an Intel GPU in front of an nVidia GPU. > > Everything has to go through the Intel chip, coz the nVidia > > connects to the monitors THROUGH the Intel chip. This introduces > > extra latency that Oculus spit the dummy on. They no longer > > support Optimus, in fact they deliberately refuse to work on > > Optimus chips since their 0.7 SDK. 0.6 and earlier worked fine on > > Optimus though. I think they are just being precious. The Windows > > desktop supplied by my client has Optimus, and the client > > themselves will be using Optimus based laptops. So fuck you > > Oculus, they are sticking with SDK 0.6. > > > > Personally, from my experience supporting Second Life / OpenSim > > users professionally, most people tend to use low end student / > > business laptops for that, coz they are cheap and plentiful. Which > > tend to have a hard time with SL / OS, and will have a harder time > > with HMDs. SL / OS is horrible code base, I'm sure EFL based code > > could run much faster. One of my goals is to make sure these low > > powered laptops can actually get a useful display out of them. > > Which is sorta the exact opposite of what Oculus has done, they are > > being precious about making sure VR is well accepted by the world > > in general, so they just refuse to run on anything slow. > > > >> Correct. > >> > >> /** > >> * DRM_CLIENT_CAP_STEREO_3D > >> * > >> * if set to 1, the DRM core will expose the stereo 3D > >> capabilities of the > >> * monitor by advertising the supported 3D layouts in the flags > >> of struct > >> * drm_mode_modeinfo. > >> */ > >> #define DRM_CLIENT_CAP_STEREO_3D 1 > >> > >> So basically, get the mode info and check the "flags" of the mode > >> struct for supported layouts. Possible values on "flags": > >> > >> #define DRM_MODE_FLAG_3D_MASK (0x1f<<14) > >> #define DRM_MODE_FLAG_3D_NONE (0<<14) > >> #define DRM_MODE_FLAG_3D_FRAME_PACKING (1<<14) > >> #define DRM_MODE_FLAG_3D_FIELD_ALTERNATIVE (2<<14) > >> #define DRM_MODE_FLAG_3D_LINE_ALTERNATIVE (3<<14) > >> #define DRM_MODE_FLAG_3D_SIDE_BY_SIDE_FULL (4<<14) > >> #define DRM_MODE_FLAG_3D_L_DEPTH (5<<14) > >> #define DRM_MODE_FLAG_3D_L_DEPTH_GFX_GFX_DEPTH (6<<14) > >> #define DRM_MODE_FLAG_3D_TOP_AND_BOTTOM (7<<14) > >> #define DRM_MODE_FLAG_3D_SIDE_BY_SIDE_HALF (8<<14) > >> > >> This would/should actually be quite simple to detect in the > >> ecore_drm code as we already fetch mode info for crtcs. Would just > >> be a matter of checking for these flags. > > > > Not quite so correct. Sure this probably works fine under DRM / > > Wayland, but I mentioned Mac OS X and Windows as well as Linux. > > Sure. I defer to your wisdom in those areas ;) I was mearly stating > that (as far as code goes), it would be fairly simple (wrt libdrm) to > add these pieces to ecore_drm. Yep, looks simple enough. Still wont be useful for Mac and Windows support, but at least that's Linux catered for in the "detect a HMD that wants to be a monitor" stakes. As I pointed out, not all HMDs present themselves as monitors. In the end though, I expect the existing generic HMD libraries deal with that for us, if they don't, I can send patches. That's where HMD device detection and configuration should live. > > I don't think that's gonna work on these OSes. Even in Linux, non > > Wayland support would be needed. Frame buffer support, and the > > others, might be good to, but I had not considered that yet. > > > > Well, utilizing libdrm, it Should be Display Server independant. > Frame buffer (ie: kernel mode setting) is supporting nicely via libdrm Cool. > > I actually don't know what DRM is, other than it also stands for > > Digital Rights Management, which is what Google will show me. Is > > there a short "high level coder" introduction to what DRM is all > > about, so I can quickly get up to speed? > > Not the same "DRM" you are thinking of mate ;) It is the same as I was thinking, I was just pointing out that "DRM" is ALSO used for that other thing, and that other thing is likely to saturate Google searches too much to be useful. Pirates are way more numerous than low level X developers. lol > Direct Rendering Manager > (https://en.wikipedia.org/wiki/Direct_Rendering_Manager, > http://dri.freedesktop.org/wiki/DRM/, > http://events.linuxfoundation.org/sites/events/files/slides/brezillon-drm-kms.pdf, > > https://www.bitwiz.org.uk/s/how-dri-and-drm-work.html) > > (just a couple o links ... use your google-ness) ;) > > Think "kernel mode setting and frame buffer management" .. (ie: > pushing pixels to the console) ;) So basically, it's a low level graphics API that I probably do use, since it's at the base of X and the other things? > > > > I don't use either, but would be happy to support them, I just don't > > want it to be the only way. Is EFL Wayland ready for me yet DH? > > > > EFL has needed your assistance for a couple of years now my old > friend ;) I've been busy with sysadmin work the last three years. Longest job I ever had. That finished in October, then I got this contract doing HMD stuff, which finished just before Christmas. It paid well, I can spend a couple of months not working for a living, they will likely have more work for me later. B-) On the other hand, I WAS going to rework the Edje Lua stuff, but someone else beat me to it. Left me with not much left to do in EFL, I don't think there's any part of EFL that I'm still considered the maintainer of. Ecore_Exe has been mostly fine, not a lot of fixes needed, and Efreet dumped shit all over my FDO attempt before I could finish it, coz my under construction code was apparently too ugly. That last one really pissed me off no end, stopped me from being interested in developing EFL itself. So I've been concentrating on using EFL as an ordinary developer, instead of developing EFL itself. Helps to be on both sides of that fence, you can see where the other side is coming from. (Efreet rant - It's like walking into a building site, looking at all the piles of dirt, half finished walls, mud, random tools, and unkempt builders, then saying "this is too ugly to live, tear it down and build a temple". Of course it's ugly, it's still being built. FFS) Virtual worlds is my passion though, and I'll be wanting to add things to EFL to support that. Like HMD support. B-) I may even write an alternate widget set for EFL. Elementary is OK, but it doesn't do things the way I like them. Rewrites are a way of life in EFL, but at least I'll wait until Elementary is mature before rewriting it. I've already decided that the Elementary menu system sucks too much. I've been planning a widget set (or more accurately Not A Widget Set, or NAWS) for a very long time. The basic idea being that you really only need one widget type, if it's flexible enough, then you can build arbitrary widget types by bundling together lots of this one type of widget. I'll be turning 55 later this month, and a year later I should be able to at least semi-retire, unless this horrid gubermit changes the rules again. I'll have much more free development time then. > EFL Wayland is functional (99.9% under Weston I would say)... and I > would dare to say that Enlightenment Wayland is functional now also > (for some of us anyway) and provides an Almost exact experience as > X11 desktop. > > Of course, this is all dependant on being able to Actually Get > Wayland/Weston running (ie: does Not work on Some hardware like > nvidia binary blobs), but does work with Intel OSS drivers, nouveau > drivers work well also, and I hear that some AMD drivers work too... Lack of support for nVidia binary blobs is going to be an issue. nVidia have only recently added HMD support to their drivers, and those changes will take longer to get to the open source drivers, if at all. AMD might be in the same boat. Intel GPUs I don't think cater to HMDs explicitly. Perhaps this is one reason why Oculus "temporarily" dropped Linux support, if DRM wont let us use nVidia binary blobs? > However, you will be happy to know that nothing really has to change > on your end ;) Things are designed (wrt EFL & Wayland) so that: Write > once, run anywhere ;) If your app functions in X11, then it Should > function in WL also (assuming you are not making direct X11 calls) > (read: not using ecore_x directly but rather using ecore_evas where > possible) Which is the plan, to use EFL stuff, not lower level APIs. Still leaves Mac and Windows out in the cold, unless EFL can deal with these things on those OSes transparently to me. On the other hand, I mentioned how important latency is, might be needing some assembler to help with that. The generic HMD libraries should take care of the low level nitty gritty for us, working on Linux, Mac, and Windows. So we can concentrate on the higher level stuff. For instance, the libraries likely take care of the post render distortion for us, we would just need to grab their shader and feed it into our shader infrastructure. The head tracking (and eye tracking, which is coming) is dealt with by the libraries, and we just get a position and rotation to feed into our rendering pipeline. At least in theory, last I checked they where still working on reverse engineering the Rift position stuff. Dunno how well the other HMDs are supported, I don't have any yet. I think they where close to having that completed for Rift though. The Rift position tracking uses a bunch of IR LEDs at fixed spots on the body of the Rift, including the back so you can turn around all the way, and a synced IR camera. The LEDs blink in certain patterns, so the system can identify which is which, and the precise positions of those LEDs can be read from the Rift (they are factory calibrated I believe). So the software reads the map of the LEDs, looks at the video from the camera, tries to figure out which LEDs it can see blinking are which, builds a 3D model of the LEDs, and uses that to figure out where it is in space relative to the camera. HTV Vive uses lasers that scan for printed markers AND LEDs, they can do an entire room full of people, where the Rift is only tracking a single person. With that sort of stuff going on, plus things like how far apart your eyes are, the distance between your neck and eyes (your head pivots around a point in your neck), and other esoteric measurements, lots of complicated geometry is performed. This is why I'm happy others are sorting that shit out. I can concentrate on other things. Yes, there's lots of room for errors in this, and lots of room for accumulated errors to screw things up. Which is why HMD makers are being anal about such things, or just ignoring it completely for a much lesser experience. > I'd be happy to help where I can. Feel free to contact with any drm, > wayland, etc, questions you may have mate ;) Thanks mate. B-) > Cheers, > dh > > > On the other hand, for their own reasons, Oculus in particular moved > > away from "just be a monitor" to "be a specialised non monitor > > device". A very controversial move. Oculus may not support "just > > be a monitor" mode in the future. Dunno about the plans of the > > other HMDs though. I suspect Oculus Rift might be one of the > > popular ones. I've not looked at the other HMDs yet to see if they > > might do something similar. > > > > Some HMDs in particular I know are NOT monitors. TrinusVR for > > example is a device on the end of a WiFi or USB connection, not a > > monitor. I think Google Cardboard is similar, in that the actual > > device is an ordinary smart phone that's not pretending to be a > > monitor. TrinusVR actually wraps Google Cardboard. > > > > Still, this HMD detection and configuration step is the most trivial > > part of the entire process. But non DRM methods will be needed as > > well. With so many HMDs on the market, and more coming, plus early > > days of this tech, there's no standards. So in the end, a manual > > process will be needed as well, which will have to fall back to > > "just be a monitor". > > > >>> - Add viewport definition and support to render evas scenegraph > >>> with a slight change in the camera definition for both left and > >>> right eyes > > > > There's also what's called an "eye buffer". The final scene is > > rendered to a buffer that is bigger than the resulting resolution, > > coz the lens compensation needs the extra info to get things > > correct. Some pixels around the edge get discarded, but pixels in > > the middle need to be high rez. > > > >>> We could also take advantage of a Z property on normal Evas_Object > >>> to have some kind of floating UI in 3D. > > > > Actually, one of the things recommended to help keep VR sickness at > > bay is to NOT have a floating UI. You should try to make the UI > > part of the 3D world. Still, floating UIs are gonna happen. My > > client in particular gets a bit ill with floating UIs that are at a > > fixed position relative to the HMD. He prefers the UIs to be stuck > > to the objects they represent. So clicking on a 3D object causes > > it's UI window to popup next to the object, and the window stays in > > a fixed position relative to that object. I worry that people > > might forget they have windows open on dozens of objects scattered > > throughout the world. lol > > > > On the other hand, research is showing that having some sort of > > fixed "cockpit" might help to avoid the sickness. The theory is > > that this provides a fixed frame of reference to help offset the > > rest of the world spinning wildly as your inner ear sits quietly in > > your office chair. So in-vehicle games are popular, there's even a > > popular game where you play the part of a truck driver, driving > > around Europe, making deliveries I think (I've not tried it, but I > > keep hearing about this game, sounds boring to me, that's my > > brother-in-laws job, and he's boring). > > > > The difference is that the cockpit surrounds you, the floating UI > > doesn't, otherwise this would seem to be contradictory. This sort > > of stuff is still being researched. > > > > In the end though, yes floating UIs will be made, but it's not > > encouraged. > > > >>> There are a also few tricks to handle, like the resolution > >>> changing between when you are turning on stereoscopic display > >>> from where you are not. > > > > Well yes, lots of tricks I've been learning for this contract. B-) > > > > The resolution of the HMDs don't tend to change though. Your main > > monitor might, and the HMD display might be mirrored to the main > > monitor, so others can watch what the HMD wearer is doing. Windows > > in particular is bad at this, the refresh rate of your main monitor > > has to match the rate of the HMD. Can of worms. > > > > In the near future however, nVidia at least (likely AMD as well) are > > adding multiple resolutions on the same screen tech. So that the > > pure blue sky can be rendered at a really low rez (it's all just > > solid blue, no need for detail), but the monster in front of you is > > rendered in full rez glory as it tears you apart, so you can fully > > appreciate each glistening drop of monster drool that drips from > > it's jaw while it chews on your limbs, and the reflection of your > > look of fear in those drool drops. > > > > Horror games are popular with HMDs. > > > >>> Anyway, that would be my idea on what to do for this, sadly I > >>> don't think we have any plan to move that forward at this point, > >>> but I will be happy to help you figure out detail on this. > > > > As I mentioned, I wont have this development Rift for much longer. > > I plan on pre-ordering the consumer version as soon as I can, I > > already have the money for it. When I get that, then I'll start > > working on EFL HMD support. > > > > I should have had a Google Cardboard already, but the one and only > > company in Australia that makes them was out of stock at the > > beginning of that contract, and they haven't gotten any more stock > > since. Pity, they are in the next city, would have been most > > convenient. They even tried to hire me to work on their stuff, > > alas they only use Unity, and I have no experience with that. I'll > > get a Google Cardboard sooner or later, they are cheap, so long as > > you already have a supported smart phone (I do). > > > > On the other hand, my reason for doing this is to support my virtual > > world work. Virtual world software has to simulate an entire world, > > there's lots and lots and lots of fiddly little details, in lots of > > different areas. So this means that it's a huge project, with lots > > and lots and lots of interesting things to implement. Gonna take a > > long time, but the beauty of this is that you never get bored, > > there's always many things you can decide to work on at any time. > > So if there's a reason to delay / pause HMD work, there's plenty of > > other stuff to keep me busy. This is my passion though, so I'll be > > working on it when ever I get free time. > > > > > > > > ------------------------------------------------------------------------------ > > > > > > > > _______________________________________________ > > enlightenment-devel mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > > > > > ------------------------------------------------------------------------------ > _______________________________________________ > enlightenment-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel -- A big old stinking pile of genius that no one wants coz there are too many silver coated monkeys in the world.
signature.asc
Description: PGP signature
------------------------------------------------------------------------------
_______________________________________________ enlightenment-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
