On Monday 11 December 2006 11:26, Frantisek Dufka wrote: > Yes, the result would look like video overlay works in windows or linux > on PC - overlay draws over different windows when it shouldn't :-) We > can live with that.
I thought it is actually not a problem but quite a good thing :-) Surely for mplayer as a standalone video player, supporting keboard/initializing some window is important. But if it just outputs video into some rectangular screen area (provided by some other application) and is controlled via issuing commands through a pipe, it makes possible to develop some advanced frontends which use mplayer as a video rendering engine. For example a twin of the standard Nokia 770 video player which simulates all its GUI controls could be created. My original goal of posting the previous message was an attempt to find a volunteer who would like to try developing such a frontend. I don't have that much time to devote to mplayer development myself. Up until this moment I even could not concentrate on solving some specific task but tried some bits with MP3 audio output, decoder improvements, GUI and user interface, fixing arm specific bugs, and now video output code with hardware YUV support. Also some kind of management work, integration of useful patches and support for users in the forums takes some time. I would like to concentrate on some task such as video decoder optimizations for ARM, but seeing that other parts are not in a quite good shape, distracts attention somewhat :-) > As for framebuffer permissions, it may be better to > relax device permissions than to run mplayer as root. The most right way to solve this issue is probably to add 'user' to 'video' group. Alternative solutions involve messing with mplayer binary ownership and suid/sgid bits. I wonder what is possible to do automatically in the least intrusive way when installing mplayer package? > Well, the conversion is done on the fly while the data is transferred to > internal epson video buffer. I guess it would be hard to do planar YUV > -> RGB without additional memory. I still don't understand how it is > done on the fly even in those packed formats since some color > information (U,V) is common for more lines. Seems like tough task. There > needs to be additional memory for remembering U,V parts from previous line. YUV422 is a good format as it matches 16-bit RGB format quite well. Both of them use 16 bits per pixel, and YUV422 encodes each pair of pixels into a stride of 4 bytes (16-bit RGB encodes each pixel into 2 bytes, but you can also treat it as 2 pixels in 4 bytes). So we can mix YUV422 and RGB data in a framebuffer quite conveniently. > > Another interesting possibility is to relay video scaling and color > > conversion (planer -> packed YUV) to DSP. > > I'm not sure, is there some math involved in this or it is just memory > shuffling? I guess DSP would be really bad for memory shuffling. From > previous discussions it looks like when you add DSP to the mix all kinds > of bottlenecks appears. I wonder if gstreamer/dspfbsink could keep up > with mplayer speed doing just conversion and video output. Actually DSP may be a good choice for scaling, if you check the same spru098.pdf you will find "Pixel interpolation Hardware Extension" part :-) Also looks like dspfbsink uses DSP for scaling as it provides a mapped memory for planar YV12 data (or its variant) and accepts a command to do the rendering. I looked through xserver sources and gst plugins to dig for information and I think I got some impression about how they work, but I think this all deserves a separate post along with some additional inquiries addressed to Nokia developers :-) ARM can perform YV12->YUV422 conversion quite fast if properly optimized, I even suspect that it can provide a throughoutput comparable to memcpy (as memory controller/write buffer performance is a limiting factor here and some data shuffling will not make much difference). The benchmarks in my previous message use standard color conversion/scaling code from mplayer which is not optimized for ARM. But just color format conversion is a special case, sometimes scaling is required and mplayer scaler is rather slow. Scaling performed by mplayer was completely unusable for RGB target colorspace with x11 driver, that's why maemo build of mplayer had fallback to SDL when playback for scaled video was required. Now with the target colorspace YUV422, it is slow but still usable and a bit better than SDL. If we want a fast scaler for ARM, using JIT is a good option (and I have some experience in developing JIT translator for x86). Anyway, I hope that by using DSP for scaling and running it asynchronously, it is possible to reduce ARM core usage to almost zero and keep all the resources for video decoding. A related interesting observation is that screen update ioctl does not seem to affect performance at all (commenting it out does not improve performace and naturally we get no picture on the screen). I suspect that screen update is performed asynchronously while ARM core continues to execute code after that ioctl call. With all that said, decoding performance is much more important than scaling. If decoder is not fast enough to decode video in time, it is completely unplayable. If the cpu resources remaining after decoding are not enough for scaling and video output, some frames are just skipped and the overall effect is not so disasterous. With a slow scaler and video output code, if we gradually increase bitrate and test playback, video will play perfectly first, then it will start skiping some frames until all the frames get skipped and video becomes unplayable. With a very fast scaler, video will play perfectly first (and remain perfect even if with a slow scaler it would get framedrops already) but as bitrate increases, it will suddenly become unplayable at some point. So the time spent for scaling and video output can be considered as some kind of backup resource which can be used to keep video watchable at the expense of dropping some frames when needed :-) > Oh BTW, it is off topic but I finally found what that cryptic 'Video > hardware accelerators for DCT, iDCT, pixel interpolation, and motion > estimation for video compression' feature on OMAP 1710 page means. Looks > like the DSP really has some special instructions for performing those > operations, google for spru098.pdf It is sad that default video player > is still so bad with such features implemented in hardware. Thanks for the information, I have found this pdf along with the other docs from TI quite a long time ago, downloaded their toolchain, but never had time/motivation to try it in action yet :-) _______________________________________________ maemo-developers mailing list maemo-developers@maemo.org https://maemo.org/mailman/listinfo/maemo-developers
