On 09/29, Deepthi Nandakumar wrote: > So, I dont think it's fair to upgrade the datatypes of these variables just > to satisfy debug/check code. We're unnecessarily adding cycles. > > I think the best solution here might be to remove these hacky checks in > Mode.invalidate(), and Mode.ok() and add more robust checking into > rdcost.h.
the functions are hacky, but they cannot be replaced by checks in rdcost.h. Their intention is not to find out-of-range function return values, but instead to find uninitialized cost variables. The functions did find a few bugs when they were first added, but they are mostly a liability now that main12 has increased the dynamic range of the pixel compare functions. What this has all shown, though, is that any variable which needs to sum up multiple distortion costs needs to be a uint64_t or you risk overflow in high bit depth builds (10 or 12). Many of the Mode variables fall into this category. It is definitely wrong for the output of an SSE primitive to be stored into a uint32_t. It needs to be either sse_ret_t, if the value is never incremented after returned, else a uint64_t. -- Steve Borho _______________________________________________ x265-devel mailing list [email protected] https://mailman.videolan.org/listinfo/x265-devel
