# HG changeset patch # User Praveen Tiwari # Date 1408965859 -19800 # Node ID c09f34b0ab57b4ce2f5cf4aa59c25d20eb6cbd54 # Parent fdb2a3fa112229319d63b43fb1946b9b90782923 signBitHidingHDQ optimization, downscaling coeff from int32_t* to int16_t*
diff -r fdb2a3fa1122 -r c09f34b0ab57 source/common/quant.cpp --- a/source/common/quant.cpp Mon Aug 25 16:26:49 2014 +0530 +++ b/source/common/quant.cpp Mon Aug 25 16:54:19 2014 +0530 @@ -200,7 +200,7 @@ } /* To minimize the distortion only. No rate is considered */ -uint32_t Quant::signBitHidingHDQ(coeff_t* coeff, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codeParams) +uint32_t Quant::signBitHidingHDQ(int16_t* coeff, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codeParams) { const uint32_t log2TrSizeCG = codeParams.log2TrSizeCG; const uint16_t *scan = codeParams.scan; @@ -235,7 +235,8 @@ if (signbit != (absSum & 0x1)) // compare signbit with sum_parity { - int minCostInc = MAX_INT, minPos = -1, finalChange = 0, curCost = MAX_INT, curChange = 0; + int minCostInc = MAX_INT, minPos = -1, curCost = MAX_INT; + int16_t finalChange = 0, curChange = 0; for (n = (lastCG ? lastNZPosInCG : SCAN_SET_SIZE - 1); n >= 0; --n) { @@ -396,9 +397,24 @@ if (numSig >= 2 && cu->m_slice->m_pps->bSignHideEnabled) { + /* This section of code is to safely convert int32_t coefficients to int16_t, once the caller function is + * optimize to take coefficients as int16_t*, it will be cleanse.*/ + ALIGN_VAR_16(int16_t, qCoeff[32 * 32]); + for (int i = 0; i < numCoeff; i++) + { + qCoeff[i] = (int16_t)Clip3(-32768, 32767, coeff[i]); + } TUEntropyCodingParameters codeParams; cu->getTUEntropyCodingParameters(codeParams, absPartIdx, log2TrSize, isLuma); - return signBitHidingHDQ(coeff, deltaU, numSig, codeParams); + uint32_t numSign = signBitHidingHDQ(qCoeff, deltaU, numSig, codeParams); + + /* This section of code is to safely convert int32_t coefficients to int16_t, once the caller function is + * optimize to take coefficients as int16_t*, it will be cleanse.*/ + for (int i = 0; i < numCoeff; i++) + { + coeff[i] = qCoeff[i]; + } + return numSign; } else return numSig; diff -r fdb2a3fa1122 -r c09f34b0ab57 source/common/quant.h --- a/source/common/quant.h Mon Aug 25 16:26:49 2014 +0530 +++ b/source/common/quant.h Mon Aug 25 16:54:19 2014 +0530 @@ -106,7 +106,7 @@ void setChromaQP(int qpin, TextType ttype, int chFmt); - uint32_t signBitHidingHDQ(coeff_t* qcoeff, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codingParameters); + uint32_t signBitHidingHDQ(int16_t* qcoeff, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codingParameters); uint32_t rdoQuant(TComDataCU* cu, int16_t* dstCoeff, uint32_t log2TrSize, TextType ttype, uint32_t absPartIdx, bool usePsy); inline uint32_t getRateLast(uint32_t posx, uint32_t posy) const; _______________________________________________ x265-devel mailing list x265-devel@videolan.org https://mailman.videolan.org/listinfo/x265-devel