Diego Biurrun <[email protected]> writes: > I don't see that saving our skin, what if chset->downmix_ncoeffs is equal > to DCA_XLL_DMIX_NCOEFFS_MAX? Why shouldn't it overflow on the second > iteration if chset->channels is 16?
The thing is that chset->downmix_ncoeffs is the total number of coefficients which will be read in the loop (and they are stored consecutively, even if that's not entirely obvious), and if that value happens to equal DCA_XLL_DMIX_NCOEFFS_MAX, that just means that the values fit precisely in the buffer. And I don't think it is possible (in the !chset->primary_ch_set case, which I think is what we're discussing) to have chset->channels == 16 and downmix_ncoeffs <= DCA_XLL_DMIX_NCOEFFS_MAX. Since xll_channels is the total number of channels in all channel sets, it will also be at least 16, so chset->channels == 16 implies that downmix_ncoeffs >= 16 * (16 + 1) = 272. > That's probably a good idea, care to prepare a patch? :) I'm afraid I don't have time for libav hacking in the near future. I'd suggest just returning failure for now. I have no idea how many downmix coefficients are used in practice, so if/when there are samples which hit the current limit, we can decide if it's good enough to just raise the limit a bit, or do proper dynamic allocation to handle unusual cases with a large number of coefficients. Regards, /Niels -- Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
