siju-samuel edited a comment on pull request #5507: URL: https://github.com/apache/incubator-tvm/pull/5507#issuecomment-625525365
@tqchen. Thanks for the review. I work in a device with very limited ram availability. Actually even a few kbs is crucial for me. Im using a quantized model and the pool is 80% unit8 data. So i dont want to allocate all pool with float and 32bit. If i consider the bit i can restrict all pool to allocate 32bit. So while allocating space, to compute `size` we either need `bits` or `dytpe`. Now both are not available in `TVMGraphRuntimePoolEntry`. So i added bits in `TVMGraphRuntimePoolEntry` As you suggested, we can compute `size` while finding maximum space needed and save to `pool_entry[sid].size` But it requires either 1. while making TVMNDArray_Empty, always use `dtype.code = kDLInt` & `dtype.bits = 8`. 2. while making TVMNDArray_Empty, use Float & 32bit (like before this change) and reduce the `size` as per `bits` and align to 4bytes. I would like to go with 2nd approach. I havnet done the modification yet. I will do the modifications, test and will update this PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
