[GitHub] [incubator-tvm] tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime
tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime URL: https://github.com/apache/incubator-tvm/issues/5060#issuecomment-603910501 @liangfu can you try to do a arena based approach given that it is simpler? We could adopt the counter based approach to enable early free of sub-arenas(when the free counters in the arena decreases to zero, we can free the space) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime
tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime URL: https://github.com/apache/incubator-tvm/issues/5060#issuecomment-603632782 re: fragmentation issue, think the allocation strategies carefully and adopt an arena-style allocator(counter based as above) can likely resolve the issue of fragementation. In terms of the total memory cost, we can indeed found the cost out during compile time for simple graph programs This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime
tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime URL: https://github.com/apache/incubator-tvm/issues/5060#issuecomment-603374417 The workspace memory could have a different strategy. The way it works is that we create a different arena for workspace, along with a counter. - When a memory is allocated, we allocate memory from the arena, and add the counter - When a memory is de-allocated, we decrease the counter - When the counter goes to zero, we free all the memory. This will work because all workspace memory are temporal. It also guarantees a constant time allocation As a generalization. If most memory allocation happens in a RAII style lifecycle. e.g. everything de-allocates onces we exit a scope, then the counter based strategy(per scope) is should work pretty well. I am not fixated about the arena allocator, but would like to challenge us to think a bit how much simpler can we make the allocation strategy looks like given what we know about the workload. Of course, we could certainly bring sub-allocator strategies that are more complicated, or fallback to libraries when needed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime
tqchen commented on issue #5060: [uTVM][Runtime] Deprecate uTVM Standalone Runtime URL: https://github.com/apache/incubator-tvm/issues/5060#issuecomment-598802082 Cross posting to here. I think it worth to think about memory allocation strategy. Specificially, we should design an API that contains a simple allocator(which is arena like and allocate memory from a stack, and release everything once done), and use that allocator for all memories in the program(including data structures and tensors). This will completely eliminate the usage of system calls and allow the program o run in bare metal. ### Example API ```c // call use system call to get the memory, or directly points to memory segments in ucontroller UTVMAllocator* arena = UTVMCreateArena(1); // Subsequent data structures are allocated from the allocator // The free calls will recycle data into the allocator // The simplest strategy is not to recycle at all UTVMSetAllocator(arena); // normal TVM API calls ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
