junrushao commented on PR #15810:
URL: https://github.com/apache/tvm/pull/15810#issuecomment-1742214593
Unfortunately, this PR broke MLC LLM's build pipeline, more specifically,
the command below
```python
python3 -m mlc_llm.build --model dist/models/llama-2-13b-chat-hf/
--quantization q4f16_1
```
leads to the the following error message:
<details>
```
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/opt/scratch/junrushao/mlc-llm/mlc_llm/build.py", line 13, in
<module>
main()
File "/opt/scratch/junrushao/mlc-llm/mlc_llm/build.py", line 10, in main
core.build_model_from_args(parsed_args)
File "/opt/scratch/junrushao/mlc-llm/mlc_llm/core.py", line 616, in
build_model_from_args
new_params = utils.convert_weights(param_manager, params, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/mlc-llm/mlc_llm/utils.py", line 258, in
convert_weights
vm["transform_params"]()
File
"/opt/scratch/junrushao/tvm-dev/python/tvm/_ffi/_ctypes/packed_func.py", line
239, in __call__
raise_last_ffi_error()
File "/opt/scratch/junrushao/tvm-dev/python/tvm/_ffi/base.py", line 476,
in raise_last_ffi_error
raise py_err
File "/opt/scratch/junrushao/tvm-dev/src/runtime/relax_vm/vm.cc", line
634, in
tvm::runtime::relax_vm::VirtualMachineImpl::InvokeClosurePacked(tvm::runtime::ObjectRef
const&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)
clo->impl.CallPacked(TVMArgs(values.data(), tcodes.data(), args.size() +
1), rv);
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/tvm-dev/src/runtime/relax_vm/vm.cc", line
708, in operator()
*rv = static_cast<VirtualMachineImpl*>(ctx_ptr)->InvokeBytecode(gf_idx,
inputs);
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/tvm-dev/src/runtime/relax_vm/vm.cc", line
765, in tvm::runtime::relax_vm::VirtualMachineImpl::InvokeBytecode(long,
std::vector<tvm::runtime::TVMRetValue,
std::allocator<tvm::runtime::TVMRetValue> > const&)
RunLoop();
File "/opt/scratch/junrushao/tvm-dev/src/runtime/relax_vm/vm.cc", line
890, in tvm::runtime::relax_vm::VirtualMachineImpl::RunLoop()
this->RunInstrCall(curr_frame, instr);
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/tvm-dev/src/runtime/relax_vm/vm.cc", line
843, in
tvm::runtime::relax_vm::VirtualMachineImpl::RunInstrCall(tvm::runtime::relax_vm::VMFrame*,
tvm::runtime::relax_vm::Instruction)
this->InvokeClosurePacked(func_pool_[instr.func_idx], args, &ret);
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/scratch/junrushao/mlc-llm/mlc_llm/relax_model/param_manager.py", line
579, in set_item
loaded_params[i] = tvm.nd.array(computed_param, device=device_cpu)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/tvm-dev/python/tvm/runtime/ndarray.py", line
635, in array
return empty(arr.shape, arr.dtype, device, mem_scope).copyfrom(arr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/tvm-dev/python/tvm/runtime/ndarray.py", line
390, in empty
dtype = DataType(dtype)
^^^^^^^^^^^^^^^
File "/opt/scratch/junrushao/tvm-dev/python/tvm/_ffi/runtime_ctypes.py",
line 174, in __init__
raise ValueError("Do not know how to handle type %s" % type_str)
ValueError: Do not know how to handle type object
```
</details>
To briefly explain the root cause of this issue, `set_item` method, as
defined
[here](https://github.com/mlc-ai/mlc-llm/blob/4f4a93f03fed3d900605c02de575d7d5f429ed79/mlc_llm/relax_model/param_manager.py#L576-L579),
gets `computed_param=None` after this commit.
As a temporary solution, I'd love to propose that we revert this commit for
now to quickly unblock us from building any LLM model, but I'm happy to get it
back immediately as soon as the issue is fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
