gemini-code-assist[bot] commented on code in PR #19539:
URL: https://github.com/apache/tvm/pull/19539#discussion_r3221720358
##########
src/runtime/contrib/vllm/attention_kernels.cu:
##########
@@ -768,13 +769,19 @@ TVM_FFI_STATIC_INIT_BLOCK() {
bool use_v1 =
max_context_len <= 8192 && (max_num_partitions == 1 || num_seqs *
num_heads > 512);
if (use_v1) {
- single_query_cached_kv_attention_v1(query, key_cache, value_cache,
block_tables,
- context_lens, block_size,
max_context_len_tensor,
- out);
+ single_query_cached_kv_attention_v1(
+ query.GetDLTensorPtr(), key_cache.GetDLTensorPtr(),
value_cache.GetDLTensorPtr(),
+ block_tables.GetDLTensorPtr(), context_lens.GetDLTensorPtr(),
block_size,
+ max_context_len_tensor.GetDLTensorPtr(),
const_cast<DLTensor*>(out.GetDLTensorPtr()));
Review Comment:
The use of `const_cast<DLTensor*>(out.GetDLTensorPtr())` is quite verbose
and potentially redundant if the `Tensor` handle provides a non-const pointer
via `operator->()` or `.get()`. While the PR description mentions this matches
patterns in other modules, it's worth verifying if a more direct access to the
mutable `DLTensor*` is available in the new FFI `Tensor` class to avoid
unnecessary casting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
