MasterJH5574 opened a new pull request, #16578: URL: https://github.com/apache/tvm/pull/16578
This PR adds a new function to PagedKVCache to return in-sequence positions for each location in a batch of sequences that is being forwarded. This function helps apply positional embeddings for language models that do not use Rotary positional embeddings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
