On May 28, 2025, at 1:21 PM, Christian Kastner <[email protected]> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. On 2025-05-28 00:53, Petitpierre, Arthur wrote: On May 23, 2025, at 11:56 AM, Christian Kastner <[email protected]> wrote: Graviton4 is a neoverse-v2 core, so armv9.0-a. Let me know if you need help to get access to Graviton4-based instances. Thanks for the offer! I don’t know for the other cloud providers, but on AWS the only way to run your own hypervisor will be to run on .metal instances (ex: c8g.metal-24xl), as nested is disabled on virtual instances. Hm, I need to go as low as armv8.0-a as apparently, that is Debian's baseline for arm64 [1]. But there is a simple compromise: Graviton4 for armv9.0-a, and some older but well supported SBC for QEMU+kvm and armv8.N-a. I realize that it might not make too much sense to run llama.cpp on an RPi, but it's a use case upstream supports, so I think the package should also do that. Graviton2 (aka c6g/m6g) would give you armv8.2-a, and for armv8.0-a you have plenty of options. Arthur -- Arthur Petitpierre WW Graviton & EC2 Performance SSA :: Amazon Web Services E-mail: [email protected] :: Cell-phone: +1 (425) 436 9327
