The problem is not actually restricted to arm64 - I would expect it will also effect all other non-x86 platforms.
The attached patch disables the memory scanning for all architectures other than i386 and x86_64, and enables it for all UEFI architectures that provide an SMBIOS entry point address in /sys/firmware/efi/systab.
smbios-noscan.patch
Description: Binary data

