Hi, I tried something like this out of interest on arm64 with my own hypervisor (custom kvm, https://github.com/jkrh/kvms) and indeed I get the android vm up. Thanks!
In order to get the AOSP up I first started to port the 'ranchu' machine to qemu6, but the 'android-emu' library I'd need to hack in the qemu6 main loop just does not feel 'clean', so this remains unfinished in that regard: https://github.com/jkrh/kvms/blob/master/patches/0001-target-ranchu-add-support-for-android-ranchu-board.patch That being said, would be nice to make it do GPU too and I'm new to the graphics pipeline. From the hypervisor point of view I'd need it to do full VIRTIO since my hypervisor only opens the virtio channels between the host and the guest. Besides, I'm not entirely sure what to think of the 'pipe'. First attempt with that was to see if it would do 'gltransport=virtio': KERNEL_OPTS="root=/dev/vda rootfstype=ext4 ro init=/init selinux=1 checkreqprot=1 androidboot.selinux=permissive console=ttyAMA0 androidboot.hardware=ranchu qemu.uirenderer=skiagl qemu.gltransport=virtio qemu.gles=1 qemu.opengles.version=131072 qemu.dalvik.vm.heapsize=192m loglevel=8" and then for the QEMU: SCREEN="-nographic -device virtio-gpu-pci -spice $SPICESOCK,disable-ticketing=on $VDAGENT" Thoughts if something like this is supposed to be supported? Certainly it does not work: [ 36.445042] DEBUG: Build fingerprint: 'Android/aosp_arm64/generic_arm64:10/QT/eng.jmk.20210614.160415:userdebug/test-keys' [ 36.450901] DEBUG: Revision: '0' [ 36.458537] DEBUG: ABI: 'arm64' [ 36.462533] DEBUG: Timestamp: 2021-06-01 11:28:17+0000 [ 36.465839] DEBUG: pid: 219, tid: 219, name: surfaceflinger >>> /system/bin/surfaceflinger <<< [ 36.468971] DEBUG: uid: 1000 [ 36.474921] DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x10 [ 36.476737] DEBUG: Cause: null pointer dereference [ 36.481510] DEBUG: x0 0000000000000000 x1 0000e23fdddf66c0 x2 0000e23fd08b59f4 x3 0000e23fddcef5f8 [ 36.484085] DEBUG: x4 0000000000000002 x5 0000000000000000 x6 0000000000000000 x7 000000000000a000 [ 36.490734] DEBUG: x8 0000e23fddeef0e0 x9 0000000000000001 x10 0000e23fddeef060 x11 0000000000000000 [ 36.497538] DEBUG: x12 0000000000000000 x13 0000000000000000 x14 0000000000000020 x15 0200000000000000 [ 36.502861] DEBUG: x16 0000e23fd081da70 x17 0000e23fd08b4168 x18 0000e23fdf142000 x19 0000000000000000 [ 36.509025] DEBUG: x20 0000e23fddd192d0 x21 0000000000000001 x22 0000000000000000 x23 0000000000000000 [ 36.514828] DEBUG: x24 0000e23fddeef020 x25 0000e23fddd19280 x26 0000000000000000 x27 0000e23fddd90000 [ 36.520535] DEBUG: x28 0000e23fdc3b7070 x29 0000ffffdf3749e0 [ 36.528408] DEBUG: sp 0000ffffdf3749d0 lr 0000e23fd0813464 pc 0000e23fd08b4178 [ 36.610037] DEBUG: [ 36.615637] DEBUG: backtrace: [ 36.616452] DEBUG: #00 pc 0000000000005178 /vendor/lib64/libOpenglSystemCommon.so (HostConnection::gl2Encoder()+16) (BuildId: 8ca38ff3265c54f31e1f70e8a19c683f) [ 36.618386] DEBUG: #01 pc 0000000000013460 /vendor/lib64/egl/libGLESv2_emulation.so (glGetString+24) (BuildId: 1bc4a27ae3f25fffe3541af804629511) [ 36.628225] DEBUG: #02 pc 0000000000018d20 /system/lib64/libEGL.so (android::egl_context_t::onMakeCurrent(void*, void*)+112) (BuildId: 5abb3faa9d72952310ca01a9da8941e3) [ 36.636811] DEBUG: #03 pc 0000000000016450 /system/lib64/libEGL.so (android::egl_display_t::makeCurrent(android::egl_context_t*, android::egl_context_t*, void*, void*, void*, void*, void*, void*)+268) (BuildId: 5abb3faa9d72952310ca01a9da8941e3) [ 36.646320] DEBUG: #04 pc 000000000001ff94 /system/lib64/libEGL.so (android::eglMakeCurrentImpl(void*, void*, void*, void*)+384) (BuildId: 5abb3faa9d72952310ca01a9da8941e3) [ 36.660598] DEBUG: #05 pc 0000000000113d58 /system/lib64/libsurfaceflinger.so (android::renderengine::gl::GLESRenderEngine::create(int, unsigned int, unsigned int)+380) (BuildId: becfe6218fde450840f4bc3f14cb68c5) [ 36.670297] DEBUG: #06 pc 0000000000113aa4 /system/lib64/libsurfaceflinger.so (android::renderengine::RenderEngine::create(int, unsigned int, unsigned int)+164) (BuildId: becfe6218fde450840f4bc3f14cb68c5) [ 36.681625] DEBUG: #07 pc 00000000000ce97c /system/lib64/libsurfaceflinger.so (android::SurfaceFlinger::init()+1164) (BuildId: becfe6218fde450840f4bc3f14cb68c5) [ 36.694419] DEBUG: #08 pc 00000000000031bc /system/bin/surfaceflinger (main+364) (BuildId: ff9215aa2c7b96de71b0a256004edc79) [ 36.703441] DEBUG: #09 pc 000000000007d798 /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108) (BuildId: 676a709a0ee633ec9cf6ab05ec6410ae) -- Janne On Monday, 24 May 2021 at 21:14:44 UTC+3 [email protected] wrote: > Hi again, > > Here is the AOSP 10 on ARM64 QEMU version of the procedure I follow. > Andoid 5.10 arm64 kernel build included, like for x86_64. > Appart from using "vdb" and "vdc" into fstab instead of sdb and sdc, and > removing some "rild.rc" file, the approach is the same. With some > additionnal information at the end > > repo init -u https://android.googlesource.com/platform/manifest -b > android-10.0.0_r47 > repo sync > source build/envsetup.sh > lunch > m -j $(nproc) > > # out/target/product/generic_arm64/system.img > # out/target/product/generic_arm64/vendor.img > > Create and ext4 format the partition image files: > > qemu-img create rootfs.img 5G > qemu-img create userdata.img 1G > qemu-img create cache.img 500M > > losetup /dev/loop0 rootfs.img > losetup /dev/loop1 userdata.img > losetup /dev/loop2 cache.img > > mkfs -t ext4 /dev/loop0 > mkfs -t ext4 /dev/loop1 > mkfs -t ext4 /dev/loop2 > > losetup -d /dev/loop1 > losetup -d /dev/loop2 > > *Mouting the rootfs partition * > mount -t ext4 /dev/loop0 /where/you/want > > cp -a [system.img mount point] [rootfs/mount/point] > The content of system.img may be copied inside a folder which is named > like the mount point. Move everything 1 folder above. > > > cp -a [vendor.img mount point] [rootfs/mount/point] > If the name of the "vendor.img" mount point is "vendor", everything should > now be inside the "/vendor" folder of the rootfs. If not, you have to move > it to /vendor folder > > > Delete these files: > > /vendor/lib/hw/camera.ranchu.so > /vendor/lib/hw64/camera.ranchu.so > /vendor/lib/hw/camera.ranchu.jpeg.so > /vendor/lib/hw64/camera.ranchu.jpeg.so > /vendor/lib/hw/vendor/lib/hw/gralloc.ranchu > /vendor/lib/hw64/gralloc.ranchu > /vendor/lib/hw/hwcomposer.ranchu > /vendor/lib/hw64/hwcomposer.ranchu > /vendor/lib/hw/vulkan.ranchu.so > /vendor/lib/hw64/vulkan.ranchu.so > /vendor/etc/init/rild.rc > > > Change /vendor/etc/fstab.ranchu, those 2 lines should be enough: > > /dev/block/vdb /data ext4 > noatime,nosuid,nodev,nomblk_io_submit,errors=panic > wait,check,quota,reservedsize=128M,first_stage_mount > /dev/block/vdc /cache ext4 > noatime,nosuid,nodev,nomblk_io_submit,errors=panic > wait,check,quota,reservedsize=128M,first_stage_mount > > > Change /vendor/etc/init/hw/init.ranchu.rc by changing this line: > setprop ro.hardware.egl emulation > to > setprop ro.hardware.egl swiftshader > > > *Manufacturing an up to date 5.10 ARM64 Android kernel:* > > > git clone https://android.googlesource.com/kernel/configs > > /!\ From configs/android-5.10/android-recommended.config, if you want to > use a mouse, remove the following line : # CONFIG_INPUT_MOUSE is not set > > git clone https://android.googlesource.com/kernel/common -b android12-5.10 > > Start from copying "arch/arm64/configs/defconfig" file to the root of the > kernel tree, name it ".config", and append to it: > > CONFIG_NETFILTER_ADVANCED=y > Content of configs/android-5.10/android-base.config > Content of configs/android-5.10/android-recommended.config > Content of configs/android-5.10/android-recommended-arm64.config > > make ARCH=arm64 menuconfig, exit, save: yes. > > > By looking with some tools, some of the things included into > x86_64_defconfig, android-base.config, android-recommended.config are not > included into the resulting ".config". > It works anyway, but it would be nice to know why theese option aren't > enabling. > > Still missing from arm64/configs/defconfig > CONFIG_SYSVIPC=y > CONFIG_ACPI_APEI_PCIEAER=y > CONFIG_KSM=y > CONFIG_IP6_NF_NAT=m > CONFIG_IP6_NF_TARGET_MASQUERADE=m > CONFIG_LEGACY_PTY_COUNT=16 > CONFIG_POWER_AVS=y > CONFIG_BACKLIGHT_GENERIC=m > CONFIG_QCOM_IOMMU=y > CONFIG_NFS_FS=y > CONFIG_NFS_V4=y > CONFIG_NFS_V4_1=y > CONFIG_NFS_V4_2=y > > CONFIG_ROOT_NFS=y > > Still missing from android-base.config: > CONFIG_TRACE_GPU_MEM=y > > Still missing from android-recommended.config: > CONFIG_BACKLIGHT_LCD_SUPPORT=y > CONFIG_REFCOUNT_FULL=y > CONFIG_SDCARD_FS=y > > Now, some options that are still disabled, but should be enabled according > to android-base-conditional.xml: > CONFIG_ARMV8_DEPRECATED=y > CONFIG_CP15_BARRIER_EMULATION=y > CONFIG_SETEND_EMULATION=y > CONFIG_SHADOW_CALL_STACK=y > CONFIG_SWP_EMULATION=y > CONFIG_BPF_JIT_ALWAYS_ON=y > CONFIG_KFENCE=y > CONFIG_USERFAULTFD=y > > > > CONFIG_AIO and CONFIG_ANDROID_BINDERFS options (from android-base.config) > are causing early reboot without explanations. Would be nice to know why, > to keep them enabled and have something running... Until so, disable them > by adding at the end of .config: > # CONFIG_AIO is not set > # CONFIG_ANDROID_BINDERFS is not set > > Adding VirtIO things: > > CONFIG_VIRTIO=y > CONFIG_VIRTIO_PCI=y > CONFIG_VIRTIO_PCI_LEGACY=y > CONFIG_VIRTIO_BALLOON=y > CONFIG_VIRTIO_INPUT=y > CONFIG_VIRTIO_MMIO=y > CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y > CONFIG_VIRTIO_DMA_SHARED_BUFFER=y > CONFIG_BLK_MQ_VIRTIO=y > CONFIG_MEMORY_BALLOON=y > CONFIG_BALLOON_COMPACTION=y > CONFIG_PAGE_REPORTING=y > CONFIG_DRM_GEM_SHMEM_HELPER=y > CONFIG_DRM_VIRTIO_GPU=y > > > make ARCH=arm64 menuconfig, exit, save: yes. > make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j $(nproc) > make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- > INSTALL_MOD_PATH=/home/user/Desktop/ INSTALL_MOD_STRIP=1 modules_install > > *"Installing" the kernel:* > Place the the arch/arm64/boot/Image.gz file at the same place as the > rootfs.img, userdata.img and cache.img files > > Place the content of /home/user/Desktop/lib/modules/5.10.37+/ into > /vendor/lib/modules/ > > *Unmount rootfs.img* > umount /its/mount/point > losetup -d /dev/loop0 > > *Ready to go:* > > From an x86_64 computer: > qemu-system-aarch64 -M virt -cpu cortex-a72 -accel tcg,thread=multi > -smp 4 -m 2048 -kernel Image.gz -monitor none -parallel none -append > "root=/dev/vda rootfstype=ext4 ro init=/init selinux=1 checkreqprot=1 > androidboot.selinux=permissive console=ttyAMA0 androidboot.hardware=ranchu > loglevel=8" -serial mon:stdio -vga std -device ramfb -device nec-usb-xhci > -device usb-kbd -device usb-mouse -bios > /usr/share/qemu-efi-aarch64/QEMU_EFI.fd -drive format=raw,file=rootfs.img > -drive format=raw,file=userdata.img -drive format=raw,file=cache.img > > If you are lucky enough to have an arm64 machine with KVM available on it, > and enough RAM, into the above command line, remplace: > qemu-system-aarch64 -M virt -cpu cortex-a72 -accel tcg,thread=multi > -smp 4 -m 2048 > by > kvm -M virt -cpu host -accel kvm -smp 4 -m 2048 > > In summary, from an arm64 computer : > kvm -M virt -cpu host -accel kvm -smp 4 -m 2048 -kernel Image.gz > -monitor none -parallel none -append "root=/dev/vda rootfstype=ext4 ro > init=/init selinux=1 checkreqprot=1 androidboot.selinux=permissive > console=ttyAMA0 androidboot.hardware=ranchu loglevel=8" -serial mon:stdio > -vga std -device ramfb -device nec-usb-xhci -device usb-kbd -device > usb-mouse -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd -drive > format=raw,file=rootfs.img -drive format=raw,file=userdata.img -drive > format=raw,file=cache.img > > *Bits of additionnal information:* > Having ARM64 AOSP running on QEMU from x86_64, even with "-accel > tcg,thread=multi" which makes things much better, you still need to wait 30 > seconds before having "android" boot logo (on an AMD 3700X) and 350 seconds > (almost 6 minutes) on first boot before having the android main screen with > icons and all stuff (2 minutes at the following boots). > > There are some repeated messages about camera 2.4 stuff and [email protected]. You > may remove the /vendor/etc/init/[email protected], but > messages about "[email protected]" will still appear, so > you may remove the associated entries (camera and radio) from > /vendor/manifext.xml. Then no more messages about it. > > In case of need to debug, you may type "logcat" in the console: this give > way too much informations but you may spot information on what you are > looking for, or few lines above some service crashing (sometimes some > messages contains usefull information to tell you why it is going to crash > just after). Into the "userdata.img" partition you may also find > "/tombstones" folder, containing crash logs of services. > > In order to cleanly poweroff your emulated system you can type "reboot -p" > so that the "userdata.img" is kept as clean as possible. > -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/ac699b48-f177-4be7-bcd5-5a310e6e334fn%40googlegroups.com.
