Hi, I might be wrong, but maybe the graphics pipeline from the guest to the host GPU with QEMU could look something like this:
1) virgl-gallium, https://gitlab.freedesktop.org/mesa/mesa. Android.mk: BOARD_GPU_DRIVERS=virgl 2) virtio-gpu (guest) 3) virtio-gpu (host) 4) virglrenderer (upstream version; not AVDVirglrenderer) 5) host drivers (via libhybris if not a bionic qemu) 6) QEMU display: -spice ...,gl=on Aka. pretty much the standard Linux way of doing things. Since QEMU is such a great development tool for almost all the imaginable targets out there, I'm really surprised Google does not enable this by default. Anyone tried above or is there an easier alternative? I think I'll give it a spin. That said, looks like I got the 'qemu.gltransport' wrong in the prior post. 'virtio-gpu' was the right answer. -- Janne On Wednesday, 23 June 2021 at 08:06:32 UTC+3 Janne Karhunen wrote: > Hi, > > I tried something like this out of interest on arm64 with my own > hypervisor (custom kvm, https://github.com/jkrh/kvms) and indeed I get > the android vm up. Thanks! > > In order to get the AOSP up I first started to port the 'ranchu' machine > to qemu6, but the 'android-emu' library I'd need to hack in the qemu6 main > loop just does not feel 'clean', so this remains unfinished in that regard: > > https://github.com/jkrh/kvms/blob/master/patches/0001-target-ranchu-add-support-for-android-ranchu-board.patch > > That being said, would be nice to make it do GPU too and I'm new to the > graphics pipeline. From the hypervisor point of view I'd need it to do > full VIRTIO since my hypervisor only opens the virtio channels between the > host and the guest. Besides, I'm not entirely sure what to think of the > 'pipe'. > > First attempt with that was to see if it would do 'gltransport=virtio': > KERNEL_OPTS="root=/dev/vda rootfstype=ext4 ro init=/init selinux=1 > checkreqprot=1 androidboot.selinux=permissive console=ttyAMA0 > androidboot.hardware=ranchu qemu.uirenderer=skiagl qemu.gltransport=virtio > qemu.gles=1 qemu.opengles.version=131072 qemu.dalvik.vm.heapsize=192m > loglevel=8" > > and then for the QEMU: > SCREEN="-nographic -device virtio-gpu-pci -spice > $SPICESOCK,disable-ticketing=on $VDAGENT" > > Thoughts if something like this is supposed to be supported? Certainly it > does not work: > > [ 36.445042] DEBUG: Build fingerprint: > 'Android/aosp_arm64/generic_arm64:10/QT/eng.jmk.20210614.160415:userdebug/test-keys' > [ 36.450901] DEBUG: Revision: '0' > [ 36.458537] DEBUG: ABI: 'arm64' > [ 36.462533] DEBUG: Timestamp: 2021-06-01 11:28:17+0000 > [ 36.465839] DEBUG: pid: 219, tid: 219, name: surfaceflinger >>> > /system/bin/surfaceflinger <<< > [ 36.468971] DEBUG: uid: 1000 > [ 36.474921] DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault > addr 0x10 > [ 36.476737] DEBUG: Cause: null pointer dereference > [ 36.481510] DEBUG: x0 0000000000000000 x1 0000e23fdddf66c0 x2 > 0000e23fd08b59f4 x3 0000e23fddcef5f8 > [ 36.484085] DEBUG: x4 0000000000000002 x5 0000000000000000 x6 > 0000000000000000 x7 000000000000a000 > [ 36.490734] DEBUG: x8 0000e23fddeef0e0 x9 0000000000000001 x10 > 0000e23fddeef060 x11 0000000000000000 > [ 36.497538] DEBUG: x12 0000000000000000 x13 0000000000000000 x14 > 0000000000000020 x15 0200000000000000 > [ 36.502861] DEBUG: x16 0000e23fd081da70 x17 0000e23fd08b4168 x18 > 0000e23fdf142000 x19 0000000000000000 > [ 36.509025] DEBUG: x20 0000e23fddd192d0 x21 0000000000000001 x22 > 0000000000000000 x23 0000000000000000 > [ 36.514828] DEBUG: x24 0000e23fddeef020 x25 0000e23fddd19280 x26 > 0000000000000000 x27 0000e23fddd90000 > [ 36.520535] DEBUG: x28 0000e23fdc3b7070 x29 0000ffffdf3749e0 > [ 36.528408] DEBUG: sp 0000ffffdf3749d0 lr 0000e23fd0813464 pc > 0000e23fd08b4178 > [ 36.610037] DEBUG: > [ 36.615637] DEBUG: backtrace: > [ 36.616452] DEBUG: #00 pc 0000000000005178 > /vendor/lib64/libOpenglSystemCommon.so (HostConnection::gl2Encoder()+16) > (BuildId: 8ca38ff3265c54f31e1f70e8a19c683f) > [ 36.618386] DEBUG: #01 pc 0000000000013460 > /vendor/lib64/egl/libGLESv2_emulation.so (glGetString+24) (BuildId: > 1bc4a27ae3f25fffe3541af804629511) > [ 36.628225] DEBUG: #02 pc 0000000000018d20 > /system/lib64/libEGL.so (android::egl_context_t::onMakeCurrent(void*, > void*)+112) (BuildId: 5abb3faa9d72952310ca01a9da8941e3) > [ 36.636811] DEBUG: #03 pc 0000000000016450 > /system/lib64/libEGL.so > (android::egl_display_t::makeCurrent(android::egl_context_t*, > android::egl_context_t*, void*, void*, void*, void*, void*, void*)+268) > (BuildId: 5abb3faa9d72952310ca01a9da8941e3) > [ 36.646320] DEBUG: #04 pc 000000000001ff94 > /system/lib64/libEGL.so (android::eglMakeCurrentImpl(void*, void*, void*, > void*)+384) (BuildId: 5abb3faa9d72952310ca01a9da8941e3) > [ 36.660598] DEBUG: #05 pc 0000000000113d58 > /system/lib64/libsurfaceflinger.so > (android::renderengine::gl::GLESRenderEngine::create(int, unsigned int, > unsigned int)+380) (BuildId: becfe6218fde450840f4bc3f14cb68c5) > [ 36.670297] DEBUG: #06 pc 0000000000113aa4 > /system/lib64/libsurfaceflinger.so > (android::renderengine::RenderEngine::create(int, unsigned int, unsigned > int)+164) (BuildId: becfe6218fde450840f4bc3f14cb68c5) > [ 36.681625] DEBUG: #07 pc 00000000000ce97c > /system/lib64/libsurfaceflinger.so (android::SurfaceFlinger::init()+1164) > (BuildId: becfe6218fde450840f4bc3f14cb68c5) > [ 36.694419] DEBUG: #08 pc 00000000000031bc > /system/bin/surfaceflinger (main+364) (BuildId: > ff9215aa2c7b96de71b0a256004edc79) > [ 36.703441] DEBUG: #09 pc 000000000007d798 > /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108) (BuildId: > 676a709a0ee633ec9cf6ab05ec6410ae) > > > -- > Janne > > On Monday, 24 May 2021 at 21:14:44 UTC+3 [email protected] wrote: > >> Hi again, >> >> Here is the AOSP 10 on ARM64 QEMU version of the procedure I follow. >> Andoid 5.10 arm64 kernel build included, like for x86_64. >> Appart from using "vdb" and "vdc" into fstab instead of sdb and sdc, and >> removing some "rild.rc" file, the approach is the same. With some >> additionnal information at the end >> >> repo init -u https://android.googlesource.com/platform/manifest -b >> android-10.0.0_r47 >> repo sync >> source build/envsetup.sh >> lunch >> m -j $(nproc) >> >> # out/target/product/generic_arm64/system.img >> # out/target/product/generic_arm64/vendor.img >> >> Create and ext4 format the partition image files: >> >> qemu-img create rootfs.img 5G >> qemu-img create userdata.img 1G >> qemu-img create cache.img 500M >> >> losetup /dev/loop0 rootfs.img >> losetup /dev/loop1 userdata.img >> losetup /dev/loop2 cache.img >> >> mkfs -t ext4 /dev/loop0 >> mkfs -t ext4 /dev/loop1 >> mkfs -t ext4 /dev/loop2 >> >> losetup -d /dev/loop1 >> losetup -d /dev/loop2 >> >> *Mouting the rootfs partition * >> mount -t ext4 /dev/loop0 /where/you/want >> >> cp -a [system.img mount point] [rootfs/mount/point] >> The content of system.img may be copied inside a folder which is named >> like the mount point. Move everything 1 folder above. >> >> >> cp -a [vendor.img mount point] [rootfs/mount/point] >> If the name of the "vendor.img" mount point is "vendor", everything >> should now be inside the "/vendor" folder of the rootfs. If not, you have >> to move it to /vendor folder >> >> >> Delete these files: >> >> /vendor/lib/hw/camera.ranchu.so >> /vendor/lib/hw64/camera.ranchu.so >> /vendor/lib/hw/camera.ranchu.jpeg.so >> /vendor/lib/hw64/camera.ranchu.jpeg.so >> /vendor/lib/hw/vendor/lib/hw/gralloc.ranchu >> /vendor/lib/hw64/gralloc.ranchu >> /vendor/lib/hw/hwcomposer.ranchu >> /vendor/lib/hw64/hwcomposer.ranchu >> /vendor/lib/hw/vulkan.ranchu.so >> /vendor/lib/hw64/vulkan.ranchu.so >> /vendor/etc/init/rild.rc >> >> >> Change /vendor/etc/fstab.ranchu, those 2 lines should be enough: >> >> /dev/block/vdb /data ext4 >> noatime,nosuid,nodev,nomblk_io_submit,errors=panic >> wait,check,quota,reservedsize=128M,first_stage_mount >> /dev/block/vdc /cache ext4 >> noatime,nosuid,nodev,nomblk_io_submit,errors=panic >> wait,check,quota,reservedsize=128M,first_stage_mount >> >> >> Change /vendor/etc/init/hw/init.ranchu.rc by changing this line: >> setprop ro.hardware.egl emulation >> to >> setprop ro.hardware.egl swiftshader >> >> >> *Manufacturing an up to date 5.10 ARM64 Android kernel:* >> >> >> git clone https://android.googlesource.com/kernel/configs >> >> /!\ From configs/android-5.10/android-recommended.config, if you want to >> use a mouse, remove the following line : # CONFIG_INPUT_MOUSE is not set >> >> git clone https://android.googlesource.com/kernel/common -b >> android12-5.10 >> >> Start from copying "arch/arm64/configs/defconfig" file to the root of the >> kernel tree, name it ".config", and append to it: >> >> CONFIG_NETFILTER_ADVANCED=y >> Content of configs/android-5.10/android-base.config >> Content of configs/android-5.10/android-recommended.config >> Content of configs/android-5.10/android-recommended-arm64.config >> >> make ARCH=arm64 menuconfig, exit, save: yes. >> >> >> By looking with some tools, some of the things included into >> x86_64_defconfig, android-base.config, android-recommended.config are not >> included into the resulting ".config". >> It works anyway, but it would be nice to know why theese option aren't >> enabling. >> >> Still missing from arm64/configs/defconfig >> CONFIG_SYSVIPC=y >> CONFIG_ACPI_APEI_PCIEAER=y >> CONFIG_KSM=y >> CONFIG_IP6_NF_NAT=m >> CONFIG_IP6_NF_TARGET_MASQUERADE=m >> CONFIG_LEGACY_PTY_COUNT=16 >> CONFIG_POWER_AVS=y >> CONFIG_BACKLIGHT_GENERIC=m >> CONFIG_QCOM_IOMMU=y >> CONFIG_NFS_FS=y >> CONFIG_NFS_V4=y >> CONFIG_NFS_V4_1=y >> CONFIG_NFS_V4_2=y >> >> CONFIG_ROOT_NFS=y >> >> Still missing from android-base.config: >> CONFIG_TRACE_GPU_MEM=y >> >> Still missing from android-recommended.config: >> CONFIG_BACKLIGHT_LCD_SUPPORT=y >> CONFIG_REFCOUNT_FULL=y >> CONFIG_SDCARD_FS=y >> >> Now, some options that are still disabled, but should be enabled >> according to android-base-conditional.xml: >> CONFIG_ARMV8_DEPRECATED=y >> CONFIG_CP15_BARRIER_EMULATION=y >> CONFIG_SETEND_EMULATION=y >> CONFIG_SHADOW_CALL_STACK=y >> CONFIG_SWP_EMULATION=y >> CONFIG_BPF_JIT_ALWAYS_ON=y >> CONFIG_KFENCE=y >> CONFIG_USERFAULTFD=y >> >> >> >> CONFIG_AIO and CONFIG_ANDROID_BINDERFS options (from android-base.config) >> are causing early reboot without explanations. Would be nice to know why, >> to keep them enabled and have something running... Until so, disable them >> by adding at the end of .config: >> # CONFIG_AIO is not set >> # CONFIG_ANDROID_BINDERFS is not set >> >> Adding VirtIO things: >> >> CONFIG_VIRTIO=y >> CONFIG_VIRTIO_PCI=y >> CONFIG_VIRTIO_PCI_LEGACY=y >> CONFIG_VIRTIO_BALLOON=y >> CONFIG_VIRTIO_INPUT=y >> CONFIG_VIRTIO_MMIO=y >> CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y >> CONFIG_VIRTIO_DMA_SHARED_BUFFER=y >> CONFIG_BLK_MQ_VIRTIO=y >> CONFIG_MEMORY_BALLOON=y >> CONFIG_BALLOON_COMPACTION=y >> CONFIG_PAGE_REPORTING=y >> CONFIG_DRM_GEM_SHMEM_HELPER=y >> CONFIG_DRM_VIRTIO_GPU=y >> >> >> make ARCH=arm64 menuconfig, exit, save: yes. >> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j $(nproc) >> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- >> INSTALL_MOD_PATH=/home/user/Desktop/ INSTALL_MOD_STRIP=1 modules_install >> >> *"Installing" the kernel:* >> Place the the arch/arm64/boot/Image.gz file at the same place as the >> rootfs.img, userdata.img and cache.img files >> >> Place the content of /home/user/Desktop/lib/modules/5.10.37+/ into >> /vendor/lib/modules/ >> >> *Unmount rootfs.img* >> umount /its/mount/point >> losetup -d /dev/loop0 >> >> *Ready to go:* >> >> From an x86_64 computer: >> qemu-system-aarch64 -M virt -cpu cortex-a72 -accel tcg,thread=multi >> -smp 4 -m 2048 -kernel Image.gz -monitor none -parallel none -append >> "root=/dev/vda rootfstype=ext4 ro init=/init selinux=1 checkreqprot=1 >> androidboot.selinux=permissive console=ttyAMA0 androidboot.hardware=ranchu >> loglevel=8" -serial mon:stdio -vga std -device ramfb -device nec-usb-xhci >> -device usb-kbd -device usb-mouse -bios >> /usr/share/qemu-efi-aarch64/QEMU_EFI.fd -drive format=raw,file=rootfs.img >> -drive format=raw,file=userdata.img -drive format=raw,file=cache.img >> >> If you are lucky enough to have an arm64 machine with KVM available on >> it, and enough RAM, into the above command line, remplace: >> qemu-system-aarch64 -M virt -cpu cortex-a72 -accel tcg,thread=multi >> -smp 4 -m 2048 >> by >> kvm -M virt -cpu host -accel kvm -smp 4 -m 2048 >> >> In summary, from an arm64 computer : >> kvm -M virt -cpu host -accel kvm -smp 4 -m 2048 -kernel Image.gz >> -monitor none -parallel none -append "root=/dev/vda rootfstype=ext4 ro >> init=/init selinux=1 checkreqprot=1 androidboot.selinux=permissive >> console=ttyAMA0 androidboot.hardware=ranchu loglevel=8" -serial mon:stdio >> -vga std -device ramfb -device nec-usb-xhci -device usb-kbd -device >> usb-mouse -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd -drive >> format=raw,file=rootfs.img -drive format=raw,file=userdata.img -drive >> format=raw,file=cache.img >> >> *Bits of additionnal information:* >> Having ARM64 AOSP running on QEMU from x86_64, even with "-accel >> tcg,thread=multi" which makes things much better, you still need to wait 30 >> seconds before having "android" boot logo (on an AMD 3700X) and 350 seconds >> (almost 6 minutes) on first boot before having the android main screen with >> icons and all stuff (2 minutes at the following boots). >> >> There are some repeated messages about camera 2.4 stuff and [email protected]. >> You may remove the /vendor/etc/init/[email protected], but >> messages about "[email protected]" will still appear, so >> you may remove the associated entries (camera and radio) from >> /vendor/manifext.xml. Then no more messages about it. >> >> In case of need to debug, you may type "logcat" in the console: this give >> way too much informations but you may spot information on what you are >> looking for, or few lines above some service crashing (sometimes some >> messages contains usefull information to tell you why it is going to crash >> just after). Into the "userdata.img" partition you may also find >> "/tombstones" folder, containing crash logs of services. >> >> In order to cleanly poweroff your emulated system you can type "reboot >> -p" so that the "userdata.img" is kept as clean as possible. >> > -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/082a94c2-2490-45b9-bf4e-3d5a3e9c2967n%40googlegroups.com.
