I think the swift driver uses 200M: DEFAULT_LARGE_OBJECT_CHUNK_SIZE = 200 # 200M
On Wed, Jul 7, 2021 at 1:50 PM Nobuto Murata <[email protected]> wrote: > And by using "4 * units.Mi" it can be 20s. > > -- > You received this bug notification because you are a member of Ubuntu > OpenStack, which is subscribed to python-glance-store in Ubuntu. > https://bugs.launchpad.net/bugs/1934849 > > Title: > s3 backend takes time exponentially > > Status in glance_store: > New > Status in python-glance-store package in Ubuntu: > New > > Bug description: > I have a test Ceph cluster as an object storage with both Swift and S3 > protocols enabled for Glance (Ussuri). When I use Swift backend with > Glance, an image upload completes quickly enough. But with S3 backend > Glance, it takes much more time to upload an image and it seems to > rise exponentially. > > It's worth noting that when uploading an image with S3 backend, a > single core is consumed 100% by glance-api process. > > for backend in swift s3; do > for i in {8,16,32,64,128,512}; do > dd if=/dev/zero of=my-image.img bs=1M count=$i oflag=sync > time glance image-create \ > --store $backend \ > --file my-image.img --name my-image \ > --disk-format raw --container-format bare \ > --progress > done > done > > [swift] > 8MB - 2.4s > 16MB - 2.8s > 32MB - 2.6s > 64MB - 2.7s > 128MB - 3.1s > ... > 512MB - 5.9s > > [s3] > 8MB - 2.2s > 16MB - 2.9s > 32MB - 5.5s > 64MB - 16.3s > 128MB - 54.9s > ... > 512MB - 14m26s > > Btw, downloading of 512MB image with S3 backend can complete with less > than 10 seconds. > > $ time openstack image save --file downloaded.img > 917c5424-4350-4bc5-98ca-66d40e101843 > real 0m5.673s > > $ du -h downloaded.img > 512M downloaded.img > > > [/etc/glance/glance-api.conf] > > enabled_backends = local:file, ceph:rbd, swift:swift, s3:s3 > > [swift] > auth_version = 3 > auth_address = http://192.168.151.131:5000/v3 > ... > container = glance > large_object_size = 5120 > large_object_chunk_size = 200 > > > [s3] > s3_store_host = http://192.168.151.137:80/ > ... > s3_store_bucket = zaza-glance-s3-test > s3_store_large_object_size = 5120 > s3_store_large_object_chunk_size = 200 > > ProblemType: Bug > DistroRelease: Ubuntu 20.04 > Package: python3-glance-store 2.0.0-0ubuntu1 > ProcVersionSignature: Ubuntu 5.4.0-77.86-generic 5.4.119 > Uname: Linux 5.4.0-77-generic x86_64 > NonfreeKernelModules: bluetooth ecdh_generic ecc tcp_diag inet_diag > binfmt_misc veth zfs zunicode zlua zavl icp zcommon znvpair spl unix_diag > nft_masq nft_chain_nat bridge stp llc vhost_vsock > vmw_vsock_virtio_transport_common vhost vsock ebtable_filter ebtables > ip6table_raw ip6table_mangle ip6table_nat ip6table_filter ip6_tables > iptable_raw iptable_mangle iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 > nf_defrag_ipv4 iptable_filter bpfilter nf_tables nfnetlink dm_multipath > scsi_dh_rdac scsi_dh_emc scsi_dh_alua kvm_amd ccp input_leds kvm joydev > mac_hid serio_raw qemu_fw_cfg sch_fq_codel ip_tables x_tables autofs4 btrfs > zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq > async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear > crct10dif_pclmul crc32_pclmul cirrus ghash_clmulni_intel drm_kms_helper > virtio_net syscopyarea aesni_intel sysfillrect sysimgblt fb_sys_fops > crypto_simd cryptd drm virtio_blk glue_helper net_failover psmouse failover > floppy i2c_piix4 pata_acpi > ApportVersion: 2.20.11-0ubuntu27.18 > Architecture: amd64 > CasperMD5CheckResult: skip > Date: Wed Jul 7 04:46:05 2021 > PackageArchitecture: all > ProcEnviron: > TERM=screen-256color > PATH=(custom, no user) > LANG=C.UTF-8 > SHELL=/bin/bash > SourcePackage: python-glance-store > UpgradeStatus: No upgrade log present (probably fresh install) > > To manage notifications about this bug go to: > https://bugs.launchpad.net/glance-store/+bug/1934849/+subscriptions > > -- > Ubuntu-openstack-bugs mailing list > [email protected] > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-openstack-bugs > -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1934849 Title: s3 backend takes time exponentially To manage notifications about this bug go to: https://bugs.launchpad.net/glance-store/+bug/1934849/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
