Public bug reported: Description ===========
In OpenStack release Xena with VMware vSphere I experience an issue while creating Nova instance with config drive. Steps to reproduce ================== 1. build Kolla images with Neutron plugins according https://docs.openstack.org/kolla/xena/admin/image-building.html#neutron- plugins $ kolla-build -b ubuntu --tag xena --config-file vmware-nsx.ini --push 2. deploy OpenStack controller using Kolla-ansible all-in-one inventory with globals.yml options customized (see below) 3. create Cirros image (qemu-img converted to VMDK), create flavor, create provider network type VLAN 4. create Nova instance openstack server create \ --image cirros \ --flavor m1.tiny \ --network 'network1' \ --use-config-drive \ vm-$(date +%H%M%S) Expected result =============== Nova instance / VMware virtual machine with CD/DVD drive connected to file configdrive.iso in VM datastore path. Actual result ============= Nova instance in error state, VM deleted from VMware vSphere. Logs & Configs ============== I can provide more details if needed. ==> nova-compute.log <== 2023-05-02 19:54:41.461 8 DEBUG oslo_concurrency.processutils [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] Running cmd (subprocess): genisoimage -o /tmp/tmpxffhad19/configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 24.2.1 -quiet -J -r -V config-2 /tmp/tmpvwpb0rsh execute /var/lib/kolla/venv/lib/python3.8/site-packages/oslo_concurrency/processutils.py:384 2023-05-02 19:54:41.517 8 DEBUG oslo_concurrency.processutils [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] CMD "genisoimage -o /tmp/tmpxffhad19/configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 24.2.1 -quiet -J -r -V config-2 /tmp/tmpvwpb0rsh" returned: 0 in 0.056s execute /var/lib/kolla/venv/lib/python3.8/site-packages/oslo_concurrency/processutils.py:422 2023-05-02 19:54:41.523 8 DEBUG nova.virt.vmwareapi.images [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Uploading iso /tmp/tmpxffhad19/configdrive.iso to datastore upload_iso_to_datastore /var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/images.py:216 2023-05-02 19:54:41.526 8 DEBUG oslo_vmware.rw_handles [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] Creating HTTP connection to write to file with size = 493568 and URL = https://100.78.168.240:443/folder/10c093ea-5cb8-4cf8-8a70-156aed87621c/configdrive.iso?dcPath=%2FAT1&dsName=DS_LOC_VMC01_T2-01. _create_write_connection /var/lib/kolla/venv/lib/python3.8/site-packages/oslo_vmware/rw_handles.py:120 2023-05-02 19:54:41.565 8 DEBUG nova.virt.vmwareapi.images [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] Uploading iso of size : 493568 upload_iso_to_datastore /var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/images.py:228 2023-05-02 19:54:41.569 8 ERROR nova.virt.vmwareapi.vmops [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Creating config drive failed with error: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Instance failed to spawn: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Traceback (most recent call last): 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 2652, in _build_resources 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] yield resources 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 2412, in _build_and_run_instance 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self.driver.spawn(context, instance, image_meta, 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/driver.py", line 532, in spawn 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self._vmops.spawn(context, instance, image_meta, injected_files, 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 821, in spawn 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self._configure_config_drive( 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 217, in _configure_config_drive 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] uploaded_iso_path = self._create_config_drive(context, 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 881, in _create_config_drive 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] LOG.error('Creating config drive failed with error: %s', 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self.force_reraise() 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] raise self.value 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 870, in _create_config_drive 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] images.upload_iso_to_datastore( 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/images.py", line 231, in upload_iso_to_datastore 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] data = iso_file.read(block_size) 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/usr/lib/python3.8/codecs.py", line 322, in decode 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] (result, consumed) = self._buffer_decode(data, self.errors, final) 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] 2023-05-02 19:54:41.602 8 INFO nova.compute.manager [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Terminating instance Environment =========== Kolla: pip install git+https://opendev.org/openstack/kolla@stable/xena Kolla-ansible: pip install git+https://opendev.org/openstack/kolla-ansible@stable/xena OpenStack release Xena: nova --version 17.6.0 openstack --version openstack 5.6.2 VMware vSphere 6.7 --- begin of /etc/kolla/globals.yml --- kolla_base_distro: "ubuntu" nova_compute_virt_type: "vmware" vmware_dvs_host_ip: <IP> vmware_dvs_host_port: "443" vmware_dvs_host_username: "<userName>" vmware_dvs_dvs_name: "<dvsName>" vmware_dvs_dhcp_override_mac: "" enable_haproxy: "no" enable_cinder: "yes" glance_backend_file: "yes" glance_backend_vmware: "no" cinder_backend_vmwarevc_vmdk: "yes" nova_compute_virt_type: "vmware" vmware_vcenter_host_ip: <IP> vmware_vcenter_host_username: <username> vmware_vcenter_host_password: <password> vmware_datastore_name: <dsName> vmware_vcenter_name: <hostname> vmware_vcenter_cluster_name: <clusterName> --- end of /etc/kolla/globals.yml --- ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2018973 Title: Create Nova instance fails while uploading config drive ISO to VMware datastore Status in OpenStack Compute (nova): New Bug description: Description =========== In OpenStack release Xena with VMware vSphere I experience an issue while creating Nova instance with config drive. Steps to reproduce ================== 1. build Kolla images with Neutron plugins according https://docs.openstack.org/kolla/xena/admin/image- building.html#neutron-plugins $ kolla-build -b ubuntu --tag xena --config-file vmware-nsx.ini --push 2. deploy OpenStack controller using Kolla-ansible all-in-one inventory with globals.yml options customized (see below) 3. create Cirros image (qemu-img converted to VMDK), create flavor, create provider network type VLAN 4. create Nova instance openstack server create \ --image cirros \ --flavor m1.tiny \ --network 'network1' \ --use-config-drive \ vm-$(date +%H%M%S) Expected result =============== Nova instance / VMware virtual machine with CD/DVD drive connected to file configdrive.iso in VM datastore path. Actual result ============= Nova instance in error state, VM deleted from VMware vSphere. Logs & Configs ============== I can provide more details if needed. ==> nova-compute.log <== 2023-05-02 19:54:41.461 8 DEBUG oslo_concurrency.processutils [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] Running cmd (subprocess): genisoimage -o /tmp/tmpxffhad19/configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 24.2.1 -quiet -J -r -V config-2 /tmp/tmpvwpb0rsh execute /var/lib/kolla/venv/lib/python3.8/site-packages/oslo_concurrency/processutils.py:384 2023-05-02 19:54:41.517 8 DEBUG oslo_concurrency.processutils [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] CMD "genisoimage -o /tmp/tmpxffhad19/configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 24.2.1 -quiet -J -r -V config-2 /tmp/tmpvwpb0rsh" returned: 0 in 0.056s execute /var/lib/kolla/venv/lib/python3.8/site-packages/oslo_concurrency/processutils.py:422 2023-05-02 19:54:41.523 8 DEBUG nova.virt.vmwareapi.images [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Uploading iso /tmp/tmpxffhad19/configdrive.iso to datastore upload_iso_to_datastore /var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/images.py:216 2023-05-02 19:54:41.526 8 DEBUG oslo_vmware.rw_handles [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] Creating HTTP connection to write to file with size = 493568 and URL = https://100.78.168.240:443/folder/10c093ea-5cb8-4cf8-8a70-156aed87621c/configdrive.iso?dcPath=%2FAT1&dsName=DS_LOC_VMC01_T2-01. _create_write_connection /var/lib/kolla/venv/lib/python3.8/site-packages/oslo_vmware/rw_handles.py:120 2023-05-02 19:54:41.565 8 DEBUG nova.virt.vmwareapi.images [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] Uploading iso of size : 493568 upload_iso_to_datastore /var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/images.py:228 2023-05-02 19:54:41.569 8 ERROR nova.virt.vmwareapi.vmops [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Creating config drive failed with error: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Instance failed to spawn: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Traceback (most recent call last): 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 2652, in _build_resources 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] yield resources 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/compute/manager.py", line 2412, in _build_and_run_instance 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self.driver.spawn(context, instance, image_meta, 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/driver.py", line 532, in spawn 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self._vmops.spawn(context, instance, image_meta, injected_files, 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 821, in spawn 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self._configure_config_drive( 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 217, in _configure_config_drive 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] uploaded_iso_path = self._create_config_drive(context, 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 881, in _create_config_drive 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] LOG.error('Creating config drive failed with error: %s', 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] self.force_reraise() 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] raise self.value 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/vmops.py", line 870, in _create_config_drive 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] images.upload_iso_to_datastore( 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/var/lib/kolla/venv/lib/python3.8/site-packages/nova/virt/vmwareapi/images.py", line 231, in upload_iso_to_datastore 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] data = iso_file.read(block_size) 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] File "/usr/lib/python3.8/codecs.py", line 322, in decode 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] (result, consumed) = self._buffer_decode(data, self.errors, final) 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 32848: invalid continuation byte 2023-05-02 19:54:41.576 8 ERROR nova.compute.manager [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] 2023-05-02 19:54:41.602 8 INFO nova.compute.manager [req-c537ecba-e141-452a-b550-145fc33ad95e 7e9f152bbd104af6923cfdfadf957991 f07e4a8c218e4183a7b8e8d8e5153a13 - default default] [instance: 10c093ea-5cb8-4cf8-8a70-156aed87621c] Terminating instance Environment =========== Kolla: pip install git+https://opendev.org/openstack/kolla@stable/xena Kolla-ansible: pip install git+https://opendev.org/openstack/kolla-ansible@stable/xena OpenStack release Xena: nova --version 17.6.0 openstack --version openstack 5.6.2 VMware vSphere 6.7 --- begin of /etc/kolla/globals.yml --- kolla_base_distro: "ubuntu" nova_compute_virt_type: "vmware" vmware_dvs_host_ip: <IP> vmware_dvs_host_port: "443" vmware_dvs_host_username: "<userName>" vmware_dvs_dvs_name: "<dvsName>" vmware_dvs_dhcp_override_mac: "" enable_haproxy: "no" enable_cinder: "yes" glance_backend_file: "yes" glance_backend_vmware: "no" cinder_backend_vmwarevc_vmdk: "yes" nova_compute_virt_type: "vmware" vmware_vcenter_host_ip: <IP> vmware_vcenter_host_username: <username> vmware_vcenter_host_password: <password> vmware_datastore_name: <dsName> vmware_vcenter_name: <hostname> vmware_vcenter_cluster_name: <clusterName> --- end of /etc/kolla/globals.yml --- To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2018973/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
