Andrew Bogott has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/329021 )
Change subject: toollabs: Don't use wikitech API to find labs instances in
tools-clush-generator
......................................................................
toollabs: Don't use wikitech API to find labs instances in tools-clush-generator
Use the OpenStack API instead now that we hav the new observer credentials.
Some parts tested.
Bug: T104575
Change-Id: I6c26aac3c19e3ff7dec0e831926a535fef7480cb
---
M modules/role/files/toollabs/clush/tools-clush-generator
M modules/role/manifests/toollabs/clush/master.pp
2 files changed, 29 insertions(+), 14 deletions(-)
Approvals:
Andrew Bogott: Looks good to me, approved
jenkins-bot: Verified
Objections:
Alex Monk: There's a problem with this change, please improve
diff --git a/modules/role/files/toollabs/clush/tools-clush-generator
b/modules/role/files/toollabs/clush/tools-clush-generator
index 4fbf845..833f218 100644
--- a/modules/role/files/toollabs/clush/tools-clush-generator
+++ b/modules/role/files/toollabs/clush/tools-clush-generator
@@ -9,10 +9,13 @@
This is run in a cron every hour.
"""
-import json
import yaml
import argparse
-from urllib.request import urlopen
+
+from keystoneclient.session import Session as KeystoneSession
+from keystoneclient.auth.identity.v3 import Password as KeystonePassword
+
+from novaclient import client as novaclient
# Maps prefixes to hostgroup names
@@ -46,17 +49,21 @@
}
-def get_hostgroups(classifier):
+def get_hostgroups(classifier, observer_pass):
hostgroups = {name: [] for name in classifier.values()}
-
- api_url = 'https://wikitech.wikimedia.org/w/api.php' \
- '?action=query&list=novainstances&niregion=eqiad&format=json' \
- '&niproject=tools'
-
- data = json.loads(urlopen(api_url).read().decode('utf-8'))
-
- for instance in data['query']['novainstances']:
- name = instance['name']
+ client = novaclient.Client(
+ "2.0",
+ session=KeystoneSession(auth=KeystonePassword(
+ auth_url="http://labcontrol1001.wikimedia.org:5000/v3";,
+ username="novaobserver",
+ password=observer_pass,
+ project_name='tools',
+ user_domain_name='default',
+ project_domain_name='default'
+ ))
+ )
+ for instance in client.servers.list():
+ name = instance.name
for prefix in classifier:
if name.startswith('tools-' + prefix):
role = classifier[prefix]
@@ -71,8 +78,13 @@
'outputpath',
help='Path to output hostgroup to host mappings'
)
+ parser.add_argument(
+ '--observer-pass',
+ required=True,
+ help='Password for the OpenStack observer account'
+ )
args = parser.parse_args()
- hostgroups = get_hostgroups(TOOLS_PREFIX_CLASSIFIER)
+ hostgroups = get_hostgroups(TOOLS_PREFIX_CLASSIFIER, args.observer_pass)
with open(args.outputpath, 'w') as f:
f.write(yaml.safe_dump(hostgroups, default_flow_style=False))
diff --git a/modules/role/manifests/toollabs/clush/master.pp
b/modules/role/manifests/toollabs/clush/master.pp
index eb869a2..7a50a29 100644
--- a/modules/role/manifests/toollabs/clush/master.pp
+++ b/modules/role/manifests/toollabs/clush/master.pp
@@ -24,6 +24,7 @@
require_package('python3-yaml')
+ include ::openstack::clientlib
file { '/usr/local/sbin/tools-clush-generator':
ensure => file,
source =>
'puppet:///modules/role/toollabs/clush/tools-clush-generator',
@@ -51,9 +52,11 @@
mode => '0555',
}
+ $novaconfig = hiera_hash('novaconfig', {})
+ $observer_pass = $novaconfig['observer_password']
cron { 'update_tools_clush':
ensure => present,
- command => '/usr/local/sbin/tools-clush-generator
/etc/clustershell/tools.yaml',
+ command => "/usr/local/sbin/tools-clush-generator
/etc/clustershell/tools.yaml --observer-pass ${observer_pass}",
hour => '*/1',
user => 'root'
}
--
To view, visit https://gerrit.wikimedia.org/r/329021
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I6c26aac3c19e3ff7dec0e831926a535fef7480cb
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Alex Monk <[email protected]>
Gerrit-Reviewer: Alex Monk <[email protected]>
Gerrit-Reviewer: Andrew Bogott <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
