Re: [Cluster-devel] [GFS2][Patch 0/10] GFS2 performance tweaks
Hi, On Tue, 2007-12-11 at 18:43 -0600, Bob Peterson wrote: Hi GFS2 Folks, The following is a set of ten patches designed to give gfs2 somewhat better performance than before. Some are trivial and may or may not have a real impact on performance. I'll let Steve Whitehouse decide which ones should go upstream and which ones aren't worth it. The patches may be summarized as follows: I've applied most of these with the exceptions of: 1. Journal extent mapping 2. Get rid of useless found variable in quota.c 3. Run through full bitmaps quicker in gfs2_bitfit 4. Get rid of sd_statfs_mutex - Needs fixing as per previous email. 5. Shortcut in gfs2_write_alloc_required if writing past eof. 6. Reorganize function gfs2_glmutex_lock 7. Only fetch the dinode once in block_map 8. Only find indirect pointer buffers once in block_map - I'd really far rather see a fix to bz #307091. Also, although it might make writes faster, I rather suspect that it will have the opposite effect on reads, since the indirect blocks will not be in the journal at all. So on balance I think I'd rather not take this patch right now. 9. Move meta_inval to glops.c and declare static, more attach_bufdata to trans.c and declare static. - This is also done in bz #349271 and I only want to apply one or the other of these two patches so I'm deferring this until we know what we are doing with the patches for #349271. 10. Function meta_read optimization. Regards, Bob Peterson Red Hat GFS Also, please use git-send-email if you are sending multiple patches. I'm not sure what you did to git to get the rather strange paths in these patches, but I had to edit all the patches except the first one by hand in order to get them to apply, Steve.
[Cluster-devel] cluster/rgmanager ChangeLog include/ds.h
CVSROOT:/cvs/cluster Module name:cluster Changes by: [EMAIL PROTECTED] 2007-12-12 15:27:35 Modified files: rgmanager : ChangeLog Added files: rgmanager/include: ds.h Log message: Add missing ds.h Patches: http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/rgmanager/ChangeLog.diff?cvsroot=clusterr1=1.62r2=1.63 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/rgmanager/include/ds.h.diff?cvsroot=clusterr1=NONEr2=1.1 --- cluster/rgmanager/ChangeLog 2007/11/30 21:36:28 1.62 +++ cluster/rgmanager/ChangeLog 2007/12/12 15:27:34 1.63 @@ -1,3 +1,6 @@ +2007-12-12 Lon Hohberger lhh at redhat.com + * Misc changes; add missing ds.h + 2007-11-30 Lon Hohberger lhh at redhat.com * Commit RIND / S-Lang script engine [untested] /cvs/cluster/cluster/rgmanager/include/ds.h,v -- standard output revision 1.1 --- cluster/rgmanager/include/ds.h +++ - 2007-12-12 15:27:35.772745000 + @@ -0,0 +1,13 @@ +#ifndef _DS_H +#define _DS_H + +int ds_init(void); +int ds_key_init(char *keyid, int maxsize, int timeout); +int ds_key_finish(char *keyid); +int ds_write(char *keyid, void *buf, size_t maxlen); +int ds_read(char *keyid, void *buf, size_t maxlen); +int ds_finish(void); + +#define DS_MIN_SIZE 512 + +#endif
Re: [Cluster-devel] [GFS2] [Patch 4/10] Get rid of sd_statfs_mutex
On Wed, 2007-12-12 at 09:18 +, Steven Whitehouse wrote: You can't do gfs2_trans_add_bh under a spinlock, but there is no reason why you can't just reverse the order of these two statements to fix it, Steve. Hi Steve, If we reverse the two statements, the trans_add_bh is not protected at all, which I assume was the purpose of the mutex in the first place. I'm not sure this is buying us much anyway, so perhaps we should forget it. Regards, Bob Peterson
Re: [Cluster-devel] [GFS2] [Patch 3/10] Run through full bitmaps quicker in gfs2_bitfit
On Wed, 2007-12-12 at 11:07 +, Steven Whitehouse wrote:
Hi,
fs/gfs2/rgrp.c: In function ‘gfs2_bitfit’:
fs/gfs2/rgrp.c:139: warning: assignment from incompatible pointer type
Please can you send me a fix for the above. Thanks,
Steve.
Hi,
Sorry about that. Here is a replacement for patch3 for the compile
error.
Note that the previous patch3 I sent also had some leftover useless
debug code in it (regarding static int c) which needed to be taken out.
Regards,
Bob Peterson
--
Signed-off-by: Bob Peterson [EMAIL PROTECTED]
--
.../fs/gfs2/rgrp.c | 49 ++--
1 files changed, 25 insertions(+), 24 deletions(-)
diff --git a/fs/gfs2/rgrp.c b/fs/gfs2/rgrp.c
index e0ee195..7a28339 100644
--- a/fs/gfs2/rgrp.c
+++ b/fs/gfs2/rgrp.c
@@ -126,41 +126,43 @@ static unsigned char gfs2_testbit(struct gfs2_rgrpd *rgd,
unsigned char *buffer,
* Return: the block number (bitmap buffer scope) that was found
*/
-static u32 gfs2_bitfit(struct gfs2_rgrpd *rgd, unsigned char *buffer,
- unsigned int buflen, u32 goal,
- unsigned char old_state)
+static u32 gfs2_bitfit(unsigned char *buffer, unsigned int buflen, u32 goal,
+ unsigned char old_state)
{
- unsigned char *byte, *end, alloc;
+ unsigned char *byte;
u32 blk = goal;
- unsigned int bit;
+ unsigned int bit, bitlong;
+ unsigned long *plong, plong55;
byte = buffer + (goal / GFS2_NBBY);
+ plong = (unsigned long *)buffer + (goal / GFS2_NBBY);
bit = (goal % GFS2_NBBY) * GFS2_BIT_SIZE;
- end = buffer + buflen;
- alloc = (old_state == GFS2_BLKST_FREE) ? 0x55 : 0;
-
- while (byte end) {
- /* If we're looking for a free block we can eliminate all
- bitmap settings with 0x55, which represents four data
- blocks in a row. If we're looking for a data block, we can
- eliminate 0x00 which corresponds to four free blocks. */
- if ((*byte 0x55) == alloc) {
- blk += (8 - bit) 1;
-
- bit = 0;
- byte++;
-
+ bitlong = bit;
+#if BITS_PER_LONG == 32
+ plong55 = 0x;
+#else
+ plong55 = 0x;
+#endif
+ while (byte buffer + buflen) {
+
+ if (bitlong == 0 old_state == 0 *plong == plong55) {
+ plong++;
+ byte += sizeof(unsigned long);
+ blk += sizeof(unsigned long) * GFS2_NBBY;
continue;
}
-
if (((*byte bit) GFS2_BIT_MASK) == old_state)
return blk;
-
bit += GFS2_BIT_SIZE;
if (bit = 8) {
bit = 0;
byte++;
}
+ bitlong += GFS2_BIT_SIZE;
+ if (bitlong = sizeof(unsigned long) * 8) {
+ bitlong = 0;
+ plong++;
+ }
blk++;
}
@@ -1318,11 +1320,10 @@ static u32 rgblk_search(struct gfs2_rgrpd *rgd, u32
goal,
/* The GFS2_BLKST_UNLINKED state doesn't apply to the clone
bitmaps, so we must search the originals for that. */
if (old_state != GFS2_BLKST_UNLINKED bi-bi_clone)
- blk = gfs2_bitfit(rgd, bi-bi_clone + bi-bi_offset,
+ blk = gfs2_bitfit(bi-bi_clone + bi-bi_offset,
bi-bi_len, goal, old_state);
else
- blk = gfs2_bitfit(rgd,
- bi-bi_bh-b_data + bi-bi_offset,
+ blk = gfs2_bitfit(bi-bi_bh-b_data + bi-bi_offset,
bi-bi_len, goal, old_state);
if (blk != BFITNOENT)
break;
[Cluster-devel] conga/luci/site/luci/Extensions LuciValidation.py
CVSROOT:/cvs/cluster
Module name:conga
Changes by: [EMAIL PROTECTED] 2007-12-12 15:43:15
Added files:
luci/site/luci/Extensions: LuciValidation.py
Log message:
Separate out cluster form post validation to allow async validation
Patches:
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/site/luci/Extensions/LuciValidation.py.diff?cvsroot=clusterr1=NONEr2=1.1
/cvs/cluster/conga/luci/site/luci/Extensions/LuciValidation.py,v -- standard
output
revision 1.1
--- conga/luci/site/luci/Extensions/LuciValidation.py
+++ - 2007-12-12 15:43:15.733027000 +
@@ -0,0 +1,1185 @@
+# Copyright (C) 2007 Red Hat, Inc.
+#
+# This program is free software; you can redistribute
+# it and/or modify it under the terms of version 2 of the
+# GNU General Public License as published by the
+# Free Software Foundation.
+
+from ClusterModel.RefObject import RefObject
+from ClusterModel.Service import Service
+from ClusterModel.Method import Method
+from ClusterModel.Fence import Fence
+from ClusterModel.FailoverDomain import FailoverDomain
+from ClusterModel.FailoverDomainNode import FailoverDomainNode
+from ClusterModel.QuorumD import QuorumD
+from ClusterModel.Heuristic import Heuristic
+from ClusterModel.FenceXVMd import FenceXVMd
+from ClusterModel.Lockserver import Lockserver
+from ClusterModel.Vm import Vm
+
+from conga_constants import LUCI_DEBUG_MODE, FDOM, FDOM_ADD, SERVICE_CONFIG,
SERVICE_ADD, VM_CONFIG, VM_ADD
+from FenceHandler import validateFenceDevice, validateNewFenceDevice,
validate_fenceinstance, FD_VAL_SUCCESS
+
+from LuciSyslog import get_logger
+from xml.dom import minidom
+from ResourceHandler import create_resource
+from LuciZope import GetReqVars
+
+luci_log = get_logger()
+
+def validate_node_fence_config(model, request):
+ errors = list()
+
+ fvar = GetReqVars(request,
+ [ 'fence_xml', 'fence_level', 'nodename', 'clustername'
])
+
+ if fvar['fence_xml'] is None:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC0: no fence_xml for node
%s' \
+ % fvar['nodename'])
+ return (False, {'errors': ['No fence data was supplied']})
+
+ if fvar['fence_level'] is None:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC1: no fence level for %s' \
+ % fvar['nodename'])
+ return (False, {'errors': ['No fence level was supplied']})
+
+ try:
+ fence_level = int(fvar['fence_level'])
+ except Exception, e:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC2: invalid fence level: %s:
%r %s' \
+ % (fvar['fence_level'], e, str(e)))
+ return (False, {'errors': ['%s is an invalid fence level' %
fvar['fence_level'] ]})
+
+ nodename = fvar['nodename']
+ if nodename is None:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC3: no nodename: %r %s' %
(e, str(e)))
+ return (False, {'errors': ['No node name was given']})
+
+ clustername = fvar['clustername']
+ if clustername is None:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC4: no clustername: %r %s' %
(e, str(e)))
+ return (False, {'errors': ['No cluster name was given']})
+
+ try:
+ doc = minidom.parseString(fvar['fence_xml'])
+ except Exception, e:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC7: error: %r %s' % (e,
str(e)))
+ return (False, {'errors': ['The fence data submitted is not
properly formed']})
+
+ try:
+ node = model.retrieveNodeByName(nodename)
+ except Exception, e:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('vNFC8: unable to find node name
%s in current node list: %r %s' % (nodename, e, str(e)))
+ return (False, {'errors': ['Unable to find the cluster node %s
in the node list' % nodename ]})
+
+ levels = node.getFenceLevels()
+ try:
+ method_id = levels[fence_level - 1].getAttribute('name')
+ if not method_id:
+ raise Exception, 'No method ID'
+ fence_method = Method()
+ fence_method.addAttribute('name', str(method_id))
+ levels[fence_level - 1] = fence_method
+ except Exception, e:
+ method_id = fence_level
+ fence_method = Method()
+ fence_method.addAttribute('name', str(method_id))
+
+ forms = doc.getElementsByTagName('form')
+ if len(forms) 1:
+ delete_target = None
+ for l in levels:
+ # delete the fence level
+
[Cluster-devel] conga/luci/site/luci/Extensions LuciZope.py He ...
CVSROOT:/cvs/cluster
Module name:conga
Changes by: [EMAIL PROTECTED] 2007-12-12 15:48:30
Modified files:
luci/site/luci/Extensions: LuciZope.py HelperFunctions.py
LuciZopeAsync.py LuciZopeExternal.py
Log message:
More shuffling stuff around to use the new form validation
Patches:
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/site/luci/Extensions/LuciZope.py.diff?cvsroot=clusterr1=1.5r2=1.6
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/site/luci/Extensions/HelperFunctions.py.diff?cvsroot=clusterr1=1.10r2=1.11
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/site/luci/Extensions/LuciZopeAsync.py.diff?cvsroot=clusterr1=1.3r2=1.4
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/site/luci/Extensions/LuciZopeExternal.py.diff?cvsroot=clusterr1=1.3r2=1.4
--- conga/luci/site/luci/Extensions/LuciZope.py 2007/08/20 16:31:13 1.5
+++ conga/luci/site/luci/Extensions/LuciZope.py 2007/12/12 15:48:29 1.6
@@ -145,3 +145,22 @@
pval = None
ret[i] = pval
return ret
+
+def getClusterURL(self, request, model):
+ try:
+ clustername = request['clustername'].strip()
+ if not clustername:
+ raise Exception, 'cluster name from request is blank'
+ except:
+ try:
+ clustername = model.getClusterName()
+ if not clustername:
+ raise Exception, 'cluster name from model is
blank'
+ except:
+ if LUCI_DEBUG_MODE is True:
+ luci_log.debug_verbose('GCURL0: unable to get
cluster name')
+ return ''
+
+ return '/luci/cluster/index_html?pagetype=7clustername=%s' %
clustername
+
+
--- conga/luci/site/luci/Extensions/HelperFunctions.py 2007/09/24 21:19:42
1.10
+++ conga/luci/site/luci/Extensions/HelperFunctions.py 2007/12/12 15:48:29
1.11
@@ -53,7 +53,8 @@
args.insert(0, rc)
r['batch_result'] =
self.query_func(*args)
except Exception, e:
- pass
+ r['error'] = True
+ r['err_msg'] = '%s' % str(e)
self.mutex.acquire()
self.riccis[host] = r
--- conga/luci/site/luci/Extensions/LuciZopeAsync.py2007/09/25 04:00:24
1.3
+++ conga/luci/site/luci/Extensions/LuciZopeAsync.py2007/12/12 15:48:30
1.4
@@ -178,5 +178,11 @@
% (request['nodenames'], e, str(e)))
write_err_async(request, 'Error retrieving information')
return None
- xml_obj = result_to_xml(True, { 'result': ret })
+ xml_obj = result_to_xml((True, { 'result': ret }))
+ write_xml_resp(request, xml_obj)
+
+def validate_clusvc_async(self, request):
+ from LuciValidation import validate_clusvc_add
+ ret = validate_clusvc_add(self, request)
+ xml_obj = result_to_xml((True, { 'result': ret }))
write_xml_resp(request, xml_obj)
--- conga/luci/site/luci/Extensions/LuciZopeExternal.py 2007/08/08 21:00:07
1.3
+++ conga/luci/site/luci/Extensions/LuciZopeExternal.py 2007/12/12 15:48:30
1.4
@@ -14,11 +14,10 @@
getDefaultUser
from cluster_adapters import clusterTaskProcess, \
- resourceAdd, resourceDelete, serviceDelete, \
- getClusterURL, getSystemLogs, getRicciAgentForCluster, \
+ resourceDelete, serviceDelete, \
isClusterBusy, nodeTaskProcess, process_cluster_conf_editor, \
serviceMigrate, serviceRestart, serviceStart, serviceStop, \
- getResourceInfo
+ getResourceInfo, getSystemLogs
from HelperFunctions import add_commas, bytes_to_value_units, convert_bytes
@@ -28,12 +27,15 @@
getNodeInfo, getnodes, getNodesInfo, getResourcesInfo, \
getServiceInfo, getServicesInfo, getVMInfo, getClusterOS
+from LuciClusterActions import getRicciAgentForCluster
+
from LuciDB import access_to_host_allowed, allowed_systems, \
check_clusters, getRicciAgent, getSystems, getClusters, \
getStorage
from LuciZope import appendModel, bytes_to_value_prefunits, \
- set_persistent_var, strFilter, getTabs, siteIsSetup
+ set_persistent_var, strFilter, getTabs, siteIsSetup, \
+ getClusterURL
from LuciZopeClusterPortal import createCluChooser, createCluConfigTree
@@ -52,4 +54,4 @@
from system_adapters import get_sys_svc_list, validate_manage_svc
-from LuciZopeAsync import get_cluster_nodes_async, get_sysinfo_async
+from LuciZopeAsync import get_cluster_nodes_async, get_sysinfo_async,
validate_clusvc_async
[Cluster-devel] conga/luci/plone-custom conga.js conga_ajax.js ...
CVSROOT:/cvs/cluster
Module name:conga
Changes by: [EMAIL PROTECTED] 2007-12-12 15:50:18
Modified files:
luci/plone-custom: conga.js conga_ajax.js luci.css
Log message:
UI support for new validation
Patches:
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/plone-custom/conga.js.diff?cvsroot=clusterr1=1.8r2=1.9
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/plone-custom/conga_ajax.js.diff?cvsroot=clusterr1=1.5r2=1.6
http://sourceware.org/cgi-bin/cvsweb.cgi/conga/luci/plone-custom/luci.css.diff?cvsroot=clusterr1=1.1r2=1.2
--- conga/luci/plone-custom/conga.js2007/11/27 16:58:18 1.8
+++ conga/luci/plone-custom/conga.js2007/12/12 15:50:18 1.9
@@ -189,3 +189,28 @@
}
elem.parentNode.removeChild(elem);
}
+
+function swap_tabs(new_label, cur_tab, new_tab) {
+ if (cur_tab == new_tab) {
+ return (cur_tab);
+ }
+ var new_tab_elem = document.getElementById(new_tab);
+ if (!new_tab_elem) {
+ return (cur_tab);
+ }
+ var cur_tab_elem = document.getElementById(cur_tab);
+ if (!cur_tab_elem) {
+ return (cur_tab);
+ }
+ try {
+ cur_tab_elem.className += ' invisible';
+
+ new_tab_elem.className =
new_tab_elem.className.replace(/invisible/gi, '');
+ new_label.className += 'configTab configTabActive';
+ var cur_label = document.getElementById(cur_tab + '_label');
+ cur_label.className = 'configTab';
+ } catch (e) {
+ return (cur_tab);
+ }
+ return (new_tab);
+}
--- conga/luci/plone-custom/conga_ajax.js 2007/11/27 16:58:18 1.5
+++ conga/luci/plone-custom/conga_ajax.js 2007/12/12 15:50:18 1.6
@@ -136,8 +136,8 @@
}
try {
- vd.className.replace(/invisible/gi, '');
- } catch (...) {
+ vd.className = vd.className.replace(/invisible/gi, '');
+ } catch (e) {
/* ignore it */;
}
}
--- conga/luci/plone-custom/luci.css2007/11/27 16:58:18 1.1
+++ conga/luci/plone-custom/luci.css2007/12/12 15:50:18 1.2
@@ -138,7 +138,7 @@
border-bottom: none ! important;
}
-#configTabContent {
+div.configTabMainContent {
background: #dee7ec !important;
margin-top: 0px !important;
padding: 20px !important;
