Hi Adam, We already have been running with "echo 0 > /proc/sys/portals/debug" across all Lustre boxes: MDS, OSSs, clients.
Dear Lustre community: Please jump to the bottom for specific questions. Thank you. Since we switched to Lustre 2 weeks ago, we've noticed the following: - The site and its various components seem to load faster - The same measured through Keynote, a metrics and monitoring company, report that on average, pages now load a bit slower. The _average_ page load time in recent weeks and months used to be 1.7 - 2.1 sec; for the past 2 weeks it has been 2.0 - 2.4 sec. Traffic levels have remained unchanged. - When we crawl the Lustre file system, say for indexing or regular-expression substitution purposes, processing about 1MM files takes ~50 min over NFS, and ~2 hrs on Lustre, for an identical set of files. We used to mount NFS with: rw,rsize=8192,wsize=8192,soft,intr,async,nodev We mount Lustre with: defaults,_netdev on most clients and defaults,_netdev,flock on a couple of special clients running applications like Subversion requiring exclusive file locks defaults,_netdev,ro on a few special clients Additionally, we mount the OSTs on the OSSs with: fstype ldiskfs mountfsoptions extents,mballoc On the MDS: ls -1 /proc/fs/lustre/ldlm/namespaces/*/lru_size /proc/fs/lustre/ldlm/namespaces/OSC_lustre1_ost1p_mds-prod/lru_size /proc/fs/lustre/ldlm/namespaces/OSC_lustre1_ost2p_mds-prod/lru_size cat /proc/fs/lustre/ldlm/namespaces/*/lru_size 1000 1000 On the clients: ls -1 /proc/fs/lustre/ldlm/namespaces/*/lru_size /proc/fs/lustre/ldlm/namespaces/MDC_lustre1_mds-prod_MNT_client-prod-f7f 23600/lru_size /proc/fs/lustre/ldlm/namespaces/OSC_lustre1_ost1p_MNT_client-prod-f7f236 00/lru_size /proc/fs/lustre/ldlm/namespaces/OSC_lustre1_ost2p_MNT_client-prod-f7f236 00/lru_size cat /proc/fs/lustre/ldlm/namespaces/*/lru_size 400 400 400 /proc/fs/lustre/llite/fs0/max_read_ahead_mb 4 /proc/fs/lustre/llite/fs0/max_read_ahead_whole_mb 1 Specific questions: 1) I am not sure about max_read_ahead_whole_mb - it's unclear to me whether 0 or 1 should be preferred here. 2) Should I prefer 2500 for /proc/fs/lustre/ldlm/namespaces/MDC*/lru_size and 1000 or 1500 for /proc/fs/lustre/ldlm/namespaces/OSC*/lru_size? 3) If yes, should I be setting this across the board, on the clients only, or on the MDS/OSSs only? 4) In general, should I still stick to avoiding mounting with flock unless explicitly required? Thank you very much, Zlatin _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
