Very good info indeed. I will be looking on the script, Kevin.
It would be sufficient for a while, but about the feature, it would be nice to
be fixed/documented/explained how to use it, because it’s a pretty common use
case of stateless nodes.
Perhaps someone on the dev team can look at this? Should we open a ticket on
the issue tracker?
V.
> On 27 Nov 2017, at 17:02, Gilad Berman <gber...@lenovo.com> wrote:
>
> THX Kevin for the tip!! <>
>
> We actually used similar method, but the first post on the thread reminded me
> of the localdisk feature and I thought it can be very nice to use it, if
> working.
>
> <image001.jpg>
> Gilad Berman
> HPC Architect
> Lenovo EMEA
> <image002.png>+972-52-2554262
> <image003.png>gber...@lenovo.com <mailto:gber...@lenovo.com>
>
> Lenovo.com <http://www.lenovo.com/>
> Twitter <http://twitter.com/lenovo> | Facebook
> <http://www.facebook.com/lenovo> | Instagram <https://instagram.com/lenovo> |
> Blogs <http://blog.lenovo.com/> | Forums <http://forums.lenovo.com/>
> <image004.jpg>
>
>
> From: Kevin Keane [mailto:kke...@sandiego.edu <mailto:kke...@sandiego.edu>]
> Sent: Monday, November 27, 2017 7:22 PM
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net
> <mailto:xcat-user@lists.sourceforge.net>>
> Subject: Re: [xcat-user] Local scratch for stateless compute nodes
>
> To address this, we are using the syncfiles mechanism to copy an rc.local
> file into the compute node (we could probably also put it directly into the
> image)
>
> This rc.local contains statements to mount the /tmp volume. We originally
> also used it to partition and format the physical hard disk, but that proved
> too dangerous when somebody accidentally ran the script on the management
> node and wiped out the partition table... You could probably do something
> similar with a swap partition.
>
> Here is the script we are using.
>
> A few notes:
> - a prerequisite is an entry that mounts /dev/sda1 as /localscratch in fstab.
> - moving the content from /tmp to /localscratch/tmp actually isn't working
> flawlessly; it is just good enough for our purposes.
>
> #!/bin/sh
> #
> # This script will be executed *after* all the other init scripts.
> # You can put your own initialization stuff in here if you don't
> # want to do the full Sys V style init stuff.
>
> touch /var/lock/subsys/local
>
> # Let's see if the local disk is already formatted and set up - if so,
> # we won't redo it.
> if [ -d /localscratch/tmp ]
> then
> echo "Localscratch disk is already formatted"
> else
> umount /dev/sda1
>
> ##################################################################
> # DANGER DANGER DANGER! This code can, and will, blindly destroy
> # partitions on whatever computer it is run. Partition recovery
> # is not easy.
> ##################################################################
> # Format the disk in the compute node. Single partition, mounted
> # as localscratch.
> #dd if=/dev/zero of=/dev/sda bs=1M count=100
> #parted -s /dev/sda mklabel gpt
> #parted -s -a optimal /dev/sda mkpart primary ext3 0% 100%
> #mkfs -t ext3 /dev/sda1
>
> mount -a
>
> fi
>
> # Create the /tmp and /var/tmp directories.
>
> for i in /tmp /var/tmp /var/log
> do
> # Make sure mv includes .dotfiles
> shopt -s dotglob
> mkdir -p /localscratch$i
> chmod 755 /localscratch
>
> case "$i" in
> /tmp)
> # The first digit in the mode is the sticky bit.
> chmod 1777 /localscratch$i
> ;;
> /var/tmp)
> # The first digit in the mode is the sticky bit.
> chmod 1777 /localscratch$i
> ;;
> *)
> chmod 755 /localscratch$i
> esac
>
> if [ ! -h $i ]
> then
> if [ -n "$(ls -A $i)" ]
> then
> mv $i/* /localscratch$i
> fi
> # In theory, the directory should be empty because we moved everything
> # out of the way. But that may have failed if the localscratch directory
> # was already used.
> rm -rf $i
> ln -sf /localscratch$i $i
> fi
> shopt -u dotglob
> done
>
> mkdir -p /localscratch/ansys
> chmod 777 /localscratch/ansys
>
>
> On Mon, Nov 27, 2017 at 6:57 AM, Gilad Berman <gber...@lenovo.com
> <mailto:gber...@lenovo.com>> wrote:
> 1. <> I use local disk for scratch and swap. Somethings logs as well (in
> this case you can think of it as sort of statelite, but from xCAT
> perspective, it is still stateless).
>
> 2. I took only the part that not relate to statelite from the
> instructions – not working.
>
>
> <image009.jpg>
> Gilad Berman
> HPC Architect
> Lenovo EMEA
> <image010.png>+972-52-2554262 <tel:+972%2052-255-4262>
> <image011.png>gber...@lenovo.com <mailto:gber...@lenovo.com>
>
> Lenovo.com <http://www.lenovo.com/>
> Twitter <http://twitter.com/lenovo> | Facebook
> <http://www.facebook.com/lenovo> | Instagram <https://instagram.com/lenovo> |
> Blogs <http://blog.lenovo.com/> | Forums <http://forums.lenovo.com/>
> <image012.jpg>
>
>
> From: Russ Auld [mailto:russa...@comcast.net <mailto:russa...@comcast.net>]
> Sent: Monday, November 27, 2017 4:50 PM
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net
> <mailto:xcat-user@lists.sourceforge.net>>
> Subject: Re: [xcat-user] Local scratch for stateless compute nodes
>
> If you're using netboot and local disk, then isn't that "statelite"?
> Do the satellite instructions not work?
>
> On Nov 27, 2017 9:26 AM, Gilad Berman <gber...@lenovo.com
> <mailto:gber...@lenovo.com>> wrote:
> All,
>
>
>
> I would like to join this question –
>
> Does even localdisk works with stateless? From the docs it seems that should
> be supported (because it is under stateless), however –
>
> - the instructions are taken from statelite and refer to statelite
> code (litefile)
>
> - The rc.localdisk code is under statelite
>
> - In the linuximage man – “Partitionfile - Only available for diskful
> osimages and statelite osimages(localdisk enabled)“
>
>
>
> A very quick trial on my statless nodes results in nothing J, it seems there
> is simply no reference to localdisk with stateless.
>
>
>
> So, can someone please help clarify it?
>
>
>
> ** as always, there is a chance I missed something very basic and it should
> be working J
>
>
>
> THX in advance!
>
>
>
>
> Gilad Berman
> HPC Architect
> Lenovo EMEA
>
> +972-52-2554262 <tel:+972%2052-255-4262>
> gber...@lenovo.com <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> Lenovo.com
> Twitter | Facebook | Instagram | Blogs | Forums <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> From: Vinícius Ferrão [mailto:fer...@versatushpc.com.br]
> Sent: Wednesday, November 22, 2017 4:09 AM
> To: xcat-user@lists.sourceforge.net
> Subject: [xcat-user] Local scratch for stateless compute nodes
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> Hello, <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> I would like to enable swap and local /tmp on my stateless nodes, but after
> following the documentation on the following link nothing appears to work:
> <mailto:gber...@lenovo.com>
> http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> I’m aware that the documentation is for service nodes and not for compute
> nodes, but I was thinking the procedure would be similar.
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> At this point I’m with this settings on osimage: <mailto:gber...@lenovo.com>
> [root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute
> exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist
> imagetype=linux
> osarch=x86_64
> osdistroname=centos7.4-x86_64
> osname=Linux
> osvers=centos7.4
> otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64
> <mailto:gber...@lenovo.com>
> ****
> partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
> <mailto:gber...@lenovo.com>
> permission=755
> pkgdir=/install/centos7.4/x86_64
> pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist
>
> postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall
> profile=compute
> provmethod=netboot
> rootimgdir=/install/netboot/centos7.4/x86_64/compute
> synclists=/install/custom/netboot/compute.synclist
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> And the content of the partition file is the following:
> <mailto:gber...@lenovo.com>
> cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
> enable=yes
> enablepart=yes
>
> [disk]
> dev=/dev/sda
> clear=yes
> parts=10,90
>
> [swapspace]
> dev=/dev/sda1
>
> [localspace]
> dev=/dev/sda2
> fstype=xfs <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> Finally the following commands were executed: <mailto:gber...@lenovo.com>
> chtab priority=7.1 policy.commands=getpartition policy.rule=allow
> <mailto:gber...@lenovo.com>
> chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/
> litefile.options=localdisk <mailto:gber...@lenovo.com>
> chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/
> litefile.options=localdisk <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> After a new genimage/packimage the local scratch does not appears to be made
> nor working. <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> Someone knows what to do next? <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> Thanks, <mailto:gber...@lenovo.com>
> V. <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
> <mailto:gber...@lenovo.com>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
> <mailto:gber...@lenovo.com>
>
>
>
> -- <mailto:gber...@lenovo.com>
> _______________________________________________________________________
> Kevin Keane | Systems Architect | University of San Diego ITS |
> kke...@sandiego.edu
> Maher Hall, 192 |5998 Alcalá Park | San Diego, CA 92110-2492 | 619.260.6859
> <mailto:gber...@lenovo.com>------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org <http://slashdot.org/>!
> http://sdm.link/slashdot_______________________________________________
> <http://sdm.link/slashdot_______________________________________________>
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net <mailto:xCAT-user@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/xcat-user
> <https://lists.sourceforge.net/lists/listinfo/xcat-user>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
