Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-29 Thread Vinícius Ferrão 
Wow… thank you so much Bin.

“nodeset" done the trick too. It worked on CentOS 7.4.

Just an additional note, when creating definitions with chtab the trailling 
slash on directories must be written or else a file will be created instead of 
a directory:

Mount /dev/sda2 to /.sllocal/localmnt with xfs
mount -t xfs /dev/sda2 /sysroot/.sllocal/localmnt
DEBUG: SYNCTOLOCAL=1
mkdir -p /sysroot/tmp/
mkdir -p /sysroot/.sllocal/localmnt/tmp/
mount --bind /sysroot/.sllocal/localmnt/tmp/ /sysroot/tmp/
touch /sysroot/work
touch /sysroot/.sllocal/localmnt/work
mount --bind /sysroot/.sllocal/localmnt/work /sysroot/work

After editing the lines with tabedit litefile it worked as expected.

Thanks,
V.

> On 29 Nov 2017, at 01:42, Bin XA Xu <bx...@cn.ibm.com> wrote:
> 
> 'localdisk' feature is also supported for 'stateless' node in case there are 
> some special requirements to leverage the local disk.
>  
> You could check your rootfs directory for your stateless osimage, you can see 
> './etc/rc.d/init.d/localdisk' is there (it is installed from `rc.localdisk`). 
> And it will be run each time when the stateless or statelite nodes booting.
>  
> In this script, it will parse /proc/cmdline to see if containing variable 
> like PARTITION*, if yes, then it try to get the partitonfile via 
> `getpartition`
> and do the partitioning.
>  
> So, the first thing we need to check is that if you boot configuration file 
> contains variables like 'PARTITION_DOMOUNT_RH'.  If not there, maybe you miss 
> to run `nodeset` again after you change the osimage.
>  
> If the variable is there,  I think you can check the log files (/.sllocal/log 
> on CN) to see what happens on you compute node.
>  
> ```
> cat /.sllocal/log/localdisk.log
> kernel parameter key is: imgurl
> kernel parameter key is: XCAT
> kernel parameter key is: NODE
> kernel parameter key is: FC
> kernel parameter key is: LOGSERVER
> kernel parameter key is: syslog.server
> kernel parameter key is: syslog.type
> kernel parameter key is: syslog.filter
> kernel parameter key is: xcatdebugmode
> kernel parameter key is: BOOTIF
> kernel parameter key is: selinux
> kernel parameter key is: PARTITION_DOMOUNT_RH
> Run localdisk configuration
> DEBUG: XCATSERVER=[172.10.253.27:3001]; DOPART=[yes]; DOMOUNT=[yes]; OS=[rh]
> DEBUG: enable=[yes]; enablepart=[no]; disk=[0]; localspace=[0]; 
> swapspace=[0]; dev=[]; parts=[]; clear=[];
> DEBUG: enable=[yes]; enablepart=[no]; disk=[1]; localspace=[0]; 
> swapspace=[0]; dev=[/dev/sda]; parts=[10,50]; clear=[yes];
> DEBUG: enable=[yes]; enablepart=[no]; disk=[0]; localspace=[1]; 
> swapspace=[0]; dev=[/dev/sda2]; parts=[10,50]; clear=[yes];
> Mount /dev/sda2 to /.sllocal/localmnt with ext4
> mount -t ext4 /dev/sda2 /sysroot/.sllocal/localmnt
> DEBUG: enable=[yes]; enablepart=[no]; disk=[0]; localspace=[0]; 
> swapspace=[1]; dev=[/dev/sda1]; parts=[10,50]; clear=[yes];
> Create swap on /dev/sda1
>   mkswap /dev/sda1
>   swapon /dev/sda1
> DEBUG: Ignored syncing to local as flag file exists.
> DEBUG: SYNCTOLOCAL=0
> mkdir -p /sysroot/install/
> mount --bind /sysroot/.sllocal/localmnt/install/ /sysroot/install/
> mount --bind /sysroot/.sllocal/localmnt/tftpboot/ /sysroot/tftpboot/
> mkdir -p /sysroot/tmp/
> mount --bind /sysroot/.sllocal/localmnt/tmp/ /sysroot/tmp/
> mount --bind /sysroot/.sllocal/localmnt/var/log/ /sysroot/var/log/
> ```
>  
> BTW: we just test this feature on RHEL, and not try it on CentOS before.
>  
> Note:  the local space will be mounted on '/.sllocal/localmnt' and
> `litefile` here is just for re-mount the sub-directory, if you only want a 
> `scratch` directory, not required for it.
>  
> Or an easy method is to using a postscript just like 
> /install/postscripts/setupscratch
>  
> Bin Xu
> HPC Software Development
> Software Defined Infrastructure, IBM Systems
> Phone: 86-010-82454067
> E-mail: bx...@cn.ibm.com <mailto:bx...@cn.ibm.com>
>  
>  
> - Original message -
> From: "Yuan Y Bai" <by...@cn.ibm.com>
> To: xcat-user@lists.sourceforge.net
> Cc: xcat-user@lists.sourceforge.net
> Subject: Re: [xcat-user] Local scratch for stateless compute nodes
> Date: Wed, Nov 29, 2017 11:00 AM
>  
> Hi Bin,
>  
> Could you help look into this issue?
>  
> Best Regards
> --
> Yuan Bai (白媛)
> 
> CSTL HPC System Management Development
> Tel:86-10-82451401
> E-mail: by...@cn.ibm.com
> Address: IBM ZGC Campus. Ring Building 28,
> ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,
> Beijing P.R.China 100193
> 
> IBM环宇大厦
> 北京市海淀区东北旺西路8号，中关村软件园28号楼
> 邮编:100193
>  
>  
> - Original message -
>

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-28 Thread Bin XA Xu 
 it?    I can’t find anything related to partition nor local disk setup in the logs. 
 
Here is my local disk file – 
enable=yes
enablepart=yes
 
[disk]
dev=/dev/sda
clear=yes
parts=1,19,80
 
[swapspace]
dev=/dev/sda1
 
[localspace]
dev=/dev/sda2
fstype=ext4
 
[localspace]
dev=/dev/sda3
fstype=ext4
 
 
 
# lsdef -t osimage -o rhels7.3-x86_64-netboot-compute
Object name: rhels7.3-x86_64-netboot-compute
    exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.exlist
    imagetype=linux
    osarch=x86_64
    osdistroname=rhels7.3-x86_64
    osname=Linux
    osvers=rhels7.3
    otherpkgdir=/install/post/otherpkgs/rhels7.3/x86_64
    partitionfile=/install/custom/netboot/localdisk
    permission=755
    pkgdir=/install/rhels7.3/x86_64
    pkglist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.pkglist
    postinstall=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.postinstall
    profile="">
    provmethod=netboot
    rootimgdir=/install/netboot/rhels7.3/x86_64/compute
    synclists=/install/custom/netboot/compute.synclist

THX!!
 
 
Gilad BermanHPC ArchitectLenovo EMEA+972-52-2554262gber...@lenovo.com  Lenovo.com Twitter | Facebook | Instagram | Blogs | Forums 
 
 
From: Yuan Y Bai [mailto:by...@cn.ibm.com]Sent: Tuesday, November 28, 2017 7:30 AMTo: xcat-user@lists.sourceforge.netCc: xcat-user@lists.sourceforge.netSubject: Re: [xcat-user] Local scratch for stateless compute nodes
 
Hello,
 
Please refer to "Enabling the localdisk Option" section under "diskless installation" section :  http://xcat-docs.readthedocs.io/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.html
 
The main diskless installation doc link is here:
http://xcat-docs.readthedocs.io/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskless/index.html
 
 
Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: by...@cn.ibm.comAddress: IBM ZGC Campus. Ring Building 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,Beijing P.R.China 100193IBM环宇大厦北京市海淀区东北旺西路8号，中关村软件园28号楼邮编:100193
 
 
- Original message -From: Vinícius Ferrão <fer...@versatushpc.com.br>To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>Cc:Subject: Re: [xcat-user] Local scratch for stateless compute nodesDate: Tue, Nov 28, 2017 12:40 PM 
Hello,
 
The major problem is the missing documentation on how to use the local disks on a stateless (diskless) installation. Tricks are being deployed instead. 
 
Trying to mimic the steps of a statelite installation is not sufficient to make it work on stateless deployments.
 
V. 
  
On 28 Nov 2017, at 00:15, Yuan Y Bai <by...@cn.ibm.com> wrote:
  
That is correct, diskless and statelite both support local disk for different purposes now.
 
Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: by...@cn.ibm.comAddress: IBM ZGC Campus. Ring Building 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,Beijing P.R.C

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-28 Thread Yuan Y Bai 
  +972-52-2554262gber...@lenovo.com  Lenovo.com Twitter | Facebook | Instagram | Blogs | Forums 
 
 
From: Vinícius Ferrão [mailto:fer...@versatushpc.com.br]Sent: Wednesday, November 22, 2017 4:09 AMTo: xcat-user@lists.sourceforge.netSubject: [xcat-user] Local scratch for stateless compute nodes
 
Hello,
 
I would like to enable swap and local /tmp on my stateless nodes, but after following the documentation on the following link nothing appears to work:
http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html
 
I’m aware that the documentation is for service nodes and not for compute nodes, but I was thinking the procedure would be similar.
 
At this point I’m with this settings on osimage:
[root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute    exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist    imagetype=linux    osarch=x86_64    osdistroname=centos7.4-x86_64    osname=Linux    osvers=centos7.4    otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64
    partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
    permission=755    pkgdir=/install/centos7.4/x86_64    pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist    postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall    profile="">    provmethod=netboot    rootimgdir=/install/netboot/centos7.4/x86_64/compute    synclists=/install/custom/netboot/compute.synclist
 
And the content of the partition file is the following:
cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfileenable=yesenablepart=yes[disk]dev=/dev/sdaclear=yesparts=10,90[swapspace]dev=/dev/sda1[localspace]dev=/dev/sda2fstype=xfs
 
Finally the following commands were executed:
chtab priority=7.1 policy.commands=getpartition policy.rule=allow
chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/ litefile.options=localdisk
chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/ litefile.options=localdisk
 
After a new genimage/packimage the local scratch does not appears to be made nor working.
 
Someone knows what to do next?
 
Thanks,
V.
 
--Check out the vibrant tech community on one of the world's mostengaging tech sites,  Slashdot.org!  http://sdm.link/slashdot
___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/xcat-user
--Check out the vibrant tech community on one of the world's mostengaging tech sites,  Slashdot.org!  https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot=DwICAg=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=dz2jXUMV84xRcXmPqkIFax-QT2ohLF-q9GVvyjAwq4E=
___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser=DwICAg=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=pxDjbuNwMM5N-ML97kogip_ZqRD1ioRXc3AVwGp6v_0=
 
--Check out the vibrant tech community on one of the world's mostengaging tech sites,  Slashdot.org!  http://sdm.link/slashdot___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/xcat-user
--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org!  https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot=DwICAg=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=bJlh5sWbgdtZM7vwR5StzuCLPzUxwmdKcNMoKV431G8=MZdE_Br9c0Q15XZHVl7kH-W9YEuArDRrhJ2PRrwwquI=
___xCAT-user mailing listxCA

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-28 Thread Yuan Y Bai 
 
Hi Gilad,
Could you add an entry in policy table to permit the running of the "getpartition" command from the node?
 
chtab priority=7.1 policy.commands=getpartition policy.rule=allow
 
Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: by...@cn.ibm.comAddress: IBM ZGC Campus. Ring Building 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,Beijing P.R.China 100193IBM环宇大厦北京市海淀区东北旺西路8号，中关村软件园28号楼邮编:100193
 
 
- Original message -From: Gilad Berman <gber...@lenovo.com>To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>Cc:Subject: Re: [xcat-user] Local scratch for stateless compute nodesDate: Tue, Nov 28, 2017 8:34 PM  
Yuan Bai hello,
 
I did follow exactly those instructions (although I am pretty sure the litefile table part is not needed for the stateless image) but I still see nothing happening on the nodes. I suspect that either there is no piece of code for stateless that handle localdisk or I am missing something very basic. 
 
Have you tried it with stateless? Where can I find the code and how I can debug it?    I can’t find anything related to partition nor local disk setup in the logs. 
 
Here is my local disk file – 
enable=yes
enablepart=yes
 
[disk]
dev=/dev/sda
clear=yes
parts=1,19,80
 
[swapspace]
dev=/dev/sda1
 
[localspace]
dev=/dev/sda2
fstype=ext4
 
[localspace]
dev=/dev/sda3
fstype=ext4
 
 
 
# lsdef -t osimage -o rhels7.3-x86_64-netboot-compute
Object name: rhels7.3-x86_64-netboot-compute
    exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.exlist
    imagetype=linux
    osarch=x86_64
    osdistroname=rhels7.3-x86_64
    osname=Linux
    osvers=rhels7.3
    otherpkgdir=/install/post/otherpkgs/rhels7.3/x86_64
    partitionfile=/install/custom/netboot/localdisk
    permission=755
    pkgdir=/install/rhels7.3/x86_64
    pkglist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.pkglist
    postinstall=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.postinstall
    profile="">
    provmethod=netboot
    rootimgdir=/install/netboot/rhels7.3/x86_64/compute
    synclists=/install/custom/netboot/compute.synclist

THX!!
 
 
Gilad BermanHPC ArchitectLenovo EMEA+972-52-2554262gber...@lenovo.com  Lenovo.com Twitter | Facebook | Instagram | Blogs | Forums 
 
 
From: Yuan Y Bai [mailto:by...@cn.ibm.com]Sent: Tuesday, November 28, 2017 7:30 AMTo: xcat-user@lists.sourceforge.netCc: xcat-user@lists.sourceforge.netSubject: Re: [xcat-user] Local scratch for stateless compute nodes
 
Hello,
 
Please refer to "Enabling the localdisk Option" section under "diskless installation" section :  http://xcat-docs.readthedocs.io/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.html
 
The main diskless installation doc link is here:
http://xcat-docs.readthedocs.io/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskless/index.html
 
 
Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: by

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-28 Thread Gilad Berman 
Yuan Bai hello,

I did follow exactly those instructions (although I am pretty sure the litefile 
table part is not needed for the stateless image) but I still see nothing 
happening on the nodes. I suspect that either there is no piece of code for 
stateless that handle localdisk or I am missing something very basic.

Have you tried it with stateless? Where can I find the code and how I can debug 
it?I can’t find anything related to partition nor local disk setup in the 
logs.

Here is my local disk file –
enable=yes
enablepart=yes

[disk]
dev=/dev/sda
clear=yes
parts=1,19,80

[swapspace]
dev=/dev/sda1

[localspace]
dev=/dev/sda2
fstype=ext4

[localspace]
dev=/dev/sda3
fstype=ext4



# lsdef -t osimage -o rhels7.3-x86_64-netboot-compute
Object name: rhels7.3-x86_64-netboot-compute
exlist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.exlist
imagetype=linux
osarch=x86_64
osdistroname=rhels7.3-x86_64
osname=Linux
osvers=rhels7.3
otherpkgdir=/install/post/otherpkgs/rhels7.3/x86_64
partitionfile=/install/custom/netboot/localdisk
permission=755
pkgdir=/install/rhels7.3/x86_64
pkglist=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.pkglist

postinstall=/opt/xcat/share/xcat/netboot/rh/compute.rhels7.x86_64.postinstall
profile=compute
provmethod=netboot
rootimgdir=/install/netboot/rhels7.3/x86_64/compute
synclists=/install/custom/netboot/compute.synclist
THX!!


[http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]

Gilad Berman
HPC Architect
Lenovo EMEA

[Phone]+972-52-2554262
[Email]gber...@lenovo.com<mailto:gber...@lenovo.com>



Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>


[DCG-Hardware]




From: Yuan Y Bai [mailto:by...@cn.ibm.com]
Sent: Tuesday, November 28, 2017 7:30 AM
To: xcat-user@lists.sourceforge.net
Cc: xcat-user@lists.sourceforge.net
Subject: Re: [xcat-user] Local scratch for stateless compute nodes

Hello,

Please refer to "Enabling the localdisk Option" section under "diskless 
installation" section : 
http://xcat-docs.readthedocs.io/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/localdisk.html

The main diskless installation doc link is here:
http://xcat-docs.readthedocs.io/en/latest/guides/admin-guides/manage_clusters/ppc64le/diskless/index.html


Best Regards
--
Yuan Bai (白媛)

CSTL HPC System Management Development
Tel:86-10-82451401
E-mail: by...@cn.ibm.com<mailto:by...@cn.ibm.com>
Address: IBM ZGC Campus. Ring Building 28,
ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,
Beijing P.R.China 100193

IBM环宇大厦
北京市海淀区东北旺西路8号，中关村软件园28号楼
邮编:100193


- Original message -
From: Vinícius Ferrão 
<fer...@versatushpc.com.br<mailto:fer...@versatushpc.com.br>>
To: xCAT Users Mailing list 
<xcat-user@lists.sourceforge.net<mailto:xcat-user@lists.sourceforge.net>>
Cc:
Subject: Re: [xcat-user] Local scratch for stateless compute nodes
Date: Tue, Nov 28, 2017 12:40 PM

Hello,

The major problem is the missing documentation on how to use the local disks on 
a stateless (diskless) installation. Tricks are being deployed instead.

Trying to mimic the steps of a statelite installation is not sufficient to make 
it work on stateless deployments.

V.

On 28 Nov 2017, at 00:15, Yuan Y Bai 
<by...@cn.ibm.com<mailto:by...@cn.ibm.com>> wrote:

That is correct, diskless and statelite both support local disk for different 
purposes now.

Best Regards
--
Yuan Bai (白媛)

CSTL HPC System Management Development
Tel:86-10-82451401
E-mail: by...@cn.ibm.com<mailto:by...@cn.ibm.com>
Address: IBM ZGC Campus. Ring Building 28,
ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,
Beijing P.R.China 100193

IBM环宇大厦
北京市海淀区东北旺西路8号，中关村软件园28号楼
邮编:100193


- Original message -
From: Vinícius Ferrão 
<fer...@versatushpc.com.br<mailto:fer...@versatushpc.com.br>>
To: xCAT Users Mailing list 
<xcat-user@lists.sourceforge.net<mailto:xcat-user@lists.sourceforge.net>>
Cc:
Subject: Re: [xcat-user] Local scratch for stateless compute nodes
Date: Tue, Nov 28, 2017 12:14 AM


Hello,

Accordingly to the documentation they are different on a specific way:

. Stateless: nodes boot from a RAMdisk OS image downloaded from the xCAT mgmt 
node or service node at boot time.

. Statelite: nodes boot from an NFS-root diskless OS image.

And both support local disk for different purposes, what we are targeting are 
local disks just for scratch and swap. They will not handle any state, and this 
is perfectly supported as said on the documentation.

H

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Vinícius Ferrão 
Hello,

The major problem is the missing documentation on how to use the local disks on 
a stateless (diskless) installation. Tricks are being deployed instead.

Trying to mimic the steps of a statelite installation is not sufficient to make 
it work on stateless deployments.

V.

> On 28 Nov 2017, at 00:15, Yuan Y Bai <by...@cn.ibm.com> wrote:
> 
> That is correct, diskless and statelite both support local disk for different 
> purposes now.
>  
> Best Regards
> --
> Yuan Bai (白媛)
> 
> CSTL HPC System Management Development
> Tel:86-10-82451401
> E-mail: by...@cn.ibm.com
> Address: IBM ZGC Campus. Ring Building 28,
> ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,
> Beijing P.R.China 100193
> 
> IBM环宇大厦
> 北京市海淀区东北旺西路8号，中关村软件园28号楼
> 邮编:100193
>  
>  
> - Original message -
> From: Vinícius Ferrão <fer...@versatushpc.com.br>
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> Cc:
> Subject: Re: [xcat-user] Local scratch for stateless compute nodes
> Date: Tue, Nov 28, 2017 12:14 AM
>  
>  
> Hello,
>  
> Accordingly to the documentation they are different on a specific way:
>  
> . Stateless: nodes boot from a RAMdisk OS image downloaded from the xCAT mgmt 
> node or service node at boot time.
>  
> . Statelite: nodes boot from an NFS-root diskless OS image.
>  
> And both support local disk for different purposes, what we are targeting are 
> local disks just for scratch and swap. They will not handle any state, and 
> this is perfectly supported as said on the documentation.
>  
> Here’s the documentation: 
> https://sourceforge.net/p/xcat/wiki/XCAT_Overview,_Architecture,_and_Planning/#xcat-cluster-node-types
>  
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__sourceforge.net_p_xcat_wiki_XCAT-5FOverview-2C-5FArchitecture-2C-5Fand-5FPlanning_-23xcat-2Dcluster-2Dnode-2Dtypes=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=fH1AxDWPRLVZLj9RfODpdKPJOENCWrDCRvH0_M8TsjA=>
>  
> Thanks,
> V.
>  
> Sent from my iPhone
> 
> On 27 Nov 2017, at 12:50, Russ Auld <russa...@comcast.net 
> <mailto:russa...@comcast.net>> wrote:
>  
>> If you're using netboot and local disk,  then isn't that "statelite"?
>> Do the satellite instructions not work? 
>>  
>> On Nov 27, 2017 9:26 AM, Gilad Berman <gber...@lenovo.com 
>> <mailto:gber...@lenovo.com>> wrote:
>> All, <>
>>  
>> I would like to join this question –
>> 
>> Does even localdisk works with stateless? From the docs it seems that should 
>> be supported (because it is under stateless), however –
>> 
>> - the instructions are taken from statelite and refer to statelite 
>> code (litefile)
>> 
>> - The rc.localdisk code is under statelite
>> 
>> - In the linuximage man – “Partitionfile - Only available for 
>> diskful osimages and statelite osimages(localdisk enabled)“
>> 
>>  
>> A very quick trial on my statless nodes results in nothing J, it seems there 
>> is simply no reference to localdisk with stateless.
>> 
>>  
>> So, can someone please help clarify it?
>> 
>>  
>> ** as always, there is a chance I missed something very basic and it should 
>> be working J
>> 
>>  
>> THX in advance!
>> 
>>  
>> 
>> 
>> Gilad Berman
>> HPC Architect
>> Lenovo EMEA
>> 
>> +972-52-2554262
>> gber...@lenovo.com <mailto:gber...@lenovo.com>
>>  
>>  
>> Lenovo.com  
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.lenovo.com_=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=X4IxhzhlpZBcc7jTaTYACA25o1VeJYQRFpalyqxbTAA=>
>> Twitter 
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_lenovo=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=gho5DwChsdnzs9Dw5ZV_88KpPl44Ip7sh52Wg0EBHhY=>
>>  | Facebook 
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.facebook.com_lenovo=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=JOvwli3J9WR0J39QGoi4B0QeJy10-ovWRDSiLphZUq8=>
>>  | Instagram 
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__instagram.com_lenovo=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=uiTcPxjMR44SPRNNb6l_nA=oAQVc_Sh4trqtOHvVelQBzsfz8vF7IL0CwQcnYf1Nbs=4AAUTt59CFyotMoxHfUfncLrD9IWapwC9-OIIZLV9zQ=>
>>  | Blogs 
>> <https://urldefense.proofpoint.com/v2/u

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Yuan Y Bai 
That is correct, diskless and statelite both support local disk for different purposes now.
 
Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: by...@cn.ibm.comAddress: IBM ZGC Campus. Ring Building 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,Beijing P.R.China 100193IBM环宇大厦北京市海淀区东北旺西路8号，中关村软件园28号楼邮编:100193
 
 
- Original message -From: Vinícius Ferrão <fer...@versatushpc.com.br>To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>Cc:Subject: Re: [xcat-user] Local scratch for stateless compute nodesDate: Tue, Nov 28, 2017 12:14 AM 
 
Hello,
 
Accordingly to the documentation they are different on a specific way:
 
. Stateless: nodes boot from a RAMdisk OS image downloaded from the xCAT mgmt node or service node at boot time.
 
. Statelite: nodes boot from an NFS-root diskless OS image.
 
And both support local disk for different purposes, what we are targeting are local disks just for scratch and swap. They will not handle any state, and this is perfectly supported as said on the documentation.
 
Here’s the documentation: https://sourceforge.net/p/xcat/wiki/XCAT_Overview,_Architecture,_and_Planning/#xcat-cluster-node-types
 
Thanks,
V. 
Sent from my iPhone
On 27 Nov 2017, at 12:50, Russ Auld <russa...@comcast.net> wrote: 
If you're using netboot and local disk,  then isn't that "statelite"?
Do the satellite instructions not work? 
 
On Nov 27, 2017 9:26 AM, Gilad Berman <gber...@lenovo.com> wrote:

All, 
 
I would like to join this question – 
Does even localdisk works with stateless? From the docs it seems that should be supported (because it is under stateless), however – 
- the instructions are taken from statelite and refer to statelite code (litefile)
- The rc.localdisk code is under statelite
- In the linuximage man – “Partitionfile - Only available for diskful osimages and statelite osimages(localdisk enabled)“ 
 
A very quick trial on my statless nodes results in nothing J, it seems there is simply no reference to localdisk with stateless. 
 
So, can someone please help clarify it? 
 
** as always, there is a chance I missed something very basic and it should be working J 
 
THX in advance!
 
Gilad BermanHPC ArchitectLenovo EMEA+972-52-2554262gber...@lenovo.com  Lenovo.com Twitter | Facebook | Instagram | Blogs | Forums 
 
 
From: Vinícius Ferrão [mailto:fer...@versatushpc.com.br]Sent: Wednesday, November 22, 2017 4:09 AMTo: xcat-user@lists.sourceforge.netSubject: [xcat-user] Local scratch for stateless compute nodes
 
Hello,
 
I would like to enable swap and local /tmp on my stateless nodes, but after following the documentation on the following link nothing appears to work:
http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html
 
I’m aware that the documentation is for service nodes and not for compute nodes, but I was thinking the procedure would be similar.
 
At this point I’m with this settings on osimage:
[root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-comp

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Christian Caruthers 
We’ve used similar scripts in the past without any checks to prevent unintended 
disasters. It would be pretty easy to use an if or case statement to ensure 
anything destructive only happens on the right systems.

Regards,
Christian Caruthers
Lenovo Professional Services
Mobile: 757-289-9872

From: Vinícius Ferrão [mailto:fer...@versatushpc.com.br]
Sent: Monday, November 27, 2017 2:17 PM
To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
Subject: Re: [xcat-user] Local scratch for stateless compute nodes

Very good info indeed. I will be looking on the script, Kevin.

It would be sufficient for a while, but about the feature, it would be nice to 
be fixed/documented/explained how to use it, because it’s a pretty common use 
case of stateless nodes.

Perhaps someone on the dev team can look at this? Should we open a ticket on 
the issue tracker?

V.


On 27 Nov 2017, at 17:02, Gilad Berman 
<gber...@lenovo.com<mailto:gber...@lenovo.com>> wrote:

THX Kevin for the tip!!

We actually used similar method, but the first post on the thread reminded me 
of the localdisk feature and I thought it can be very nice to use it, if 
working.



Gilad Berman
HPC Architect
Lenovo EMEA

+972-52-2554262
gber...@lenovo.com<mailto:gber...@lenovo.com>



Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>







From: Kevin Keane [mailto:kke...@sandiego.edu]
Sent: Monday, November 27, 2017 7:22 PM
To: xCAT Users Mailing list 
<xcat-user@lists.sourceforge.net<mailto:xcat-user@lists.sourceforge.net>>
Subject: Re: [xcat-user] Local scratch for stateless compute nodes

To address this, we are using the syncfiles mechanism to copy an rc.local file 
into the compute node (we could probably also put it directly into the image)
This rc.local contains statements to mount the /tmp volume. We originally also 
used it to partition and format the physical hard disk, but that proved too 
dangerous when somebody accidentally ran the script on the management node and 
wiped out the partition table... You could probably do something similar with a 
swap partition.

Here is the script we are using.

A few notes:
- a prerequisite is an entry that mounts /dev/sda1 as /localscratch in fstab.
- moving the content from /tmp to /localscratch/tmp actually isn't working 
flawlessly; it is just good enough for our purposes.

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

# Let's see if the local disk is already formatted and set up - if so,
# we won't redo it.
if [ -d /localscratch/tmp ]
then
  echo "Localscratch disk is already formatted"
else
  umount /dev/sda1

  ##
  # DANGER DANGER DANGER! This code can, and will, blindly destroy
  # partitions on whatever computer it is run. Partition recovery
  # is not easy.
  ##
  # Format the disk in the compute node. Single partition, mounted
  # as localscratch.
  #dd if=/dev/zero of=/dev/sda bs=1M count=100
  #parted -s /dev/sda mklabel gpt
  #parted -s -a optimal /dev/sda mkpart primary ext3 0% 100%
  #mkfs -t ext3 /dev/sda1

  mount -a

fi

# Create the /tmp and /var/tmp directories.

for i in /tmp /var/tmp /var/log
do
  # Make sure mv includes .dotfiles
  shopt -s dotglob
  mkdir -p /localscratch$i
  chmod 755 /localscratch

  case "$i" in
/tmp)
   # The first digit in the mode is the sticky bit.
   chmod 1777 /localscratch$i
   ;;
/var/tmp)
   # The first digit in the mode is the sticky bit.
   chmod 1777 /localscratch$i
   ;;
*)
   chmod 755 /localscratch$i
  esac

  if [ ! -h $i ]
  then
if [ -n "$(ls -A $i)" ]
then
  mv $i/* /localscratch$i
fi
# In theory, the directory should be empty because we moved everything
# out of the way. But that may have failed if the localscratch directory
# was already used.
rm -rf $i
ln -sf /localscratch$i $i
  fi
  shopt -u dotglob
done

mkdir -p /localscratch/ansys
chmod 777 /localscratch/ansys

On Mon, Nov 27, 2017 at 6:57 AM, Gilad Berman 
<gber...@lenovo.com<mailto:gber...@lenovo.com>> wrote:

1.  I use local disk for scratch and swap. Somethings logs as well (in this 
case you can think of it as sort of statelite, but from xCAT perspective, it is 
still stateless).

2.  I took only the part that not relate to statelite from the instructions 
– not working.



Gilad Berman
HPC Architect
Lenovo EMEA

+972-52-2554262<tel:+972%2052-255-4262>
gber...@lenovo.com<mailto:gber...@leno

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Vinícius Ferrão 
Very good info indeed. I will be looking on the script, Kevin.

It would be sufficient for a while, but about the feature, it would be nice to 
be fixed/documented/explained how to use it, because it’s a pretty common use 
case of stateless nodes.

Perhaps someone on the dev team can look at this? Should we open a ticket on 
the issue tracker?

V.

> On 27 Nov 2017, at 17:02, Gilad Berman <gber...@lenovo.com> wrote:
> 
> THX Kevin for the tip!! <>
>  
> We actually used similar method, but the first post on the thread reminded me 
> of the localdisk feature and I thought it can be very nice to use it, if 
> working.
>  
> 
> Gilad Berman
> HPC Architect
> Lenovo EMEA
> +972-52-2554262
> gber...@lenovo.com <mailto:gber...@lenovo.com>
>  
> Lenovo.com  <http://www.lenovo.com/>
> Twitter <http://twitter.com/lenovo> | Facebook 
> <http://www.facebook.com/lenovo> | Instagram <https://instagram.com/lenovo> | 
> Blogs <http://blog.lenovo.com/> | Forums <http://forums.lenovo.com/> 
> 
>  
>  
> From: Kevin Keane [mailto:kke...@sandiego.edu <mailto:kke...@sandiego.edu>] 
> Sent: Monday, November 27, 2017 7:22 PM
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net 
> <mailto:xcat-user@lists.sourceforge.net>>
> Subject: Re: [xcat-user] Local scratch for stateless compute nodes
>  
> To address this, we are using the syncfiles mechanism to copy an rc.local 
> file into the compute node (we could probably also put it directly into the 
> image)
> 
> This rc.local contains statements to mount the /tmp volume. We originally 
> also used it to partition and format the physical hard disk, but that proved 
> too dangerous when somebody accidentally ran the script on the management 
> node and wiped out the partition table... You could probably do something 
> similar with a swap partition.
> 
> Here is the script we are using.
> 
> A few notes:
> - a prerequisite is an entry that mounts /dev/sda1 as /localscratch in fstab.
> - moving the content from /tmp to /localscratch/tmp actually isn't working 
> flawlessly; it is just good enough for our purposes.
> 
> #!/bin/sh
> #
> # This script will be executed *after* all the other init scripts.
> # You can put your own initialization stuff in here if you don't
> # want to do the full Sys V style init stuff.
> 
> touch /var/lock/subsys/local
> 
> # Let's see if the local disk is already formatted and set up - if so,
> # we won't redo it.
> if [ -d /localscratch/tmp ]
> then
>   echo "Localscratch disk is already formatted"
> else
>   umount /dev/sda1
> 
>   ##
>   # DANGER DANGER DANGER! This code can, and will, blindly destroy
>   # partitions on whatever computer it is run. Partition recovery
>   # is not easy.
>   ##
>   # Format the disk in the compute node. Single partition, mounted
>   # as localscratch.
>   #dd if=/dev/zero of=/dev/sda bs=1M count=100
>   #parted -s /dev/sda mklabel gpt
>   #parted -s -a optimal /dev/sda mkpart primary ext3 0% 100%
>   #mkfs -t ext3 /dev/sda1
> 
>   mount -a
> 
> fi
> 
> # Create the /tmp and /var/tmp directories.
> 
> for i in /tmp /var/tmp /var/log
> do
>   # Make sure mv includes .dotfiles
>   shopt -s dotglob
>   mkdir -p /localscratch$i
>   chmod 755 /localscratch
> 
>   case "$i" in
> /tmp)
># The first digit in the mode is the sticky bit.
>chmod 1777 /localscratch$i
>;;
> /var/tmp)
># The first digit in the mode is the sticky bit.
>chmod 1777 /localscratch$i
>;;
> *)
>chmod 755 /localscratch$i
>   esac
> 
>   if [ ! -h $i ]
>   then
> if [ -n "$(ls -A $i)" ]
> then
>   mv $i/* /localscratch$i
> fi
> # In theory, the directory should be empty because we moved everything
> # out of the way. But that may have failed if the localscratch directory
> # was already used.
> rm -rf $i
> ln -sf /localscratch$i $i
>   fi
>   shopt -u dotglob
> done
> 
> mkdir -p /localscratch/ansys
> chmod 777 /localscratch/ansys
> 
>  
> On Mon, Nov 27, 2017 at 6:57 AM, Gilad Berman <gber...@lenovo.com 
> <mailto:gber...@lenovo.com>> wrote:
> 1. <>  I use local disk for scratch and swap. Somethings logs as well (in 
> this case you can think of it as sort of statelite, but from xCAT 
> perspective, it is still stateless).
> 
> 2.  I took only the part that not relate to statelite from the 
> instructions – not work

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Gilad Berman 
THX Kevin for the tip!!

We actually used similar method, but the first post on the thread reminded me 
of the localdisk feature and I thought it can be very nice to use it, if 
working.

[http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]

Gilad Berman
HPC Architect
Lenovo EMEA

[Phone]+972-52-2554262
[Email]gber...@lenovo.com<mailto:gber...@lenovo.com>



Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>


[DCG-Hardware]




From: Kevin Keane [mailto:kke...@sandiego.edu]
Sent: Monday, November 27, 2017 7:22 PM
To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
Subject: Re: [xcat-user] Local scratch for stateless compute nodes

To address this, we are using the syncfiles mechanism to copy an rc.local file 
into the compute node (we could probably also put it directly into the image)
This rc.local contains statements to mount the /tmp volume. We originally also 
used it to partition and format the physical hard disk, but that proved too 
dangerous when somebody accidentally ran the script on the management node and 
wiped out the partition table... You could probably do something similar with a 
swap partition.

Here is the script we are using.

A few notes:
- a prerequisite is an entry that mounts /dev/sda1 as /localscratch in fstab.
- moving the content from /tmp to /localscratch/tmp actually isn't working 
flawlessly; it is just good enough for our purposes.

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

# Let's see if the local disk is already formatted and set up - if so,
# we won't redo it.
if [ -d /localscratch/tmp ]
then
  echo "Localscratch disk is already formatted"
else
  umount /dev/sda1

  ##
  # DANGER DANGER DANGER! This code can, and will, blindly destroy
  # partitions on whatever computer it is run. Partition recovery
  # is not easy.
  ##
  # Format the disk in the compute node. Single partition, mounted
  # as localscratch.
  #dd if=/dev/zero of=/dev/sda bs=1M count=100
  #parted -s /dev/sda mklabel gpt
  #parted -s -a optimal /dev/sda mkpart primary ext3 0% 100%
  #mkfs -t ext3 /dev/sda1

  mount -a

fi

# Create the /tmp and /var/tmp directories.

for i in /tmp /var/tmp /var/log
do
  # Make sure mv includes .dotfiles
  shopt -s dotglob
  mkdir -p /localscratch$i
  chmod 755 /localscratch

  case "$i" in
/tmp)
   # The first digit in the mode is the sticky bit.
   chmod 1777 /localscratch$i
   ;;
/var/tmp)
   # The first digit in the mode is the sticky bit.
   chmod 1777 /localscratch$i
   ;;
*)
   chmod 755 /localscratch$i
  esac

  if [ ! -h $i ]
  then
if [ -n "$(ls -A $i)" ]
then
  mv $i/* /localscratch$i
fi
# In theory, the directory should be empty because we moved everything
# out of the way. But that may have failed if the localscratch directory
# was already used.
rm -rf $i
ln -sf /localscratch$i $i
  fi
  shopt -u dotglob
done

mkdir -p /localscratch/ansys
chmod 777 /localscratch/ansys

On Mon, Nov 27, 2017 at 6:57 AM, Gilad Berman 
<gber...@lenovo.com<mailto:gber...@lenovo.com>> wrote:

1.  I use local disk for scratch and swap. Somethings logs as well (in this 
case you can think of it as sort of statelite, but from xCAT perspective, it is 
still stateless).

2.  I took only the part that not relate to statelite from the instructions 
– not working.

[http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]

Gilad Berman
HPC Architect
Lenovo EMEA

[Phone]+972-52-2554262<tel:+972%2052-255-4262>
[Email]gber...@lenovo.com<mailto:gber...@lenovo.com>



Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>


[DCG-Hardware]




From: Russ Auld [mailto:russa...@comcast.net<mailto:russa...@comcast.net>]
Sent: Monday, November 27, 2017 4:50 PM
To: xCAT Users Mailing list 
<xcat-user@lists.sourceforge.net<mailto:xcat-user@lists.sourceforge.net>>
Subject: Re: [xcat-user] Local scratch for stateless compute nodes

If you're using netboot and local disk,  then isn't that "statelite"?
Do the satellite instructions not work?

On Nov 27, 2017 9:26 AM, Gilad Berman 
<gber...@lenovo.com<mailto:gber...@lenovo.com>&g

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Kevin Keane 
To address this, we are using the syncfiles mechanism to copy an rc.local
file into the compute node (we could probably also put it directly into the
image)

This rc.local contains statements to mount the /tmp volume. We originally
also used it to partition and format the physical hard disk, but that
proved too dangerous when somebody accidentally ran the script on the
management node and wiped out the partition table... You could probably do
something similar with a swap partition.

Here is the script we are using.

A few notes:
- a prerequisite is an entry that mounts /dev/sda1 as /localscratch in
fstab.
- moving the content from /tmp to /localscratch/tmp actually isn't working
flawlessly; it is just good enough for our purposes.

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

# Let's see if the local disk is already formatted and set up - if so,
# we won't redo it.
if [ -d /localscratch/tmp ]
then
  echo "Localscratch disk is already formatted"
else
  umount /dev/sda1

  ##
  # DANGER DANGER DANGER! This code can, and will, blindly destroy
  # partitions on whatever computer it is run. Partition recovery
  # is not easy.
  ##
  # Format the disk in the compute node. Single partition, mounted
  # as localscratch.
  #dd if=/dev/zero of=/dev/sda bs=1M count=100
  #parted -s /dev/sda mklabel gpt
  #parted -s -a optimal /dev/sda mkpart primary ext3 0% 100%
  #mkfs -t ext3 /dev/sda1

  mount -a

fi

# Create the /tmp and /var/tmp directories.

for i in /tmp /var/tmp /var/log
do
  # Make sure mv includes .dotfiles
  shopt -s dotglob
  mkdir -p /localscratch$i
  chmod 755 /localscratch

  case "$i" in
/tmp)
   # The first digit in the mode is the sticky bit.
   chmod 1777 /localscratch$i
   ;;
/var/tmp)
   # The first digit in the mode is the sticky bit.
   chmod 1777 /localscratch$i
   ;;
*)
   chmod 755 /localscratch$i
  esac

  if [ ! -h $i ]
  then
if [ -n "$(ls -A $i)" ]
then
  mv $i/* /localscratch$i
fi
# In theory, the directory should be empty because we moved everything
# out of the way. But that may have failed if the localscratch directory
# was already used.
rm -rf $i
ln -sf /localscratch$i $i
  fi
  shopt -u dotglob
done

mkdir -p /localscratch/ansys
chmod 777 /localscratch/ansys


On Mon, Nov 27, 2017 at 6:57 AM, Gilad Berman <gber...@lenovo.com> wrote:

> 1.  I use local disk for scratch and swap. Somethings logs as well
> (in this case you can think of it as sort of statelite, but from xCAT
> perspective, it is still stateless).
>
> 2.  I took only the part that not relate to statelite from the
> instructions – not working.
>
>
>
> [image:
> http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]
>
> *Gilad Berman*
> HPC Architect
> Lenovo EMEA
>
> [image: Phone]+972-52-2554262 <+972%2052-255-4262>
> [image: Email]gber...@lenovo.com <gber...@lenovo.com>
>
>
>
> Lenovo.com <http://www.lenovo.com/>
> Twitter <http://twitter.com/lenovo> | Facebook
> <http://www.facebook.com/lenovo> | Instagram
> <https://instagram.com/lenovo> | Blogs <http://blog.lenovo.com/> | Forums
> <http://forums.lenovo.com/>
>
> [image: DCG-Hardware]
>
>
>
>
>
> *From:* Russ Auld [mailto:russa...@comcast.net]
> *Sent:* Monday, November 27, 2017 4:50 PM
> *To:* xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> *Subject:* Re: [xcat-user] Local scratch for stateless compute nodes
>
>
>
> If you're using netboot and local disk,  then isn't that "statelite"?
>
> Do the satellite instructions not work?
>
>
>
> On Nov 27, 2017 9:26 AM, Gilad Berman <gber...@lenovo.com> wrote:
>
> All,
>
>
>
> I would like to join this question –
>
> Does even localdisk works with stateless? From the docs it seems that
> should be supported (because it is under stateless), however –
>
> - the instructions are taken from statelite and refer to
> statelite code (litefile)
>
> - The rc.localdisk code is under statelite
>
> - In the linuximage man – “Partitionfile - Only available for
> diskful osimages and statelite osimages(localdisk enabled)“
>
>
>
> A very quick trial on my statless nodes results in nothing J, it seems
> there is simply no reference to localdisk with stateless.
>
>
>
> So, can someone please help clarify it?
>
>
>
> ** as always, there is a chance I missed s

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Gilad Berman 
1.  I use local disk for scratch and swap. Somethings logs as well (in this 
case you can think of it as sort of statelite, but from xCAT perspective, it is 
still stateless).

2.  I took only the part that not relate to statelite from the instructions 
– not working.

[http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]

Gilad Berman
HPC Architect
Lenovo EMEA

[Phone]+972-52-2554262
[Email]gber...@lenovo.com<mailto:gber...@lenovo.com>



Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>


[DCG-Hardware]




From: Russ Auld [mailto:russa...@comcast.net]
Sent: Monday, November 27, 2017 4:50 PM
To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
Subject: Re: [xcat-user] Local scratch for stateless compute nodes

If you're using netboot and local disk,  then isn't that "statelite"?
Do the satellite instructions not work?

On Nov 27, 2017 9:26 AM, Gilad Berman 
<gber...@lenovo.com<mailto:gber...@lenovo.com>> wrote:

All,



I would like to join this question –

Does even localdisk works with stateless? From the docs it seems that should be 
supported (because it is under stateless), however –

- the instructions are taken from statelite and refer to statelite code 
(litefile)

- The rc.localdisk code is under statelite

- In the linuximage man – “Partitionfile - Only available for diskful 
osimages and statelite osimages(localdisk enabled)“



A very quick trial on my statless nodes results in nothing ☺, it seems there is 
simply no reference to localdisk with stateless.



So, can someone please help clarify it?



** as always, there is a chance I missed something very basic and it should be 
working ☺



THX in advance!



[http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]


Gilad Berman
HPC Architect
Lenovo EMEA


[Phone]+972-52-2554262
[Email]gber...@lenovo.com<mailto:gber...@lenovo.com>





Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>



[DCG-Hardware]







From: Vinícius Ferrão [mailto:fer...@versatushpc.com.br]
Sent: Wednesday, November 22, 2017 4:09 AM
To: xcat-user@lists.sourceforge.net<mailto:xcat-user@lists.sourceforge.net>
Subject: [xcat-user] Local scratch for stateless compute nodes



Hello,



I would like to enable swap and local /tmp on my stateless nodes, but after 
following the documentation on the following link nothing appears to work:

http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html



I’m aware that the documentation is for service nodes and not for compute 
nodes, but I was thinking the procedure would be similar.



At this point I’m with this settings on osimage:

[root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute
exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist
imagetype=linux
osarch=x86_64
osdistroname=centos7.4-x86_64
osname=Linux
osvers=centos7.4
otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64


partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile

permission=755
pkgdir=/install/centos7.4/x86_64
pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist
postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall
profile=compute
provmethod=netboot
rootimgdir=/install/netboot/centos7.4/x86_64/compute
synclists=/install/custom/netboot/compute.synclist



And the content of the partition file is the following:

cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
enable=yes
enablepart=yes

[disk]
dev=/dev/sda
clear=yes
parts=10,90

[swapspace]
dev=/dev/sda1

[localspace]
dev=/dev/sda2
fstype=xfs



Finally the following commands were executed:

chtab priority=7.1 policy.commands=getpartition policy.rule=allow

chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/ 
litefile.options=localdisk

chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/ 
litefile.options=localdisk



After a new genimage/packimage the local scratch does not appears to be made 
nor working.



Someone knows what to do next?



Thanks,

V.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Russ Auld 
If you're using netboot and local disk,  then isn't that "statelite"?Do the satellite instructions not work? On Nov 27, 2017 9:26 AM, Gilad Berman <gber...@lenovo.com> wrote:

All,

 
I would like to join this question –

Does even localdisk works with stateless? From the docs it seems that should be supported (because it is under stateless), however –

-
the instructions are taken from statelite and refer to statelite code (litefile)
-
The rc.localdisk code is under statelite
-
In the linuximage man – “Partitionfile - Only available for diskful osimages and statelite osimages(localdisk enabled)“

 
A very quick trial on my statless nodes results in nothing
J, it seems there is simply no reference to localdisk with stateless.

 
So, can someone please help clarify it?

 
** as always, there is a chance I missed something very basic and it should be working
J

 
THX in advance!
 





Gilad Berman
HPC Architect
Lenovo EMEA

+972-52-2554262
gberman@lenovo.com

 

Lenovo.com

Twitter |
Facebook |
Instagram |
Blogs |
Forums








 

 


From: Vinícius Ferrão [mailto:ferrao@versatushpc.com.br]

Sent: Wednesday, November 22, 2017 4:09 AM
To: xcat-user@lists.sourceforge.net
Subject: [xcat-user] Local scratch for stateless compute nodes


 
Hello,

 


I would like to enable swap and local /tmp on my stateless nodes, but after following the documentation on the following link nothing appears to work:


http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html


 


I’m aware that the documentation is for service nodes and not for compute nodes, but I was thinking the procedure would be similar.


 


At this point I’m with this settings on osimage:


[root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute
    exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist
    imagetype=linux
    osarch=x86_64
    osdistroname=centos7.4-x86_64
    osname=Linux
    osvers=centos7.4
    otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64


    partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile


    permission=755
    pkgdir=/install/centos7.4/x86_64
    pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist
    postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall
    profile="" />
    provmethod=netboot
    rootimgdir=/install/netboot/centos7.4/x86_64/compute
    synclists=/install/custom/netboot/compute.synclist


 


And the content of the partition file is the following:


cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
enable=yes
enablepart=yes

[disk]
dev=/dev/sda
clear=yes
parts=10,90

[swapspace]
dev=/dev/sda1

[localspace]
dev=/dev/sda2
fstype=xfs


 


Finally the following commands were executed:



chtab priority=7.1 policy.commands=getpartition policy.rule=allow


chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/ litefile.options=localdisk


chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/ litefile.options=localdisk



 


After a new genimage/packimage the local scratch does not appears to be made nor working.


 


Someone knows what to do next?


 


Thanks,


V.


 



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user

Re: [xcat-user] Local scratch for stateless compute nodes

2017-11-27 Thread Gilad Berman 
All,

I would like to join this question –
Does even localdisk works with stateless? From the docs it seems that should be 
supported (because it is under stateless), however –

- the instructions are taken from statelite and refer to statelite code 
(litefile)

- The rc.localdisk code is under statelite

- In the linuximage man – “Partitionfile - Only available for diskful 
osimages and statelite osimages(localdisk enabled)“

A very quick trial on my statless nodes results in nothing ☺, it seems there is 
simply no reference to localdisk with stateless.

So, can someone please help clarify it?

** as always, there is a chance I missed something very basic and it should be 
working ☺

THX in advance!

[http://lenovocentral.lenovo.com/marketing/branding/email_signature/images/gradient.gif]

Gilad Berman
HPC Architect
Lenovo EMEA

[Phone]+972-52-2554262
[Email]gber...@lenovo.com<mailto:gber...@lenovo.com>



Lenovo.com <http://www.lenovo.com/>
Twitter<http://twitter.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | 
Instagram<https://instagram.com/lenovo> | Blogs<http://blog.lenovo.com/> | 
Forums<http://forums.lenovo.com/>


[DCG-Hardware]




From: Vinícius Ferrão [mailto:fer...@versatushpc.com.br]
Sent: Wednesday, November 22, 2017 4:09 AM
To: xcat-user@lists.sourceforge.net
Subject: [xcat-user] Local scratch for stateless compute nodes

Hello,

I would like to enable swap and local /tmp on my stateless nodes, but after 
following the documentation on the following link nothing appears to work:
http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html

I’m aware that the documentation is for service nodes and not for compute 
nodes, but I was thinking the procedure would be similar.

At this point I’m with this settings on osimage:
[root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute
exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist
imagetype=linux
osarch=x86_64
osdistroname=centos7.4-x86_64
osname=Linux
osvers=centos7.4
otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64

partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
permission=755
pkgdir=/install/centos7.4/x86_64
pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist
postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall
profile=compute
provmethod=netboot
rootimgdir=/install/netboot/centos7.4/x86_64/compute
synclists=/install/custom/netboot/compute.synclist

And the content of the partition file is the following:
cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
enable=yes
enablepart=yes

[disk]
dev=/dev/sda
clear=yes
parts=10,90

[swapspace]
dev=/dev/sda1

[localspace]
dev=/dev/sda2
fstype=xfs

Finally the following commands were executed:
chtab priority=7.1 policy.commands=getpartition policy.rule=allow
chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/ 
litefile.options=localdisk
chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/ 
litefile.options=localdisk

After a new genimage/packimage the local scratch does not appears to be made 
nor working.

Someone knows what to do next?

Thanks,
V.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user

[xcat-user] Local scratch for stateless compute nodes

2017-11-21 Thread Vinícius Ferrão 
Hello,

I would like to enable swap and local /tmp on my stateless nodes, but after 
following the documentation on the following link nothing appears to work:
http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html
 


I’m aware that the documentation is for service nodes and not for compute 
nodes, but I was thinking the procedure would be similar.

At this point I’m with this settings on osimage:
[root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute
exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist
imagetype=linux
osarch=x86_64
osdistroname=centos7.4-x86_64
osname=Linux
osvers=centos7.4
otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64

partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
permission=755
pkgdir=/install/centos7.4/x86_64
pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist
postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall
profile=compute
provmethod=netboot
rootimgdir=/install/netboot/centos7.4/x86_64/compute
synclists=/install/custom/netboot/compute.synclist

And the content of the partition file is the following:
cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile
enable=yes
enablepart=yes

[disk]
dev=/dev/sda
clear=yes
parts=10,90

[swapspace]
dev=/dev/sda1

[localspace]
dev=/dev/sda2
fstype=xfs

Finally the following commands were executed:
chtab priority=7.1 policy.commands=getpartition policy.rule=allow
chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/ 
litefile.options=localdisk
chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/ 
litefile.options=localdisk

After a new genimage/packimage the local scratch does not appears to be made 
nor working.

Someone knows what to do next?

Thanks,
V.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user