Hi Luis,

GPFS is 4.2.3 (gpfs.base-4.2.3-6.x86_64), All servers (8 in front of a DDN
SFA12K) are RHEL 7.3 (stock DDN setup). All 47 clients are CentOS 7.4.

GPFS mount:
# mount | grep gpfs
gsfs0 on /srv/gsfs0 type gpfs (rw,relatime)


NFS mount:
mount | grep $HOME
10.210.15.57:/srv/gsfs0/home/griznog on /home/griznog type nfs
(rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.210.15.57,mountvers=3,mountport=20048,mountproto=tcp,local_lock=none,addr=10.210.15.57)

Example script:

#!/bin/bash
cat pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt >
/srv/gsfs0/projects/pipetest.tmp.txt
grep L1 /srv/gsfs0/projects/pipetest.tmp.txt |  wc -l

cat  pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt >
$HOME/pipetest.tmp.txt
grep L1 $HOME/pipetest.tmp.txt | wc -l


Example output:

# ./pipetest.sh
1
1836

#  ls -aln /srv/gsfs0/projects/pipetest.tmp.txt $HOME/pipetest.tmp.txt
-rw-r--r-- 1 39073 3953 530721 Feb 14 06:10 /home/griznog/pipetest.tmp.txt
-rw-r--r-- 1 39073 3001 530721 Feb 14 06:10
/srv/gsfs0/projects/pipetest.tmp.txt

We can "fix" the user case that exposed this by not using a temp file or
inserting a sleep, but I'd still like to know why GPFS is behaving this way
and make it stop.


mmlsconfig below.

Thanks,

jbh

mmlsconfig
Configuration data for cluster SCG-GS.scg-gs0:
----------------------------------------------
clusterName SCG-GS.scg-gs0
clusterId 8456032987852400706
dmapiFileHandleSize 32
maxblocksize 4096K
cnfsSharedRoot /srv/gsfs0/GS-NFS
cnfsMountdPort 597
socketMaxListenConnections 1024
fileHeatPeriodMinutes 1440
fileHeatLossPercent 1
pingPeriod 5
minMissedPingTimeout 30
afmHashVersion 1
minReleaseLevel 4.2.0.1
[scg-gs0,scg-gs1,scg-gs2,scg-gs3,scg-gs4,scg-gs5,scg-gs6,scg-gs7]
nsdbufspace 70
[common]
healthCheckInterval 20
maxStatCache 512
maxFilesToCache 50000
nsdMinWorkerThreads 512
nsdMaxWorkerThreads 1024
deadlockDetectionThreshold 0
deadlockOverloadThreshold 0
prefetchThreads 288
worker1Threads 320
maxMBpS 2000
[scg-gs0,scg-gs1,scg-gs2,scg-gs3,scg-gs4,scg-gs5,scg-gs6,scg-gs7]
maxMBpS 24000
[common]
atimeDeferredSeconds 300
pitWorkerThreadsPerNode 2
cipherList AUTHONLY
pagepool 1G
[scg-gs0,scg-gs1,scg-gs2,scg-gs3,scg-gs4,scg-gs5,scg-gs6,scg-gs7]
pagepool 8G
[common]
cnfsNFSDprocs 256
nfsPrefetchStrategy 1
autoload yes
adminMode central

File systems in cluster SCG-GS.scg-gs0:
---------------------------------------
/dev/gsfs0




On Tue, Feb 13, 2018 at 10:53 PM, Luis Bolinches <luis.bolinc...@fi.ibm.com>
wrote:

> Sorry
>
> With cat
>
> [root@specscale01 IBM_REPO]# cp test a
> [root@specscale01 IBM_REPO]# cat a a a a > test && grep ATAG test | wc -l
> && sleep 4 && grep ATAG test | wc -l
> 0
> 0
> --
> Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations
> Luis Bolinches
> Consultant IT Specialist
> Mobile Phone: +358503112585 <+358%2050%203112585>
> https://www.youracclaim.com/user/luis-bolinches
>
> "If you always give you will always have" --  Anonymous
>
>
>
> From:        Luis Bolinches <luis.bolinc...@fi.ibm.com>
> To:        gpfsug main discussion list <gpfsug-discuss@spectrumscale.org>
> Date:        14/02/2018 08:49
> Subject:        Re: [gpfsug-discuss] Odd behavior with cat followed by
> grep.
> Sent by:        gpfsug-discuss-boun...@spectrumscale.org
> ------------------------------
>
>
>
> Hi
>
> This seems to be setup specific
>
> Care to explain a bit more of the setup. Number of nodes GPFS versions,
> number of FS, Networking, running from admin node, server / client, number
> of NSD, separated meta and data, etc?
>
> I got interested and run a quick test on a gpfs far from powerful cluster
> of 3 nodes on KVM
>
> [root@specscale01 IBM_REPO]# echo "a a a a a a a a a a" > test && grep
> ATAG test | wc -l && sleep 4 && grep ATAG test | wc -l
> 0
> 0
> [root@specscale01 IBM_REPO]#
>
>
> --
> Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations
> Luis Bolinches
> Consultant IT Specialist
> Mobile Phone: +358503112585 <+358%2050%203112585>
> *https://www.youracclaim.com/user/luis-bolinches*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youracclaim.com_user_luis-2Dbolinches&d=DwMFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=HrR-mBJ82ubcbtBin7NGVl2VenLj726Fcah6-3XFvDs&s=hGQKSxR54bjW2i9KVDRpflHxrMcJhY269FZkQGWHtLI&e=>
>
> "If you always give you will always have" --  Anonymous
>
>
>
> From:        John Hanks <griz...@gmail.com>
> To:        gpfsug-discuss <gpfsug-discuss@spectrumscale.org>
> Date:        14/02/2018 07:33
> Subject:        [gpfsug-discuss] Odd behavior with cat followed by grep.
> Sent by:        gpfsug-discuss-boun...@spectrumscale.org
> ------------------------------
>
>
>
> Hi,
>
> We have a GPFS filesystem mounted on CentOS 7.4 as type gpfs, pretty
> straightforward run of the mill stuff. But are seeing this odd behavior. If
> I do this in a shell script, given a file called "a"
>
> cat a a a a a a a a a a > /path/to/gpfs/mount/test
> grep ATAG /path/to/gpfs/mount/test | wc -l
> sleep 4
> grep ATAG /path/to/gpfs/mount/test | wc -l
>
> The first grep | wc -l returns 1, because grep outputs  "Binary file
> /path/to/gpfs/mount/test matches"
>
> The second grep | wc -l returns the correct count of ATAG in the file.
>
> Why does it take 4 seconds (3 isn't enough) for that file to be properly
> recognized as a text file and/or why is it seen as a binary file in the
> first place since a is a plain text file?
>
> Note that I have the same filesystem mounted via NFS and over an NFS mount
> it works as expected.
>
> Any illumination is appreciated,
>
> jbh_______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
>
> *https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=ut35qIIMxjZMX3obFJ2xtUMng4MtGtKz4YHxpkgQbak&s=cNt66GjRD6rVhq7nGcvT76l-0_u2C3UTz9SfwzHf1xw&e=*
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=ut35qIIMxjZMX3obFJ2xtUMng4MtGtKz4YHxpkgQbak&s=cNt66GjRD6rVhq7nGcvT76l-0_u2C3UTz9SfwzHf1xw&e=>
>
>
>
>
> Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
> Oy IBM Finland Ab
> PL 265, 00101 Helsinki, Finland
> Business ID, Y-tunnus: 0195876-3
> Registered in Finland_______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.
> org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=
> 1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=HrR-
> mBJ82ubcbtBin7NGVl2VenLj726Fcah6-3XFvDs&s=d5YiAyXz4el9bF0zjGL9gVjnTfbX4z
> -qelZodxRqlz0&e=
>
>
>
>
>
> Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
> Oy IBM Finland Ab
> PL 265, 00101 Helsinki, Finland
> Business ID, Y-tunnus: 0195876-3
> Registered in Finland
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Reply via email to