date:20191007

[Gluster-users] major problems with Samba - 0-md-cache: xattr key trusted.afr.GROUP_WORK-client-3 doesn't satisfy caching requirements & Buffer overflow of a buffer whose size limit is 5. About to flu

2019-10-07 Thread lejeczek

hi everyone,

I do not suppose it is a matter of tweaking Gluster(or Samba) as these
problems I think started appearing after upgrade of Samba
(samba-4.9.1-6.el7.x86_64) and/or Gluster(glusterfs-6.5-1.el7.x86_64)

How it manifests it that Samba operations that a user performs are
incredibly slow. To create a new file(via right-click in Windows) takes
more then dozen of seconds, sometimes fails, the same with renaming
files, etc.

At the same time I samba's part of glusterfs log I see:

...

[2019-10-07 10:42:02.252880] D [logging.c:2006:_gf_msg_internal]
0-logging-infra: Buffer overflow of a buffer whose size limit is 5.
About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:678:is_mdc_key_satisfied]
0-md-cache: xattr key glusterfs.parent-entrylk doesn't satisfy caching
requirements" repeated 4 times between [2019-10-07 10:42:02.252788] and
[2019-10-07 10:42:02.252874]
[2019-10-07 10:42:02.252878] T [MSGID: 0]
[md-cache.c:678:is_mdc_key_satisfied] 0-md-cache: xattr key
trusted.glusterfs.dht doesn't satisfy caching requirements
[2019-10-07 10:42:02.252981] D [logging.c:2006:_gf_msg_internal]
0-logging-infra: Buffer overflow of a buffer whose size limit is 5.
About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:678:is_mdc_key_satisfied]
0-md-cache: xattr key trusted.glusterfs.dht doesn't satisfy caching
requirements" repeated 4 times between [2019-10-07 10:42:02.252878] and
[2019-10-07 10:42:02.252974]
[2019-10-07 10:42:02.252978] T [MSGID: 0]
[md-cache.c:678:is_mdc_key_satisfied] 0-md-cache: xattr key
trusted.afr.GROUP_WORK-client-3 doesn't satisfy caching requirements
[2019-10-07 10:42:02.253059] D [logging.c:2006:_gf_msg_internal]
0-logging-infra: Buffer overflow of a buffer whose size limit is 5.
About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:678:is_mdc_key_satisfied]
0-md-cache: xattr key trusted.afr.GROUP_WORK-client-3 doesn't satisfy
caching requirements" repeated 4 times between [2019-10-07
10:42:02.252978] and [2019-10-07 10:42:02.253052]
[2019-10-07 10:42:02.253056] T [MSGID: 0]
[md-cache.c:678:is_mdc_key_satisfied] 0-md-cache: xattr key link-count
doesn't satisfy caching requirements
[2019-10-07 10:42:02.253140] D [logging.c:2006:_gf_msg_internal]
0-logging-infra: Buffer overflow of a buffer whose size limit is 5.
About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:678:is_mdc_key_satisfied]
0-md-cache: xattr key link-count doesn't satisfy caching requirements"
repeated 4 times between [2019-10-07 10:42:02.253056] and [2019-10-07
10:42:02.253131]
[2019-10-07 10:42:02.253138] T [MSGID: 0]
[md-cache.c:792:mdc_inode_xatt_set] 0-md-cache: xatt cache set for
(----0001) time:1570444922

I'm going to try to tweak vol's some opts/features but I'd appreciate
suggestions & thoughts.

Many thanks, L.



pEpkey.asc
Description: application/pgp-keys


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] socket.so: undefined symbol: xlator_api - bd.so: cannot open shared object file & crypt.so: cannot open shared object file: No such file or directory

2019-10-07 Thread lejeczek

hi everyone

I'm been running glusterfs 6 for a while and either I did not notice or
it just started to pop:

[2019-10-07 09:17:37.071409] I [run.c:242:runner_log]
(-->/usr/lib64/glusterfs/6.5/xlator/mgmt/glusterd.so(+0xe8faa)
[0x7fd6204d3faa]
-->/usr/lib64/glusterfs/6.5/xlator/mgmt/glusterd.so(+0xe8a75)
[0x7fd6204d3a75] -->/lib64/libglusterfs.so.0(runner_log+0x115)
[0x7fd62c360495] ) 0-management: Ran script:
/var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh
--volname=IT-RELATED --first=no --version=1 --volume-op=start
--gd-workdir=/var/lib/glusterd
[2019-10-07 09:17:37.099416] I [run.c:242:runner_log]
(-->/usr/lib64/glusterfs/6.5/xlator/mgmt/glusterd.so(+0xe8faa)
[0x7fd6204d3faa]
-->/usr/lib64/glusterfs/6.5/xlator/mgmt/glusterd.so(+0xe8a75)
[0x7fd6204d3a75] -->/lib64/libglusterfs.so.0(runner_log+0x115)
[0x7fd62c360495] ) 0-management: Ran script:
/var/lib/glusterd/hooks/1/start/post/S30samba-start.sh
--volname=IT-RELATED --first=no --version=1 --volume-op=start
--gd-workdir=/var/lib/glusterd
[2019-10-07 09:42:26.314045] W [MSGID: 101095]
[xlator.c:210:xlator_volopt_dynload] 0-xlator:
/usr/lib64/glusterfs/6.5/xlator/encryption/crypt.so: cannot open shared
object file: No such file or directory
[2019-10-07 09:42:26.328413] E [MSGID: 101097]
[xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api)
missing: /usr/lib64/glusterfs/6.5/rpc-transport/socket.so: undefined
symbol: xlator_api
[2019-10-07 09:42:26.330640] W [MSGID: 101095]
[xlator.c:210:xlator_volopt_dynload] 0-xlator:
/usr/lib64/glusterfs/6.5/xlator/nfs/server.so: cannot open shared object
file: No such file or directory
[2019-10-07 09:42:26.348399] W [MSGID: 101095]
[xlator.c:210:xlator_volopt_dynload] 0-xlator:
/usr/lib64/glusterfs/6.5/xlator/storage/bd.so: cannot open shared object
file: No such file or directory
The message "W [MSGID: 101095] [xlator.c:210:xlator_volopt_dynload]
0-xlator: /usr/lib64/glusterfs/6.5/xlator/encryption/crypt.so: cannot
open shared object file: No such file or directory" repeated 2 times
between [2019-10-07 09:42:26.314045] and [2019-10-07 09:42:26.314307]
The message "E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload]
0-xlator: dlsym(xlator_api) missing:
/usr/lib64/glusterfs/6.5/rpc-transport/socket.so: undefined symbol:
xlator_api" repeated 7 times between [2019-10-07 09:42:26.328413] and
[2019-10-07 09:42:26.328590]
The message "W [MSGID: 101095] [xlator.c:210:xlator_volopt_dynload]
0-xlator: /usr/lib64/glusterfs/6.5/xlator/nfs/server.so: cannot open
shared object file: No such file or directory" repeated 30 times between
[2019-10-07 09:42:26.330640] and [2019-10-07 09:42:26.331499]

These are not available from gluster's yum repositories.

Any suggestions on how to troubleshoot & solve this are very much
appreciated.

many thanks, L



pEpkey.asc
Description: application/pgp-keys


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Reg Performance issue in GlusterFS

2019-10-07 Thread Krutika Dhananjay

Hi Pratik,

What's the version of gluster you're running?

Also, what would help is the volume profile output. Here's what you should
do to capture it:
# gluster volume profile  start
# run_your_workload
# gluster volume profile  info > brick-profile.out

And attach brick-profile.out here.

-Krutika

On Mon, Oct 7, 2019 at 11:09 AM Satheesaran Sundaramoorthi <
sasun...@redhat.com> wrote:

> On Mon, Oct 7, 2019 at 1:24 AM Soumya Koduri  wrote:
>
>> Hi Pratik,
>>
>> Offhand I do not see any issue with the configuration. But I think for
>> VM images store, using gfapi may give better performance than compared
>> to fuse. CC'in Kritika and Gobinda who have been working on this
>> use-case and may be able to guide you.
>>
>> Thanks,
>> Soumya
>>
>> On 10/5/19 11:25 AM, Pratik Chandrakar wrote:
>> > Hello Soumya,
>> >
>> > This is Pratik from India. I am writing this mail because I am facing
>> performance issue in my cluster and searched a lot in net for the tuning
>> but not succeeded. It would be great if you can suggest me whether I should
>> stick with glusterfs or move to any other technology. Currently I am using
>> Glusterfs with fuse on CentOS for Storing images of Virtual Machine in
>> CloudStack Setup. Majority of workload is of SQL Server & MariaDB database
>> server, and some Web Servers. The issue is of slow booting and slow UI
>> response of VMs and also lot of time outs in SQL server database even on
>> small databases. I have dedicated 10G network for storage in my setup.
>> >
>> > Request you to please guide me, whether I am have miss-configured the
>> cluster or need to change the storage layer.
>> >
>> > Below is the configuration for your reference...
>> >
>> > *Volume Name: vmstore5152-v2*
>> > *Type: Replicate*
>> > *Volume ID: aa27a2cb-c0f5-41b9-a50f-fdce4d4d8358*
>> > *Status: Started*
>> > *Snapshot Count: 0*
>> > *Number of Bricks: 1 x (2 + 1) = 3*
>> > *Transport-type: tcp*
>> > *Bricks:*
>> > *Brick1: storagenode51:/datav2/brick51-v2/brick*
>> > *Brick2: storagenode52:/datav2/brick52-v2/brick*
>> > *Brick3: indphyserver2:/arbitator/arbrick5152-v2/brick (arbiter)*
>> > *Options Reconfigured:*
>> > *cluster.choose-local: off*
>> > *user.cifs: off*
>> > *features.shard: on*
>> > *cluster.shd-wait-qlength: 1*
>> > *cluster.shd-max-threads: 8*
>> > *cluster.locking-scheme: granular*
>> > *cluster.data-self-heal-algorithm: full*
>> > *cluster.server-quorum-type: server*
>> > *cluster.quorum-type: auto*
>> > *cluster.eager-lock: enable*
>> > *network.remote-dio: enable*
>>
>
> Hi Krutika,
>
> Do you think turning off remote-dio and enabling strict-o-direct, will
> improve performance ?
>
> @Sahina, @Gobinda, are you aware of performance optimization for the
> DB workload in the VMs ?
>
> -- Satheesaran
>
>> > *performance.low-prio-threads: 32*
>> > *performance.io-cache: off*
>> > *performance.read-ahead: off*
>> > *performance.quick-read: off*
>> > *storage.owner-gid: 107*
>> > *storage.owner-uid: 107*
>> > *cluster.lookup-optimize: on*
>> > *client.event-threads: 4 *
>> > *transport.address-family: inet*
>> > *nfs.disable: on*
>> > *performance.client-io-threads: on*
>> >
>> >
>> > --
>> > प्रतीक चंद्राकर | Pratik Chandrakar
>> > वैज्ञानिक - सी | Scientist-C
>> > एन.आई.सी - छत्तीसगढ़ राज्य केंद्र | NIC - Chhattisgarh State Centre
>> > हॉल क्र. एडी2-14 , मंत्रालय | Hall no.-AD2-14, Mantralaya
>> > महानदी भवन | Mahanadi Bhavan
>> > नवा रायपुर अटल नगर | Nava Raipur Atal Nagar
>> >
>> >
>> > 
>> >
>> > *Disclaimer:*
>> >
>> > This e-mail and its attachments may contain official Indian Government
>> > information. If you are not the intended recipient, please notify the
>> > sender immediately and delete this e-mail. Any dissemination or use of
>> > this information by a person other than the intended recipient is
>> > unauthorized. The responsibility lies with the recipient to check this
>> > email and any attachment for the presence of viruses.
>> >
>> 
>>
>> Community Meeting Calendar:
>>
>> APAC Schedule -
>> Every 2nd and 4th Tuesday at 11:30 AM IST
>> Bridge: https://bluejeans.com/118564314
>>
>> NA/EMEA Schedule -
>> Every 1st and 3rd Tuesday at 01:00 PM EDT
>> Bridge: https://bluejeans.com/118564314
>>
>> Gluster-users mailing list
>> Gluster-users@gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] major problems with Samba - 0-md-cache: xattr key trusted.afr.GROUP_WORK-client-3 doesn't satisfy caching requirements & Buffer overflow of a buffer whose size limit is 5. About to flu

[Gluster-users] socket.so: undefined symbol: xlator_api - bd.so: cannot open shared object file & crypt.so: cannot open shared object file: No such file or directory

Re: [Gluster-users] Reg Performance issue in GlusterFS

3 matches

Site Navigation

Mail list logo

Footer information