Re: [Gluster-devel] Replacing a host using gdeploy

[Peter Portante]

Thanks sac, I'll take a look at this.

-peter

On Mon, Jun 13, 2016 at 1:35 PM, Sachidananda URS  wrote:

> Hi,
>
> With the replace node operation being very tedious and elaborate process,
> it is
> easy to commit mistakes. And it is much tedious process to roll back in
> case of
> mistakes.
>
> This is an effort to automate the entire process, the script
> replace_node.sh is
> a wrapper script around gdeploy configuration file. It takes three
> parameters:
> `Existing node name', `new node name', and `volume name'. The script
> generates
> a configuration file, and calls gdeploy with the configuration file as an
> argument. This handles clusters with multiple hosts and just two hosts.
>
> USAGE:
>
> $ ./replace_node.sh NODE-1 NODE-2 VOLUMENAME
>
>
> ***
> Node NODE-2 will be replaced with configuration from NODE-1
> It is very important to give the node names right. Review before
> continuing.
>
> ***
>
> Continue(y/N):
>
>
> -
>
> When continued NODE-2 will be added into the cluster and self-heal is
> triggered.
>
> Prerequisites:
> ===
> 1. Password less ssh from node-1 to node-2
> 2. Passwordless ssh from the laptop to node-1, node-2
> 3. Place the replace_prep.sh under /usr/share/ansible/gdeploy/scripts/
>
> Configuration generation and additional script generation and invocation is
> managed by gdeploy.
>
> The step-3 in Prerequisite will be handled in the rpm once the code is
> merged
> upstream and rpm generated. I'll post on -users list once I build rpms.
>
> I've done some initial tests and things work fine. Any suggestions and
> contributions are welcome.
>
> -sac
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] New Defects reported by Coverity Scan for gluster/glusterfs

[scan-admin]

Hi,

Please find the latest report on new defect(s) introduced to gluster/glusterfs 
found with Coverity Scan.

125 new defect(s) introduced to gluster/glusterfs found with Coverity Scan.
95 defect(s), reported by Coverity Scan earlier, were marked fixed in the 
recent build analyzed by Coverity Scan.

New defect(s) Reported-by: Coverity Scan
Showing 20 of 125 defect(s)


** CID 1356589:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/features/locks/src/common.c: 
492 in new_posix_lock()



*** CID 1356589:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/features/locks/src/common.c: 
492 in new_posix_lock()
486 
487 lock->blocking  = blocking;
488 
489 INIT_LIST_HEAD (>list);
490 
491 out:
>>> CID 1356589:  Memory - illegal accesses  (USE_AFTER_FREE)
>>> Using freed pointer "lock".
492 return lock;
493 }
494 
495 
496 /* Delete a lock from the inode's lock list */
497 void

** CID 1356588:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/protocol/client/src/client-rpc-fops.c:
 6164 in client3_3_compound()



*** CID 1356588:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/protocol/client/src/client-rpc-fops.c:
 6164 in client3_3_compound()
6158 goto unwind;
6159 }
6160 
6161 iobref_add (rsphdr_iobref, rsphdr_iobuf);
6162 iobuf_unref (rsphdr_iobuf);
6163 rsphdr = [0];
>>> CID 1356588:  Memory - illegal accesses  (USE_AFTER_FREE)
>>> Dereferencing freed pointer "rsphdr_iobuf".
6164 rsphdr->iov_base = iobuf_ptr (rsphdr_iobuf);
6165 rsphdr->iov_len = iobuf_pagesize (rsphdr_iobuf);
6166 rsphdr_count = 1;
6167 local->iobref = rsp_iobref;
6168 rsphdr_iobuf = NULL;
6169 rsphdr_iobref = NULL;

** CID 1356587:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/protocol/client/src/client-helpers.c:
 1264 in client_handle_fop_requirements()



*** CID 1356587:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/protocol/client/src/client-helpers.c:
 1264 in client_handle_fop_requirements()
1258 iobuf_unref (rsp_iobuf);
1259 
1260 if (*rsp_count + 1 >= MAX_IOVEC) {
1261 op_errno = ENOMEM;
1262 goto out;
1263 }
>>> CID 1356587:  Memory - illegal accesses  (USE_AFTER_FREE)
>>> Dereferencing freed pointer "rsp_iobuf".
1264 rsp_vector[*rsp_count].iov_base = iobuf_ptr 
(rsp_iobuf);
1265 rsp_vector[*rsp_count].iov_len = iobuf_pagesize 
(rsp_iobuf);
1266 rsp_iobuf = NULL;
1267 if (args->size > rsp_vector[*rsp_count].iov_len) {
1268 gf_msg (this->name, GF_LOG_WARNING, ENOMEM,
1269 PC_MSG_NO_MEMORY,

** CID 1356586:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/features/locks/src/posix.c: 
2857 in new_meta_lock()



*** CID 1356586:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/features/locks/src/posix.c: 
2857 in new_meta_lock()
2851 GF_FREE (lock);
2852 goto out;
2853 }
2854 
2855 __pl_metalk_ref (lock);
2856 out:
>>> CID 1356586:  Memory - illegal accesses  (USE_AFTER_FREE)
>>> Using freed pointer "lock".
2857 return lock;
2858 }
2859 
2860 int
2861 pl_insert_metalk (pl_inode_t *pl_inode, pl_ctx_t *ctx, pl_meta_lock_t 
*lock)
2862 {

** CID 1356585:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/features/locks/src/posix.c: 
3796 in gf_lkmig_info_to_posix_lock()



*** CID 1356585:  Memory - illegal accesses  (USE_AFTER_FREE)
/home/vijay/workspace/glusterfs/glusterfs/xlators/features/locks/src/posix.c: 
3796 in gf_lkmig_info_to_posix_lock()
3790 lock->client_pid = lmi->flock.l_pid;
3791 lock->owner  = lmi->flock.l_owner;
3792 
3793 INIT_LIST_HEAD (>list);

Re: [Gluster-devel] Brick multiplexing approaches

[Jeff Darcy]

> There is *currently* no graph switch on the bricks (as I understand it).
> Configuration changes, yes, but no graph switch as the xlator pipeline
> is fixed, if that changes the bricks need to be restarted. Others can
> correct me if I am wrong.
> 
> Noting the above here, as it may not be that big a deal. Also noting the
> 'currently' in bold, as the future could mean something different.

I'd say it certainly will.  Adding bricks to JBR or DHT2, which have
server-side components sensitive to such changes, practically require it.

> The sub-graph model seems the best for certain other things, like
> preserving the inode and other tables as is by the master xlator. It
> does introduce the onus of keeping the inodes (and fd's) current on the
> xlators though (watering this down to sub-graph at the sub-brick level
> is possible, but that would be 'watering' this concept down). This needs
> some more thought, but I do like the direction.

The inode table's kind of a weird beast.  In a multiplexing model it
doesn't really belong on the server translator, but it belongs even
less on the next translator down because the server translator's the
one that uses it.  Really, the server translator is such a special
case in so many ways that I'm not even sure it's really a translator
at all.  What we kind of need is a first-class abstraction of a
sub-graph or translator stack that can be attached to a server
translator.  Such an object would "contain" not just the graph but
also the inode table, auth info, and probably other stuff (e.g. QoS
info) as well.  All this refactoring is inconvenient, but I don't
see any way around it if we want to support much larger numbers of
volumes on relatively small numbers of nodes (as e.g. container
workloads require).

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Brick multiplexing approaches

[Shyam]

On 06/13/2016 02:19 PM, Jeff Darcy wrote:

"Brick multiplexing" is a new feature, tentatively part of 4.0, that
allows multiple bricks to be served from a single glusterfsd process.
This promises to give us many benefits over the current "process per
brick" approach.

 * Lower total memory use, by having only one copy of various global
   structures instead of one per brick/process.

 * Less CPU contention.  Every glusterfsd process involves several
   threads.  If there are more total threads than physical cores, or if
   those cores are also needed for other work on the same system, we'll
   thrash pretty badly.  As with memory use, managing each thread type
   as a single pool (instead of one pool per brick/process) will help.

 * Fewer ports.  In the extreme case, we need only have one process and
   one port per node.  This avoids port exhaustion for high node/brick
   counts, and can also be more firewall-friendly.

 * Better coordination between bricks e.g. to implement QoS policies.

In short, our current infrastructure just isn't going to let us keep up
with various trends - higher node counts, containers, hyperconvergence,
even erasure coding.  The question is: how do we do it?  There are two
basic models.

 * In the "multiple graph" model, we have multiple separate graphs
   (volfiles) in a single process.  This allows them to share logging
   data and threads, polling threads, and many other resources -
   everything anchored at a glusterfs_ctx_t.  It does *not* allow them
   to share ports, or anything else anchored by a protocol/server
   translator instance.

 * In the "single graph" model, we have multiple graphs joined together
   at the single protocol/server translator.  This allows more things to
   be shared, including ports, but does introduce some new problems.
   For one thing, it doesn't work if the bricks have different transport
   characteristics (e.g. TLS vs. non-TLS).  For another, it raises the
   possibility of a configuration change for one brick causing a graph
   switch that affects all bricks in the process (even if they belong to
   separate volumes).


There is *currently* no graph switch on the bricks (as I understand it). 
Configuration changes, yes, but no graph switch as the xlator pipeline 
is fixed, if that changes the bricks need to be restarted. Others can 
correct me if I am wrong.


Noting the above here, as it may not be that big a deal. Also noting the 
'currently' in bold, as the future could mean something different.




I'd prefer to implement the single-graph model, because both the port
conservation/exhaustion and QoS-coordination issues are important.
However, that means solving some of the additional problems.  In
particular...

 * Most of the incompatible-transport issues can be solved by moving
   various things from the server translator's "private" structure
   (server_conf_t) into per-connection or per-tenant structures.  I've
   already done something similar with the inode table for subvolume
   mounts (#13659 in Gerrit) and it's a pain but it's feasible.  We
   might also (eventually) need consider implementing parts of the
   multi-graph model as well to host bricks in the same process even
   when their transports are incompatible.

 * For the graph-switch problem, we'll need to introduce some idea of
   sub-graphs or related graphs, so that we can compare and switch only
   the part relevant to a single brick.  I'd actually like to avoid this
   entirely until we get to GlusterD 2.0, but I'm not sure if we'll be
   able to get away with that.


The sub-graph model seems the best for certain other things, like 
preserving the inode and other tables as is by the master xlator. It 
does introduce the onus of keeping the inodes (and fd's) current on the 
xlators though (watering this down to sub-graph at the sub-brick level 
is possible, but that would be 'watering' this concept down). This needs 
some more thought, but I do like the direction.




Suggestions, warnings, or other thoughts are welcome.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Minutes : Gluster Community Bug Triage meeting (Today)

[Saravanakumar Arumugam]

 Hi,


 Please find minutes of June 14 Bug Triage meeting.



 Meeting summary

1.
1. agenda: https://public.pad.fsfe.org/p/gluster-bug-triage
   (Saravanakmr
   
,
   12:00:37)

2. Roll call (Saravanakmr
   
,
   12:00:49)
3. kkeithley Saravanakmr will set up Coverity, clang, etc on public
   facing machine and run it regularly (Saravanakmr
   
,
   12:04:59)
1. 
http://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2016-06-13-e0b057cf/cov-errors.txt
   looks pretty full to me (kkeithley
   
,
   12:08:39)

4. ndevos need to decide on how to provide/use debug builds
   (Saravanakmr
   
,
   12:11:54)
1. ACTION: ndevos need to decide on how to provide/use debug builds
   (Saravanakmr
   
,
   12:12:19)
2. ACTION: ndevos to propose some test-cases for minimal libgfapi
   test (Saravanakmr
   
,
   12:12:42)

5. Manikandan and gem to followup with kshlm/misc/nigelb to get access
   to gluster-infra (Saravanakmr
   
,
   12:12:58)
1. ACTION: Manikandan and gem to followup with kshlm/misc/nigelb to
   get access to gluster-infra (Saravanakmr
   
,
   12:14:21)
2. ACTION: Manikandan will host bug triage meeting on June 21st
   2016 (Saravanakmr
   
,
   12:14:30)
3. ACTION: ndevos will host bug triage meeting on June 28th 2016
   (Saravanakmr
   
,
   12:14:38)

6. Group Triage (Saravanakmr
   
,
   12:15:05)
1. you can fine the bugs to triage here in
   https://public.pad.fsfe.org/p/gluster-bugs-to-triage
   (Saravanakmr
   
,
   12:15:12)
2. http://www.gluster.org/community/documentation/index.php/Bug_triage
   (Saravanakmr
   
,
   12:15:17)
3. https://public.pad.fsfe.org/p/gluster-bugs-to-triage
   (Saravanakmr
   
,
   12:15:38)

7. Open Floor (Saravanakmr
   
,
   12:23:40)


Meeting ended at 12:29:29 UTC (full logs 
).



 Action items

1. ndevos need to decide on how to provide/use debug builds
2. ndevos to propose some test-cases for minimal libgfapi test
3. Manikandan and gem to followup with kshlm/misc/nigelb to get access
   to gluster-infra
4. Manikandan will host bug triage meeting on June 21st 2016
5. ndevos will host bug triage meeting on June 28th 2016



 Action items, by person

1. ndevos
1. ndevos need to decide on how to provide/use debug builds
2. ndevos to propose some test-cases for minimal libgfapi test
3. ndevos will host bug triage meeting on June 28th 2016
2. UNASSIGNED
1. Manikandan and gem to followup with kshlm/misc/nigelb to get
   access to gluster-infra
2. Manikandan will host bug triage meeting on June 21st 2016


 People present (lines said)

1. Saravanakmr (45)
2. kkeithley (20)
3. ndevos (10)
4. hgowtham (3)
5. zodbot (3)
6. skoduri (2)
7. jiffin (1)
8. partner (1)


Thanks,
Saravana

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] REMINDER: Gluster Community Bug Triage meeting at 12:00 UTC

[Saravanakumar Arumugam]

Hi,


This meeting is scheduled for anyone, who is interested in learning more
about, or assisting with the Bug Triage.

Meeting details:
- location: #gluster-meeting on Freenode IRC
(https://webchat.freenode.net/?channels=gluster-meeting  )
- date: every Tuesday
- time: 12:00 UTC
  (in your terminal, run: date -d "12:00 UTC")
- agenda:https://public.pad.fsfe.org/p/gluster-bug-triage

Currently the following items are listed:
* Roll Call
* Status of last weeks action items
* Group Triage
* Open Floor

The last two topics have space for additions. If you have a suitable bug
or topic to discuss, please add it to the agenda.

Appreciate your participation.


Thanks,
Saravana

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel