Re: [Gluster-devel] Events API new Port requirement

2016-09-08 Thread Sankarshan Mukhopadhyay 
On Fri, Sep 9, 2016 at 3:55 AM, Niels de Vos  wrote:
> On Thu, Sep 08, 2016 at 10:03:03PM +0530, Sankarshan Mukhopadhyay wrote:
>> On Sun, Aug 28, 2016 at 2:13 PM, Niels de Vos  wrote:
>> > This definitely has my preference too. I've always wanted to try to
>> > register port 24007/8, and maybe the time has come to look into it.
>>
>> Has someone within the project previously undertaken the process of
>> requesting the IANA for assignment of a new service name and port
>> number value?
>
> Not that I know. I think Amar had an interest several years ago, but
> I've never seen any requests or results.
>
> There are some Red Hat colleagues that might have experience with this.
> I can ask and see if they can provide any guidance. But if there is
> someone very interested and eager to request these ports, please go
> ahead (and CC me on the communications).
>

If you can reach out and obtain the necessary information, it would be
a good first step.


-- 
sankarshan mukhopadhyay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Events API new Port requirement

2016-09-08 Thread Niels de Vos 
On Thu, Sep 08, 2016 at 10:03:03PM +0530, Sankarshan Mukhopadhyay wrote:
> On Sun, Aug 28, 2016 at 2:13 PM, Niels de Vos  wrote:
> > This definitely has my preference too. I've always wanted to try to
> > register port 24007/8, and maybe the time has come to look into it.
> 
> Has someone within the project previously undertaken the process of
> requesting the IANA for assignment of a new service name and port
> number value?

Not that I know. I think Amar had an interest several years ago, but
I've never seen any requests or results.

There are some Red Hat colleagues that might have experience with this.
I can ask and see if they can provide any guidance. But if there is
someone very interested and eager to request these ports, please go
ahead (and CC me on the communications).

Niels


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Libunwind

2016-09-08 Thread Kaleb S. KEITHLEY 

On 09/08/2016 10:33 AM, Vijay Bellur wrote:

On Thu, Sep 8, 2016 at 9:07 AM, Jeff Darcy  wrote:

In a few places in our code (e.g. gf_log_callingfn) we use the "backtrace" and 
"backtrace_symbols" functions from libc to log stack traces.  Unfortunately, these 
functions don't seem very smart about dynamically loaded libraries - such as translators, where 
most of our code lives.  They give us the object plus offset from where the object was loaded into 
memory, which isn't that easy to turn into a function name (let alone a file and line number).  It 
seems like libunwind can do better, getting at least to the function name.  AFAICT it's supported 
and packaged on all of our platforms, though there might be version differences.  Newer versions 
can supposedly get to file and line, which would be even better.  Before I get further into this, 
two questions for all of you:

(1) Has somebody already gone down this path?  Does it work?

(2) Are there any other reasons we wouldn't want to switch?



Cannot think of any. The BSD platforms seem to have libunwind and Mac
OS X doesn't have it apparently [1].

I have been thinking of fixing the recent Mac OS X compilation
problems and can address issues related to libunwind as part of that
activity.


I have libunwind.h in the XCode headers and /usr/lib/libunwind.dylib.

Brew also has a libunwind-headers package, but I didn't look at what it 
provides vis-a-vis the XCode headers.


--

Kaleb

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Events API new Port requirement

2016-09-08 Thread Sankarshan Mukhopadhyay 
On Sun, Aug 28, 2016 at 2:13 PM, Niels de Vos  wrote:
> This definitely has my preference too. I've always wanted to try to
> register port 24007/8, and maybe the time has come to look into it.

Has someone within the project previously undertaken the process of
requesting the IANA for assignment of a new service name and port
number value?


-- 
sankarshan mukhopadhyay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Libunwind

2016-09-08 Thread Niels de Vos 
On Thu, Sep 08, 2016 at 10:33:25AM -0400, Vijay Bellur wrote:
> On Thu, Sep 8, 2016 at 9:07 AM, Jeff Darcy  wrote:
> > In a few places in our code (e.g. gf_log_callingfn) we use the "backtrace" 
> > and "backtrace_symbols" functions from libc to log stack traces.  
> > Unfortunately, these functions don't seem very smart about dynamically 
> > loaded libraries - such as translators, where most of our code lives.  They 
> > give us the object plus offset from where the object was loaded into 
> > memory, which isn't that easy to turn into a function name (let alone a 
> > file and line number).  It seems like libunwind can do better, getting at 
> > least to the function name.  AFAICT it's supported and packaged on all of 
> > our platforms, though there might be version differences.  Newer versions 
> > can supposedly get to file and line, which would be even better.  Before I 
> > get further into this, two questions for all of you:
> >
> > (1) Has somebody already gone down this path?  Does it work?
> >
> > (2) Are there any other reasons we wouldn't want to switch?
> 
> 
> Cannot think of any. The BSD platforms seem to have libunwind and Mac
> OS X doesn't have it apparently [1].
> 
> I have been thinking of fixing the recent Mac OS X compilation
> problems and can address issues related to libunwind as part of that
> activity.

I think libunwind support should be optional, and by default our
./configure fails if --without-libunwind is not given. Possibly there is
an alternative for Mac OS X, but it is not an OS we currently see a lot
of users with, so missing some debugging there is not critical.

Niels


> 
> -Vijay
> 
> [1] http://comments.gmane.org/gmane.comp.lib.unwind.devel/2199
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Libunwind

2016-09-08 Thread Niels de Vos 
On Thu, Sep 08, 2016 at 09:07:33AM -0400, Jeff Darcy wrote:
> In a few places in our code (e.g. gf_log_callingfn) we use the "backtrace" 
> and "backtrace_symbols" functions from libc to log stack traces.  
> Unfortunately, these functions don't seem very smart about dynamically loaded 
> libraries - such as translators, where most of our code lives.  They give us 
> the object plus offset from where the object was loaded into memory, which 
> isn't that easy to turn into a function name (let alone a file and line 
> number).  It seems like libunwind can do better, getting at least to the 
> function name.  AFAICT it's supported and packaged on all of our platforms, 
> though there might be version differences.  Newer versions can supposedly get 
> to file and line, which would be even better.  Before I get further into 
> this, two questions for all of you:
> 
> (1) Has somebody already gone down this path?  Does it work?
> 
> (2) Are there any other reasons we wouldn't want to switch?

No objections from me. We can get to function names, files and line
numbers when installing the -debuginfo packages, loading the xlator .so
in gdb and such. But that really is too difficult for most users (and
requires the exact same GlusterFS version + architecture as the logs).

Enhancing this will hopefully get us better bug reports and maybe even
some more contributors sending patches.

When you get this to work, please remove contrib/libexecinfo as well.

Thanks!
Niels


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Libunwind

2016-09-08 Thread Jeff Darcy 
In a few places in our code (e.g. gf_log_callingfn) we use the "backtrace" and 
"backtrace_symbols" functions from libc to log stack traces.  Unfortunately, 
these functions don't seem very smart about dynamically loaded libraries - such 
as translators, where most of our code lives.  They give us the object plus 
offset from where the object was loaded into memory, which isn't that easy to 
turn into a function name (let alone a file and line number).  It seems like 
libunwind can do better, getting at least to the function name.  AFAICT it's 
supported and packaged on all of our platforms, though there might be version 
differences.  Newer versions can supposedly get to file and line, which would 
be even better.  Before I get further into this, two questions for all of you:

(1) Has somebody already gone down this path?  Does it work?

(2) Are there any other reasons we wouldn't want to switch?
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] File snapshot design propsals

2016-09-08 Thread Jeff Darcy 
> 1) Doing file snapshot using shards: (This is suggested by shyam, tried to
> keep the text as is)
> If a block for such a file is written to with a higher version then the brick
> xlators can perform a block copy and then change the new block to the new
> version, and let the older version be as is.

How large are shards, and how often will partial-shard writes occur?  Doing
copy-on-write (what this really is) a lot could be costly.

> This leaves blocks with various versions on disk, and when a older snap
> (version) is deleted, then the corresponding blocks are freed.

If a shard is shared between multiple snapshots, this garbage collection
problem becomes significantly more complicated.

> 2) Doing a file snapshot using sparse files:
> This is sort of inspired from granular data self-heal idea we wanted to
> implement in afr, where we logically represent each block/shard used in the
> file by a bitmap stored either as an xattr or written to a metafile. So
> there is no physical division of the file into different shards. When a
> snapshot is taken, a new sparsefile is created of same size as before, new
> writes on the file are redirected to this file instead of the original file,
> thus preserving the old file. When a write is performed on this file, we
> mark which block is going to be written, copy out this block from older
> shard, overwrite the buffer and then write to the new version and mark the
> block as used either in xattr/metafile.

Again, this will become a lot more complicated with N snapshots, because
we'd need to record which of N files contains the relevant version of a
block.

> 3) Doing filesnapshots by using reflink functionality given by the underlying
> FS:
> When a snapshot request comes, we just do a reflink of the earlier file to
> the latest version and new writes are redirected to this new version of the
> file.

For the sake of completeness:

4) Log-structured merge tree approach.  This is actually similar to 2, but
using append-only logs instead of bitmaps.  New writes are simply added to
the most recent log.  Reads are satisfied by checking one or more logs
before going to the base file (if any).  A snapshot is simply base plus
logs up to the snapshot time, minus any logs after.

This is similar to some elements of JBR, to Cassandra, and to qcow2.  With
specific reference to the latter (which is cross-platform) is there a way
we can use that instead of implementing our own version of 2 or 4?


Lastly, how do *writable* snapshots (clones) affect the tradeoffs between
these approaches?
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Weekly community meeting - 7-Sept-2016

2016-09-08 Thread Kaushal M 
The meeting minutes are here slightly late,  because I forgot to
`#startmeeting` the meeting. The minutes and logs can be obtained at
the links below,

Minutes: 
https://meetbot.fedoraproject.org/gluster-meeting/2016-09-07/gluster-meeting.2016-09-07-17.30.html
Minutes(text): 
https://meetbot-raw.fedoraproject.org/gluster-meeting/2016-09-07/gluster-meeting.2016-09-07-17.30.txt
Log: 
https://meetbot.fedoraproject.org/gluster-meeting/2016-09-07/gluster-meeting.2016-09-07-17.30.log.html


We had another active meeting this week. A big thank you to all the attendees.

Next weeks meeting will be hosted by Samikshan(irc: samikshan), at the
same time and same place.
See you all next week.

~kaushal

Meeting summary
---
* Roll Call  (kshlm, 17:30:58)

* Next weeks host  (kshlm, 17:34:53)
  * samikshan is next week's host  (kshlm, 17:38:18)

* GlusterFS-4.0  (kshlm, 17:38:52)

* GlusterFS-3.9  (kshlm, 17:41:41)
  * LINK:
https://www.gluster.org/pipermail/gluster-devel/2016-September/050741.html
(kshlm, 17:45:39)
  * ACTION: aravindavk to update the lists with the target dates for the
3.9 release  (kshlm, 17:48:06)

* GlusterFS-3.8  (kshlm, 17:49:39)

* GlusterFS-3.7  (kshlm, 17:51:56)

* Project Infrastructure  (kshlm, 18:00:02)
  * LINK:
http://review.gluster.org/#/q/I6f57b5e8ea174dd9e3056aff5da685e497894ccf
(ndevos, 18:12:09)

* NFS-Ganesha  (kshlm, 18:15:59)

* Samba  (kshlm, 18:21:55)

* Last weeks AIs  (kshlm, 18:26:49)

* improve cleanup to control the processes that test starts  (kshlm,
  18:26:57)
  * ACTION: rastar_afk/ndevos/jdarcy to  improve cleanup to control the
processes that test starts  (kshlm, 18:29:49)

* Open Floor  (kshlm, 18:30:07)
  * LINK:

http://review.gluster.org/#/q/status:open+project:glusterfs+branch:master+topic:bug-1369124
(ndevos, 18:37:46)

Meeting ended at 18:39:27 UTC.




Action Items

* aravindavk to update the lists with the target dates for the 3.9
  release
* rastar_afk/ndevos/jdarcy to  improve cleanup to control the processes
  that test starts




Action Items, by person
---
* aravindavk
  * aravindavk to update the lists with the target dates for the 3.9
release
* ndevos
  * rastar_afk/ndevos/jdarcy to  improve cleanup to control the
processes that test starts
* rastar
  * rastar_afk/ndevos/jdarcy to  improve cleanup to control the
processes that test starts
* **UNASSIGNED**
  * (none)




People Present (lines said)
---
* kshlm (143)
* nigelb (48)
* ndevos (26)
* kkeithley (20)
* obnox (13)
* post-factum (8)
* jkroon (6)
* aravindavk (6)
* samikshan (4)
* ** (3)
* jiffin (2)
* anoopcs (1)
* misc (1)
* zodbot (1)
* msvbhat (1)
* mchangir (1)
* ira (1)
* rastar (1)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel