erasing

2014-03-28 Thread Beecham Wemhoff
truth of that great pr -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Hail license changed

2012-12-12 Thread Jeff Garzik
Hail license change was just pushed to the github hail repository. Jeff -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Hail status and update (was Re: Question about hail)

2012-12-10 Thread Hideki Yamane
Hi, Thank you for your mail, Pete and Jeff :) And also late to reply since tons of task and mail makes me busy (and a bit sick). On Tue, 27 Nov 2012 14:47:38 -0500 Jeff Garzik jgar...@pobox.com wrote: Q: Is this project is still alive? if so, where is the current main site. Could you

Hail status and update (was Re: Question about hail)

2012-11-27 Thread Jeff Garzik
(CC'd hail-devel list) On 11/26/2012 02:28 AM, Hideki Yamane wrote: Hello hail upstream authors, I'm interested in porting hail (and Aeolus) to Debian, but have some questions about it. Cool! Q: Is this project is still alive? if so, where is the current main site. Could you tell

Saia do SERASA/SPC!!!

2012-07-29 Thread serasa
Você possuí nome sujo no SPC/SERASA e deseja limpar o mesmo? Limpe agora mesmo! Agora você pode limpar seu nome sem pagar a divida toda. Trabalhamos com Seriedade e cumprimos o prometido. Envie ja um email soilicitando informações para:limpan...@r7.com -- To unsubscribe from this list: send

Project Hail wikis alive again!

2012-05-02 Thread Jeff Garzik
kernel.org fixed their wiki system, which means that all the k.org wikis are once again read-write! This includes Project Hail's home page, https://hail.wiki.kernel.org/ I hope to have the git repos moved back from https://github.com/jgarzik/ to kernel.org soon also. Jeff --

Server Rental Service in HK

2012-04-12 Thread boris
Dear All, We have our own datacenter in Hong Kong provide email/application/web rental service to clients.We are APNIC member provide clean IP to clients. Dell? PowerEdge? EnterpriseRack Mount Server -Intel(R) Xeon(R) E3-1240 Processor (3.3GHz, 8M Cache, Turbo, 4C/8T, 80W) -8GB RAM, 2x4GB,

Re: Unable to log into Hail Wiki

2012-01-26 Thread Jeff Garzik
On 01/25/2012 08:40 PM, Pete Zaitcev wrote: Jeff, looks like the wiki rots. The login points to this URL https://hail.wiki.kernel.org/articles/u/s/e/Special%7EUserLogin_94cd.html It returns 404. HALP? Yes -- all kernel.org wikis are _still_ read-only, even this many months after the

Unable to log into Hail Wiki

2012-01-25 Thread Pete Zaitcev
Jeff, looks like the wiki rots. The login points to this URL https://hail.wiki.kernel.org/articles/u/s/e/Special%7EUserLogin_94cd.html It returns 404. HALP? -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More

Re: [patch hail 1/1] Plug leak in hstor_parse_key

2011-10-18 Thread Jeff Garzik
On 10/14/2011 01:34 PM, Pete Zaitcev wrote: Signed-off-by: Pete Zaitcevzait...@kotori.zaitcev.us --- lib/hstor.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/hstor.c b/lib/hstor.c index cb9c4da..5ce9b76 100644 --- a/lib/hstor.c +++ b/lib/hstor.c @@ -761,7 +761,7

[patch hail 1/1] Plug leak in hstor_parse_key

2011-10-14 Thread Pete Zaitcev
Signed-off-by: Pete Zaitcev zait...@kotori.zaitcev.us --- lib/hstor.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/hstor.c b/lib/hstor.c index cb9c4da..5ce9b76 100644 --- a/lib/hstor.c +++ b/lib/hstor.c @@ -761,7 +761,7 @@ void hstor_free_keylist(struct

Hail git tree location

2011-10-04 Thread Pete Zaitcev
Are we going to have a git tree somewhere? It looks like our old one was purged from git.kernel.org. -- P -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Hail git tree location

2011-10-04 Thread Jeff Garzik
On 10/04/2011 12:12 PM, Pete Zaitcev wrote: Are we going to have a git tree somewhere? It looks like our old one was purged from git.kernel.org. Sorry, I should have posted. It was migrated along with the kernel.org trees to https://github.com/jgarzik/{hail,tabled,itd,nfs4d}

FYI, rpc/ is gone from Fedora 15

2011-05-05 Thread Jim Meyering
FYI, /usr/include/rpc/ no longer exists, as of F15's glibc-headers-2.13.90-10, so hail's lib/cld_msg_rpc.h will have to do something about this #include: $ grep rpc.h lib/cld_msg_rpc.h #include rpc/rpc.h -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of

Re: FYI, rpc/ is gone from Fedora 15

2011-05-05 Thread Jeff Garzik
On 05/05/2011 10:14 AM, Jim Meyering wrote: FYI, /usr/include/rpc/ no longer exists, as of F15's glibc-headers-2.13.90-10, so hail's lib/cld_msg_rpc.h will have to do something about this #include: $ grep rpc.h lib/cld_msg_rpc.h #includerpc/rpc.h hm. Surely they did not delete

Re: FYI, rpc/ is gone from Fedora 15

2011-05-05 Thread Pete Zaitcev
On Thu, 05 May 2011 11:34:42 -0400 Jeff Garzik j...@garzik.org wrote: On 05/05/2011 10:14 AM, Jim Meyering wrote: FYI, /usr/include/rpc/ no longer exists, as of F15's glibc-headers-2.13.90-10, so hail's lib/cld_msg_rpc.h will have to do something about this #include: $ grep

[hail] CLD conversion to TCP lands in git

2011-03-24 Thread Jeff Garzik
I just pushed the CLD protocol change (UDP - TCP) to hail.git[1]. See the original post[2] for more details. It seems pretty solid from my beating on it, but it's still raw code. The focus will be on hammering out the kinks in this switch over the next 7-10 days, so expect some breakage

Re: [PATCH 2/3] CLD: switch network proto from UDP to TCP

2011-01-03 Thread Jeff Garzik
On 01/02/2011 06:32 PM, Pete Zaitcev wrote: On Fri, 31 Dec 2010 05:57:28 -0500 Jeff Garzikj...@garzik.org wrote: + struct cldc_tcp *tcp = private; + ssize_t rc; + struct ubbp_header ubbp; + + memcpy(ubbp.magic, CLD1, 4); + ubbp.op_size = (buflen 8) | 1; +#ifdef

Crash with db5

2011-01-02 Thread Pete Zaitcev
Looks like Rawhide throws this if libdb-devel is in use: make check-TESTS make[3]: Entering directory `/q/zaitcev/hail/hail-tip/test/cld' PASS: prep-db DB_ENV-lsn_reset: method not permitted before handle's open method DB_ENV-dbremove: method not permitted before handle's open method cld[11548]:

Re: Crash with db5

2011-01-02 Thread Jeff Garzik
On 01/02/2011 08:20 PM, Pete Zaitcev wrote: Looks like Rawhide throws this if libdb-devel is in use: make check-TESTS make[3]: Entering directory `/q/zaitcev/hail/hail-tip/test/cld' PASS: prep-db DB_ENV-lsn_reset: method not permitted before handle's open method DB_ENV-dbremove: method not

Re: Crash with db5

2011-01-02 Thread Pete Zaitcev
On Sun, 02 Jan 2011 20:44:02 -0500 Jeff Garzik j...@garzik.org wrote: DB_ENV-dbremove: method not permitted before handle's open method cld[11548]: SIGSEGV Are you compiling with db4 headers, but linking with db5? Or vice versa? It turned out that Rawhide revved up the db5, so this

[PATCH 1/3] CLD: convert back to libevent

2010-12-31 Thread Jeff Garzik
Switch CLD from hand-rolled server poll code, to libevent. Follows similar techniques and rationale as chunkd commit c1aed7464f237e5a6309351bf003162c77d69e27. This reverts ancient commit 90b3b5edcf5aa00577f4395fdbb490ed7e9be824. Signed-off-by: Jeff Garzik jgar...@redhat.com ---

[PATCH 2/3] CLD: switch network proto from UDP to TCP

2010-12-31 Thread Jeff Garzik
Convert CLD network protocol from UDP to TCP. Server, client lib, and chunkd's cldu module are all updated. tabled's cldu module must be updated also. The original rationale for UDP use was following Google's lead, based on the advice in the original Chubby paper, describing TCP's back-off

Re: [patch tabled 6/8] Add filesystem back-end

2010-12-13 Thread Jeff Garzik
On 11/28/2010 08:41 PM, Pete Zaitcev wrote: This patch adds the first new back-end and makes some changes to the way nodes are added, to make the invariants of storage_node more sensible. The filesystem back-end itself is not intended for production use, so it makes no attempt to run any

Re: [patch tabled 8/8] Add Swift back-end

2010-12-13 Thread Jeff Garzik
On 11/28/2010 08:41 PM, Pete Zaitcev wrote: This patch allows to use tabled with OpenStack Swift object store as if it were our chunkserver, with some extra tricks. The configuration has to be entred manually into CLD, just like in case of filesystem back-end. The code is fairly experimental,

Re: [patch tabled 6/8] Add filesystem back-end

2010-12-13 Thread Pete Zaitcev
On Mon, 13 Dec 2010 16:30:59 -0500 Jeff Garzik j...@garzik.org wrote: Current chunkd intentionally prevents two random users from put'ing the same object. The FS backend should do the same. Ouch, I completely forgot about it. Back then I hoped the DB4 transaction would do the job, but

Re: [patch hail 1/2] Add subdomain calling format

2010-12-07 Thread Jeff Garzik
On 12/05/2010 10:53 PM, Pete Zaitcev wrote: Amazon appears to give up on forcing users to migrate and bucket-in-path format is going to stay. However, they still refuse to list buckets from other regions on the default endpoint, which leads to annoying indirection (need to know the region

Re: [patch tabled 1/8] Shuffle fields of storage nodes

2010-12-07 Thread Jeff Garzik
On 11/28/2010 08:39 PM, Pete Zaitcev wrote: This helps copy-paste safer later, mostly. Signed-off-by: Pete Zaitcevzait...@redhat.com --- server/object.c |2 - server/storage.c | 79 ++--- server/tabled.h | 12 +++--- 3 files changed, 53

Re: [patch hail 1/2] Add subdomain calling format

2010-12-07 Thread Pete Zaitcev
On Tue, 07 Dec 2010 05:05:38 -0500 Jeff Garzik j...@garzik.org wrote: On 12/05/2010 10:53 PM, Pete Zaitcev wrote: Amazon appears to give up on forcing users to migrate and bucket-in-path format is going to stay. However, they still refuse to list buckets from other regions on the default

Re: AC_CONFIG_MACRO_DIR([m4])

2010-12-06 Thread Jim Meyering
Pete Zaitcev wrote: Hi, Jim: Autoconf printed a warning when reconfiguting Hail, so I gave up and added this: diff --git a/configure.ac b/configure.ac index 9cfad23..d378854 100644 --- a/configure.ac +++ b/configure.ac @@ -62,6 +62,8 @@ AC_PROG_GCC_TRADITIONAL AM_PROG_CC_C_O

Re: AC_CONFIG_MACRO_DIR([m4])

2010-12-06 Thread Jeff Garzik
On 12/05/2010 04:56 PM, Pete Zaitcev wrote: Autoconf printed a warning when reconfiguting Hail, so I gave up and added this: [...] Now I have a directory m4/ with symlinks... This does not seem to be helping any portability, unless I miss where the promised macro are being saved locally. What

Re: AC_CONFIG_MACRO_DIR([m4])

2010-12-06 Thread Jeff Garzik
On 12/06/2010 12:44 PM, Pete Zaitcev wrote: On Mon, 06 Dec 2010 12:32:22 -0500 Jeff Garzikj...@garzik.org wrote: Keeping the correct libtool macros in-tree implies adding a pointless maintenance burden. The distro always gives us correct, up-to-date files. Why would hail want to potentially

Re: AC_CONFIG_MACRO_DIR([m4])

2010-12-06 Thread Jim Meyering
Jeff Garzik wrote: On 12/06/2010 12:44 PM, Pete Zaitcev wrote: On Mon, 06 Dec 2010 12:32:22 -0500 Jeff Garzikj...@garzik.org wrote: Keeping the correct libtool macros in-tree implies adding a pointless maintenance burden. The distro always gives us correct, up-to-date files. Why would

AC_CONFIG_MACRO_DIR([m4])

2010-12-05 Thread Pete Zaitcev
Hi, Jim: Autoconf printed a warning when reconfiguting Hail, so I gave up and added this: diff --git a/configure.ac b/configure.ac index 9cfad23..d378854 100644 --- a/configure.ac +++ b/configure.ac @@ -62,6 +62,8 @@ AC_PROG_GCC_TRADITIONAL AM_PROG_CC_C_O AM_PROG_LIBTOOL

[patch hail 1/2] Add subdomain calling format

2010-12-05 Thread Pete Zaitcev
Amazon appears to give up on forcing users to migrate and bucket-in-path format is going to stay. However, they still refuse to list buckets from other regions on the default endpoint, which leads to annoying indirection (need to know the region somehow before listing). Easier just use the

[patch tabled 1/8] Shuffle fields of storage nodes

2010-11-28 Thread Pete Zaitcev
This helps copy-paste safer later, mostly. Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/object.c |2 - server/storage.c | 79 ++--- server/tabled.h | 12 +++--- 3 files changed, 53 insertions(+), 40 deletions(-) commit

[patch tabled 2/8] Comment storage.c

2010-11-28 Thread Pete Zaitcev
I have a vague memory that It should be ok to return meant something related to the way our event dispatch worked, but I cannot recall any details. Ergo, useless comment. Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/storage.c |6 +++--- 1 file changed, 3 insertions(+), 3

[patch tabled 5/8] Rename in_storage to in_socket

2010-11-28 Thread Pete Zaitcev
The clause is called Socket, so the old name was confusing. Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/storparse.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) commit 6536a193f1befcf3eed8b18535990e8566518479 Author: Pete Zaitcev zait...@yahoo.com

[patch tabled 4/8] Split out chunk back-end

2010-11-28 Thread Pete Zaitcev
This patch is careful not to change anything but the strictly necessary parts in the actual code. Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/Makefile.am |3 server/stor_chunk.c | 409 ++ server/storage.c| 377

[patch tabled 6/8] Add filesystem back-end

2010-11-28 Thread Pete Zaitcev
This patch adds the first new back-end and makes some changes to the way nodes are added, to make the invariants of storage_node more sensible. The filesystem back-end itself is not intended for production use, so it makes no attempt to run any asynchronous transfers. We also add a test. Note

[patch tabled 8/8] Add Swift back-end

2010-11-28 Thread Pete Zaitcev
This patch allows to use tabled with OpenStack Swift object store as if it were our chunkserver, with some extra tricks. The configuration has to be entred manually into CLD, just like in case of filesystem back-end. The code is fairly experimental, so it retains extra messages. Also, since

[patch tabled 7/8] Clenaup stor_chunk.c

2010-11-28 Thread Pete Zaitcev
Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/stor_chunk.c |6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) commit ca0920cfe5978839ee1a35d6096754a87db6f9ac Author: Pete Zaitcev zait...@yahoo.com Date: Sun Nov 28 18:01:24 2010 -0700 Cleanup stor_chunk. diff

[patch hail 1/2] use hail_log.h

2010-11-10 Thread Pete Zaitcev
I think forward declarations are not needed if we have a nice leaf header. Signed-off-by: Pete Zaitcev zait...@redhat.com --- include/cld_common.h |3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) commit 12f0fd5e0c6925299617082d834220396fae252e Author: Pete Zaitcev zait...@yahoo.com

[patch hail 2/2] add -i to cldcli

2010-11-10 Thread Pete Zaitcev
This is purely cosmetic so that tabled build logs do not show useless prompts. The way -i works copies common shells: one cannot force non-interactive session. Signed-off-by: Pete Zaitcev zait...@redhat.com --- tools/cldcli.c | 14 +- 1 file changed, 13 insertions(+), 1

[patch hail] remove duplicated stc_readport

2010-10-26 Thread Pete Zaitcev
Now that we have a common library for Hail, an opportunity opens to trim some duplication, such as stc_readport. It even had a comment about it. Note that we leave cld_readport in the API for a few weeks, while I get my tabled trees and RPMs in order. Unfortunately we routinely neglect to set

Re: [patch hail] remove duplicated stc_readport

2010-10-26 Thread Jeff Garzik
On 10/26/2010 03:47 PM, Pete Zaitcev wrote: Now that we have a common library for Hail, an opportunity opens to trim some duplication, such as stc_readport. It even had a comment about it. Note that we leave cld_readport in the API for a few weeks, while I get my tabled trees and RPMs in order.

Re: [PATCH hail] const-correctness tweaks

2010-10-23 Thread Jim Meyering
Jeff Garzik wrote: ... But even if curl were requiring some suboptimal signature, it would be nice not to impose that on all projects that use hail. Are there older curl headers that do require the const-free signature? If there are and you want to support them, too, let me know -- maybe I

tabled + atcp

2010-10-23 Thread Jeff Garzik
Just committed this: commit 57c4be44cdfa6c0cda6cf26d19e8048a945c5a78 Author: Jeff Garzik j...@garzik.org Date: Sat Oct 23 14:01:20 2010 -0400 Use libhail's atcp rather than our own async TCP write code. Should be functionally equivalent, as atcp originated from tabled code.

Re: [PATCH hail] const-correctness tweaks

2010-10-22 Thread Jeff Garzik
On 10/20/2010 04:53 AM, Jim Meyering wrote: Jeff Garzik wrote: ... Hi Jeff. Sorry I didn't notice that the first time. I built with ./autogen.sh ./configure make. It looks like you recommend -Wall -Wshadow. The two warnings above are the only ones I see with the patch, and they're easy to

hail version 0.7.2 released

2010-10-22 Thread Jeff Garzik
Home: https://hail.wiki.kernel.org/ Git: git://git.kernel.org/pub/scm/daemon/distsrv/hail.git Download: http://www.kernel.org/pub/software/network/distsrv/hail/ Version 0.7.2 release notes (NEWS): - cld: read overrun bug fix - chunkd: add checksum table to disk format, one checksum per 64k of

tabled version 0.5.2 released

2010-10-22 Thread Jeff Garzik
Home: https://hail.wiki.kernel.org/ Git: git://git.kernel.org/pub/scm/daemon/distsrv/tabled.git Download: http://www.kernel.org/pub/software/network/distsrv/tabled/ Version 0.5.2 release notes (NEWS): - Permit randomly allocated TCP port, for db4 replication master - Install etc.tabled.conf as

Re: tabled version 0.5.2 released

2010-10-22 Thread Jeff Garzik
On 10/22/2010 11:39 PM, Jeff Garzik wrote: Home: https://hail.wiki.kernel.org/ Git: git://git.kernel.org/pub/scm/daemon/distsrv/tabled.git Download: http://www.kernel.org/pub/software/network/distsrv/tabled/ Version 0.5.2 release notes (NEWS): - Permit randomly allocated TCP port, for db4

Re: hail version 0.7.2 released

2010-10-22 Thread Jeff Garzik
On 10/22/2010 11:22 PM, Jeff Garzik wrote: Home: https://hail.wiki.kernel.org/ Git: git://git.kernel.org/pub/scm/daemon/distsrv/hail.git Download: http://www.kernel.org/pub/software/network/distsrv/hail/ It seems that kernel.org mirroring is broken or extremely slow at the moment. The

Re: [PATCH hail] const-correctness tweaks

2010-10-20 Thread Jim Meyering
Jeff Garzik wrote: On 10/06/2010 08:07 AM, Jim Meyering wrote: Make write_cb callback's buffer parameter const, like all write-like functions. Give a few char * parameters the const attribute. Signed-off-by: Jim Meyeringmeyer...@redhat.com --- It looks like most of hail's interfaces are

Re: [PATCH hail] const-correctness tweaks

2010-10-20 Thread Jeff Garzik
On 10/20/2010 04:00 AM, Jim Meyering wrote: Jeff Garzik wrote: On 10/06/2010 08:07 AM, Jim Meyering wrote: Make write_cb callback's buffer parameter const, like all write-like functions. Give a few char * parameters the const attribute. Signed-off-by: Jim Meyeringmeyer...@redhat.com --- It

Re: [PATCH hail] const-correctness tweaks

2010-10-20 Thread Jim Meyering
Jeff Garzik wrote: ... Hi Jeff. Sorry I didn't notice that the first time. I built with ./autogen.sh ./configure make. It looks like you recommend -Wall -Wshadow. The two warnings above are the only ones I see with the patch, and they're easy to fix. When storing const pointer params

[patch tabled] distribute the sample tabled.conf

2010-10-15 Thread Pete Zaitcev
The doc/etc.tabled.conf is a vital piece of documentation but we forgot to include it into the actual distribution. It was only available for those pulling the git tree, not tarballs. Also update the sample spec. Signed-off-by: Pete Zaitcev zait...@redhat.com --- doc/Makefile.am |1 +

Re: [PATCH hail] const-correctness tweaks

2010-10-07 Thread Jeff Garzik
On 10/06/2010 08:07 AM, Jim Meyering wrote: Make write_cb callback's buffer parameter const, like all write-like functions. Give a few char * parameters the const attribute. Signed-off-by: Jim Meyeringmeyer...@redhat.com --- It looks like most of hail's interfaces are const-correct, but one

Re: [PATCH hail] const-correctness tweaks

2010-10-07 Thread Jim Meyering
Jeff Garzik wrote: On 10/06/2010 08:07 AM, Jim Meyering wrote: ... It looks like most of hail's interfaces are const-correct, but one stood out because it provokes a warning when I tried to pass a const-correct write_cb function to hstor_get from iwhd: ... include/hstor.h |4 ++--

[PATCH tabled] adapt to changed signature of hstor_get's callback function

2010-10-07 Thread Jim Meyering
* test/large-object.c: Hail has changed hstor_get's callback function so that it now declares its buffer to be const, as all write-like functions do. Adjust this file's hstor_get callback parameter and propagate that, as required, to the local functions it uses to operate on that now-read-only

[PATCH hail] const-correctness tweaks

2010-10-06 Thread Jim Meyering
Make write_cb callback's buffer parameter const, like all write-like functions. Give a few char * parameters the const attribute. Signed-off-by: Jim Meyering meyer...@redhat.com --- It looks like most of hail's interfaces are const-correct, but one stood out because it provokes a warning when I

[hail patch 1/2] Change cfgfile.txt into a real config file

2010-10-05 Thread Pete Zaitcev
Not many people read Recipies page at the wiki, so let's change the in-tree cfgfile.txt so that it can be copied into /etc. Signed-off-by: Pete Zaitcev zait...@redhat.com --- doc/cfgfile.txt | 57 +++--- 1 file changed, 19 insertions(+), 38 deletions(-)

CLD multi-node status

2010-09-30 Thread Geert Jansen
Hi, is it correct that CLD is basically single-master right now? I can't find any trace of the mentioned Paxos implementation in the source. Regards, Geert -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo

Re: CLD multi-node status

2010-09-30 Thread Jeff Garzik
On 09/30/2010 04:55 AM, Geert Jansen wrote: is it correct that CLD is basically single-master right now? I can't find any trace of the mentioned Paxos implementation in the source. The current main branch is single-master, correct. The 'replica' branch of hail.git contains the multi-node

Re: [PATCH hail] chunkd: don't leak an FS object iterator

2010-09-30 Thread Jeff Garzik
On 09/29/2010 11:20 AM, Jim Meyering wrote: chk_list_objs called fs_list_objs_open without also calling fs_list_objs_close. 32,808 bytes in 1 blocks are definitely lost in loss record 413 of 419 at 0x4A0515D: malloc (vg_replace_malloc.c:195) by 0x31BA8A26D0: __alloc_dir

Re: [PATCH hail] chunkd: don't leak an FS object iterator

2010-09-30 Thread Jeff Garzik
On 09/29/2010 11:20 AM, Jim Meyering wrote: chk_list_objs called fs_list_objs_open without also calling fs_list_objs_close. 32,808 bytes in 1 blocks are definitely lost in loss record 413 of 419 at 0x4A0515D: malloc (vg_replace_malloc.c:195) by 0x31BA8A26D0: __alloc_dir

[chunkd patch] convert to libevent

2010-09-30 Thread Jeff Garzik
For a nice code savings... chunkd/Makefile.am |1 chunkd/chunkd.h| 28 + chunkd/cldu.c | 64 +-- chunkd/server.c| 289 + chunkd/util.c | 23 configure.ac |3 6 files changed, 116

Re: Autostart

2010-09-30 Thread Jeff Garzik
On Wed, Sep 29, 2010 at 7:09 PM, Pete Zaitcev zait...@redhat.com wrote: An interesting question is what to do when iwhd exits. I decided not to kill what was started. So, we have a little self-contained cell of tabled, chunkd, S3, based off a certain local directory or other namespace anchor.

[PATCH hail] chunkd: don't leak an FS object iterator

2010-09-29 Thread Jim Meyering
chk_list_objs called fs_list_objs_open without also calling fs_list_objs_close. 32,808 bytes in 1 blocks are definitely lost in loss record 413 of 419 at 0x4A0515D: malloc (vg_replace_malloc.c:195) by 0x31BA8A26D0: __alloc_dir (opendir.c:184) by 0x405619: fs_list_objs_open

Autostart

2010-09-29 Thread Pete Zaitcev
Jeff et.al.: I am working on a feature called autostart in Image Warehouse. It amounts to iwhd starting with a certain default configuration, but is a bit more: it also launches services necessary to support said default, e.g. tabled. Naturally, tabled needs cld and chunkd. An interesting

[PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam

2010-09-27 Thread Jim Meyering
Signed-off-by: Jim Meyering meyer...@redhat.com --- I would have preferred to insert a single line right before the huri_field_escape call: char *v = strdup(val); [would result in a more compact, single-hunk patch] but it looks like hail uses the anachronistic (pre-C99) declare all vars at

Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam

2010-09-27 Thread Pete Zaitcev
On Mon, 27 Sep 2010 10:53:06 +0200 Jim Meyering j...@meyering.net wrote: - stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK); + v = strdup(val); + stmp = huri_field_escape(v, QUERY_ESCAPE_MASK); str = g_string_append(str, stmp); free(stmp); + free(v); I

Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam

2010-09-27 Thread Jeff Garzik
On 09/27/2010 04:53 AM, Jim Meyering wrote: Signed-off-by: Jim Meyeringmeyer...@redhat.com --- I would have preferred to insert a single line right before the huri_field_escape call: char *v = strdup(val); [would result in a more compact, single-hunk patch] but it looks like hail uses

Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam

2010-09-27 Thread Jim Meyering
Pete Zaitcev wrote: On Mon, 27 Sep 2010 10:53:06 +0200 Jim Meyering j...@meyering.net wrote: -stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK); +v = strdup(val); +stmp = huri_field_escape(v, QUERY_ESCAPE_MASK); str = g_string_append(str, stmp); free(stmp); +

Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam

2010-09-27 Thread Jeff Garzik
On 09/27/2010 12:29 PM, Pete Zaitcev wrote: On Mon, 27 Sep 2010 10:53:06 +0200 Jim Meyeringj...@meyering.net wrote: - stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK); + v = strdup(val); + stmp = huri_field_escape(v, QUERY_ESCAPE_MASK); str =

Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam

2010-09-27 Thread Pete Zaitcev
On Mon, 27 Sep 2010 12:53:48 -0400 Jeff Garzik j...@garzik.org wrote: - stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK); + v = strdup(val); + stmp = huri_field_escape(v, QUERY_ESCAPE_MASK); str = g_string_append(str, stmp); free(stmp); + free(v); applied

Re: [hail patch 1/1] Fix calling convention of huri_field_escape

2010-09-27 Thread Jeff Garzik
On 09/27/2010 08:49 PM, Pete Zaitcev wrote: Premature optimization is the root of all evil. Use a sensible convention of not screwing with the argument, at the expense of extra strdup. Fortunately, all users are confined to Hail itself, even if huri_field_escape is exported. Signed-off-by:

Re: [tabled patch 1/1] Add a test for hstor_keys

2010-09-27 Thread Jeff Garzik
On 09/27/2010 08:52 PM, Pete Zaitcev wrote: Our current tests do not invoke hstor_keys at all, and so they did not catch a crash with double free in append_qparam. Add a very basic test which at least calls hstor_keys to verify that it does not crash right away. This test does not excercise

Re: [PATCH tabled 1/2] server/config.c: don't dereference NULL on OOM

2010-09-24 Thread Jim Meyering
Jeff Garzik wrote: On 09/23/2010 04:43 AM, Jim Meyering wrote: From fb7865d158b0d32907dde703c4d37c70a26e738c Mon Sep 17 00:00:00 2001 From: Jim Meyeringmeyer...@redhat.com Date: Thu, 23 Sep 2010 10:11:44 +0200 Subject: [PATCH tabled 1/2] server/config.c: don't dereference NULL on OOM (see

Re: [PATCH tabled 1/2] server/config.c: don't dereference NULL on OOM

2010-09-24 Thread Jim Meyering
Jeff Garzik wrote: On 09/23/2010 03:19 PM, Jeff Garzik wrote: 3) I process patches similar to how Linus and others in the kernel do it: git am /path/to/mbox_of_patches That tends to impose some restrictions on the contents of each email. FWIW, 'git pull' submissions are welcome. Standard

Re: [PATCH tabled 1/2] server/config.c: don't dereference NULL on OOM

2010-09-24 Thread Jeff Garzik
On 09/24/2010 07:32 AM, Jim Meyering wrote: You can pull from the oom branch here: git://git.infradead.org/users/meyering/tabled.git Got nearly everything perfect. Need one more minor yet important change. As described in doc/contributions.txt, every changeset MUST have a Signed-off-by

Re: [PATCH tabled 1/2] server/config.c: don't dereference NULL on OOM

2010-09-24 Thread Jim Meyering
Jeff Garzik wrote: On 09/24/2010 07:32 AM, Jim Meyering wrote: You can pull from the oom branch here: git://git.infradead.org/users/meyering/tabled.git Got nearly everything perfect. Need one more minor yet important change. As described in doc/contributions.txt, every changeset MUST

Re: [PATCH tabled 1/2] server/config.c: don't dereference NULL on OOM

2010-09-24 Thread Jeff Garzik
On 09/24/2010 01:43 PM, Jim Meyering wrote: Jeff Garzik wrote: On 09/24/2010 07:32 AM, Jim Meyering wrote: You can pull from the oom branch here: git://git.infradead.org/users/meyering/tabled.git Got nearly everything perfect. Need one more minor yet important change. As described in

Re: [PATCH tabled] server/server.c (net_write_port): Don't ignore write error.

2010-09-23 Thread Jeff Garzik
On 09/23/2010 03:55 AM, Jim Meyering wrote: Better safe than sorry... Unreported write failures can be unpleasant. I fixed the one below so that a failure indication can propagate up the call tree. You might also want to report the failure to stderr. I let my editor automatically update the

[tabled patch v2] abstract out TCP-write code

2010-09-23 Thread Jeff Garzik
Changes from v1: - avoid referencing dead struct client (grep for 'invalidate_cli'), by changing FSM callback prototype. - insert 'void *priv' member into struct atcp_wr_state, and replace cb_data1/cb_data2 callback parameters with (struct atcp_wr_state *, void *). struct client / struct

Re: [tabled patch] abstract out TCP-write code

2010-09-23 Thread Jeff Garzik
On 09/23/2010 11:28 AM, Jim Meyering wrote: Every developer should have MALLOC_PERTURB_=N (N in 1..255) set in his/her environment on glibc-based systems. Almost all the time. I heard about it a while ago, even submitted a bugzilla bug to have it documented adequately. But apparently its

Re: [tabled patch] abstract out TCP-write code

2010-09-23 Thread Jeff Garzik
On 09/22/2010 10:37 PM, Pete Zaitcev wrote: On Wed, 22 Sep 2010 21:26:13 -0400 Jeff Garzikj...@garzik.org wrote: It is a common idiom even in GLib that callbacks receive two anonymous pointers; witness the data type GFunc's 'data' and 'user_data' arguments:

Re: [tabled patch] abstract out TCP-write code

2010-09-22 Thread Jeff Garzik
On 09/22/2010 10:37 PM, Pete Zaitcev wrote: On Wed, 22 Sep 2010 21:26:13 -0400 Jeff Garzikj...@garzik.org wrote: So, we go a longer route and re-hook the list of completions to a per-server global instead of a client. The patch is straight- forward. The only thing we need to

Re: Reconsidering libevent

2010-09-21 Thread Steven Dake
On 09/21/2010 01:51 PM, Jeff Garzik wrote: Today's Project Hail hacking involves moving tcp_write_* code from itd.git to libhail. tcp_write_* is the well-debugged TCP async write queueing code from tabled, with cosmetic changes to provide a nice API encapsulation. I did this when moving the

Re: Reconsidering libevent

2010-09-21 Thread Jeff Garzik
On Tue, Sep 21, 2010 at 5:06 PM, Steven Dake sd...@redhat.com wrote: libevent version 2 has proper mutual exclusion, but the code needs some work. 1.x should work for chunkd at the moment. I need to resist my own urge to think too far ahead and overengineer for the future sometimes; I think

Re: On licensing and libhail

2010-09-21 Thread Pete Zaitcev
On Tue, 21 Sep 2010 17:04:41 -0400 Jeff Garzik j...@garzik.org wrote: Occasionally I receive a ping about libhail licensing. GPL is probably less convenient than LGPL, so I was wondering what existing copyright holders thought about changing libhail to LGPL, while leaving the remaining bits

Re: [PATCH] don't expect inode name to be NUL-terminated (avoid read overrun)

2010-09-15 Thread Jim Meyering
Jeff Garzik wrote: On 09/10/2010 08:55 AM, Jim Meyering wrote: * server/msg.c (msg_get): Copy only name_len bytes, then NUL-terminate, rather than using snprintf to copy up to and including nonexistent NUL. --- valgrind exposed this. The use of snprintf would have been correct if the

Re: [hail patch 0/3] chunkd: on-disk checksumming and get-partial operation

2010-09-15 Thread Jeff Garzik
Just pushed this out to hail.git. -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] don't expect inode name to be NUL-terminated (avoid read overrun)

2010-09-14 Thread Jeff Garzik
On 09/10/2010 08:55 AM, Jim Meyering wrote: * server/msg.c (msg_get): Copy only name_len bytes, then NUL-terminate, rather than using snprintf to copy up to and including nonexistent NUL. --- valgrind exposed this. The use of snprintf would have been correct if the inode name buffer

[hail patch 0/3] chunkd: on-disk checksumming and get-partial operation

2010-09-14 Thread Jeff Garzik
This patchset is just about ready to go upstream. Just need to write a couple tests (familiar refrain eh?:)). These changes add a new Get-Partial-Object (GET_PART) chunkd operation. GET_PART permits partial retrieval of an object, by adding an (offset,length) pair to the standard Get-Object

[hail patch 1/3] chunkd: Add checksum table to on-disk format

2010-09-14 Thread Jeff Garzik
commit f1de17a6e2b3afdbfbfa581228280b65a4a17e5f Author: Jeff Garzik j...@garzik.org Date: Thu Aug 5 17:47:03 2010 -0400 chunkd: Add checksum table to on-disk format, one sum per 64k of data Signed-off-by: Jeff Garzik jgar...@redhat.com chunkd/be-fs.c | 162

Re: [PATCH] don't expect inode name to be NUL-terminated (avoid read overrun)

2010-09-10 Thread Colin McCabe
The thing is, snprintf always NULL-terminates its output string, no matter whether the input was NULL-terminated or not. However, I looked at the snprintf man page again and found this description for %s : If no l modifier is present: The const char * argument is expected to be a pointer to

Sorry! I made an error in my last email!

2010-09-08 Thread Sam L. Carl
Hey, Sorry I made an error with the links in my last email. Here is how it should of been: Over the past few months I have taken a lot of my time to research and ask as many people as possible what the top 5 money making methods are. After weeks and weeks of different answers and even trying

Re: stor_obj_test

2010-08-16 Thread Pete Zaitcev
On Wed, 07 Jul 2010 00:02:02 -0400 Jeff Garzik j...@garzik.org wrote: This function seems to be missing the meat. It retrieves then disposes of a keylist. bool stor_obj_test(struct open_chunk *cep, uint64_t key) I do not understand what kind of meat you expect stor_obj_test to have. In

Re: [tabled patch 4/5] Support auto replicaton port

2010-08-13 Thread Jeff Garzik
On 08/12/2010 03:22 PM, Pete Zaitcev wrote: Allow random ports for replication master to listen on. The patch is somewhat larger than expected, because before we had the MASTER file written right after locking. Now we may have it written without listening parameters, and the slaves must be

  1   2   3   4   5   >