Unable to log into Hail Wiki
Jeff, looks like the wiki rots. The login points to this URL https://hail.wiki.kernel.org/articles/u/s/e/Special%7EUserLogin_94cd.html It returns 404. HALP? -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch hail 1/1] Plug leak in hstor_parse_key
Signed-off-by: Pete Zaitcev zait...@kotori.zaitcev.us
---
lib/hstor.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/hstor.c b/lib/hstor.c
index cb9c4da..5ce9b76 100644
--- a/lib/hstor.c
+++ b/lib/hstor.c
@@ -761,7 +761,7 @@ void hstor_free_keylist(struct hstor_keylist *keylist)
static void hstor_parse_key(xmlDocPtr doc, xmlNode *node,
struct hstor_keylist *keylist)
{
- struct hstor_object *obj = calloc(1, sizeof(*obj));
+ struct hstor_object *obj;
xmlChar *xs;
obj = calloc(1, sizeof(*obj));
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hail git tree location
Are we going to have a git tree somewhere? It looks like our old one was purged from git.kernel.org. -- P -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: FYI, rpc/ is gone from Fedora 15
On Thu, 05 May 2011 11:34:42 -0400 Jeff Garzik j...@garzik.org wrote: On 05/05/2011 10:14 AM, Jim Meyering wrote: FYI, /usr/include/rpc/ no longer exists, as of F15's glibc-headers-2.13.90-10, so hail's lib/cld_msg_rpc.h will have to do something about this #include: $ grep rpc.h lib/cld_msg_rpc.h #includerpc/rpc.h hm. Surely they did not delete sunrpc from glibc? Surely they did (and by they I mean Uli, obviously). The suggested fix is to use TI-RPC, whatever that is. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Crash with db5
Looks like Rawhide throws this if libdb-devel is in use: make check-TESTS make[3]: Entering directory `/q/zaitcev/hail/hail-tip/test/cld' PASS: prep-db DB_ENV-lsn_reset: method not permitted before handle's open method DB_ENV-dbremove: method not permitted before handle's open method cld[11548]: SIGSEGV PASS: start-daemon port file not found. FAIL: pid-exists libdb-5.1.19-2.fc15.x86_64 -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Crash with db5
On Sun, 02 Jan 2011 20:44:02 -0500 Jeff Garzik j...@garzik.org wrote: DB_ENV-dbremove: method not permitted before handle's open method cld[11548]: SIGSEGV Are you compiling with db4 headers, but linking with db5? Or vice versa? It turned out that Rawhide revved up the db5, so this band-aids it: commit 42d083416656e195baa79a924df0f11a3f0681c2 Author: Pete Zaitcev zait...@yahoo.com Date: Sun Jan 2 20:28:39 2011 -0700 Accept db-5.1. diff --git a/configure.ac b/configure.ac index 9cfad23..abc9330 100644 --- a/configure.ac +++ b/configure.ac @@ -75,7 +75,8 @@ dnl AC_TYPE_PID_T dnl - dnl Checks for required libraries dnl - -AC_CHECK_LIB(db-5.0, db_create, DB4_LIBS=-ldb-5.0, +AC_CHECK_LIB(db-5.1, db_create, DB4_LIBS=-ldb-5.1, + AC_CHECK_LIB(db-5.0, db_create, DB4_LIBS=-ldb-5.0, AC_CHECK_LIB(db-4.9, db_create, DB4_LIBS=-ldb-4.9, AC_CHECK_LIB(db-4.8, db_create, DB4_LIBS=-ldb-4.8, AC_CHECK_LIB(db-4.7, db_create, DB4_LIBS=-ldb-4.7, @@ -83,7 +84,7 @@ AC_CHECK_LIB(db-5.0, db_create, DB4_LIBS=-ldb-5.0, AC_CHECK_LIB(db-4.5, db_create, DB4_LIBS=-ldb-4.5, AC_CHECK_LIB(db-4.4, db_create, DB4_LIBS=-ldb-4.4, AC_CHECK_LIB(db-4.3, db_create, DB4_LIBS=-ldb-4.3, - [AC_MSG_ERROR([Missing required libdb 4.x])] + [AC_MSG_ERROR([Missing required libdb 4.x])]) AC_CHECK_LIB(event, event_base_new, EVENT_LIBS=-levent, [AC_MSG_ERROR([Missing required libevent])]) AC_CHECK_LIB(crypto, MD5_Init, CRYPTO_LIBS=-lcrypto) This needs to stop eventually, just don't know how. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch tabled 6/8] Add filesystem back-end
On Mon, 13 Dec 2010 16:30:59 -0500 Jeff Garzik j...@garzik.org wrote: Current chunkd intentionally prevents two random users from put'ing the same object. The FS backend should do the same. Ouch, I completely forgot about it. Back then I hoped the DB4 transaction would do the job, but perhaps not. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch hail 1/2] Add subdomain calling format
On Tue, 07 Dec 2010 05:05:38 -0500 Jeff Garzik j...@garzik.org wrote: On 12/05/2010 10:53 PM, Pete Zaitcev wrote: Amazon appears to give up on forcing users to migrate and bucket-in-path format is going to stay. However, they still refuse to list buckets from other regions on the default endpoint, which leads to annoying indirection (need to know the region somehow before listing). Easier just use the subdomain format in one invocation. applied 1-2 Thanks. I eventually got around this by calling buckets foo-west, foo-east, but might as well just have this. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
AC_CONFIG_MACRO_DIR([m4])
Hi, Jim: Autoconf printed a warning when reconfiguting Hail, so I gave up and added this: diff --git a/configure.ac b/configure.ac index 9cfad23..d378854 100644 --- a/configure.ac +++ b/configure.ac @@ -62,6 +62,8 @@ AC_PROG_GCC_TRADITIONAL AM_PROG_CC_C_O AM_PROG_LIBTOOL +AC_CONFIG_MACRO_DIR([m4]) + dnl Checks for header files. AC_HEADER_STDC dnl AC_CHECK_HEADERS(sys/ioctl.h unistd.h) Now I have a directory m4/ with symlinks... This does not seem to be helping any portability, unless I miss where the promised macro are being saved locally. What was this about, do you happen to know? -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch hail 1/2] Add subdomain calling format
Amazon appears to give up on forcing users to migrate and bucket-in-path
format is going to stay. However, they still refuse to list buckets from
other regions on the default endpoint, which leads to annoying indirection
(need to know the region somehow before listing). Easier just use the
subdomain format in one invocation.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
include/hstor.h |6 +
lib/hstor.c | 178 +-
2 files changed, 106 insertions(+), 78 deletions(-)
diff --git a/include/hstor.h b/include/hstor.h
index b47387b..ca63f4a 100644
--- a/include/hstor.h
+++ b/include/hstor.h
@@ -25,6 +25,8 @@
#include curl/curl.h
#include glib.h
+enum hstor_calling_format { HFMT_ORDINARY, HFMT_SUBDOMAIN };
+
struct hstor_client {
CURL*curl;
char*acc;
@@ -32,6 +34,7 @@ struct hstor_client {
char*user;
char*key;
boolverbose;
+ boolsubdomain;
};
struct hstor_bucket {
@@ -165,6 +168,9 @@ extern void hstor_free_keylist(struct hstor_keylist
*keylist);
extern struct hstor_client *hstor_new(const char *service_acc,
const char *service_host, const char *user, const char *secret_key);
+extern bool hstor_set_format(struct hstor_client *hstor,
+enum hstor_calling_format f);
+
extern bool hstor_add_bucket(struct hstor_client *hstor, const char *name);
extern bool hstor_del_bucket(struct hstor_client *hstor, const char *name);
diff --git a/lib/hstor.c b/lib/hstor.c
index 7f638ec..2892a5e 100644
--- a/lib/hstor.c
+++ b/lib/hstor.c
@@ -86,6 +86,21 @@ err_out:
return NULL;
}
+bool hstor_set_format(struct hstor_client *hstor, enum hstor_calling_format f)
+{
+ switch (f) {
+ case HFMT_ORDINARY:
+ hstor-subdomain = false;
+ break;
+ case HFMT_SUBDOMAIN:
+ hstor-subdomain = true;
+ break;
+ default:
+ return false;
+ }
+ return true;
+}
+
static size_t all_data_cb(const void *ptr, size_t size, size_t nmemb,
void *user_data)
{
@@ -177,6 +192,51 @@ next:
}
}
+static bool hstor_resplit(const struct hstor_client *hstor,
+ const char *bucket, const char *key,
+ char **url, char **hosthdr, char **path)
+{
+ char *unesc_path;
+ int rc;
+
+ if (hstor-subdomain)
+ rc = asprintf(unesc_path, /%s, key);
+ else
+ rc = asprintf(unesc_path, /%s/%s, bucket, key);
+ if (rc 0)
+ goto err_spath;
+ *path = huri_field_escape(unesc_path, PATH_ESCAPE_MASK);
+ if (!*path)
+ goto err_epath;
+
+ if (hstor-subdomain)
+ rc = asprintf(hosthdr, Host: %s.%s, bucket, hstor-host);
+ else
+ rc = asprintf(hosthdr, Host: %s, hstor-host);
+ if (rc 0)
+ goto err_host;
+
+ if (hstor-subdomain)
+ rc = asprintf(url, http://%s.%s%s;, bucket, hstor-acc, *path);
+ else
+ rc = asprintf(url, http://%s%s;, hstor-acc, *path);
+ if (rc 0)
+ goto err_url;
+
+ free(unesc_path);
+ return true;
+
+ /* free(*url); */
+ err_url:
+ free(*hosthdr);
+ err_host:
+ free(*path);
+ err_epath:
+ free(unesc_path);
+ err_spath:
+ return false;
+}
+
struct hstor_blist *hstor_list_buckets(struct hstor_client *hstor)
{
struct http_req req;
@@ -318,8 +378,8 @@ static bool __hstor_ad_bucket(struct hstor_client *hstor,
const char *name,
struct curl_slist *headers = NULL;
int rc;
- if (asprintf(orig_path, /%s/, name) 0)
- goto err_spath;
+ if (!hstor_resplit(hstor, name, , url, host, orig_path))
+ goto err_split;
memset(req, 0, sizeof(req));
req.method = delete ? DELETE : PUT;
@@ -330,13 +390,9 @@ static bool __hstor_ad_bucket(struct hstor_client *hstor,
const char *name,
hreq_hdr_push(req, Date, timestr);
- hreq_sign(req, NULL, hstor-key, hmac);
+ hreq_sign(req, hstor-subdomain? name: NULL, hstor-key, hmac);
sprintf(auth, Authorization: AWS %s:%s, hstor-user, hmac);
- if (asprintf(host, Host: %s, hstor-host) 0)
- goto err_host;
- if (asprintf(url, http://%s/%s/;, hstor-acc, name) 0)
- goto err_url;
headers = curl_slist_append(headers, host);
headers = curl_slist_append(headers, datestr);
@@ -360,11 +416,7 @@ static bool __hstor_ad_bucket(struct hstor_client *hstor,
const char *name,
free(orig_path);
return (rc == 0);
-err_url:
- free(host);
-err_host:
- free(orig_path);
-err_spath:
+err_split:
return false;
}
@@ -384,16 +436,12 @@ bool hstor_get(struct hstor_client *hstor, const char
*bucket
[patch tabled 1/8] Shuffle fields of storage nodes
This helps copy-paste safer later, mostly.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/object.c |2 -
server/storage.c | 79 ++---
server/tabled.h | 12 +++---
3 files changed, 53 insertions(+), 40 deletions(-)
commit 2d0645834468ea056223bee52d679c983945ce65
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Nov 11 11:24:00 2010 -0700
Field rearrangements. Layering violation improvement.
diff --git a/server/object.c b/server/object.c
index 71d5377..2920811 100644
--- a/server/object.c
+++ b/server/object.c
@@ -1008,7 +1008,7 @@ static bool object_get_poke(struct client *cli)
ssize_t bytes;
/* The checks for in_len in caller should protect us, but let's see. */
- if (!cli-in_ce.stc) {
+ if (!cli-in_ce.key) {
applog(LOG_ERR, read on closed chunk, in_len %ld,
(long) cli-in_len);
return false;
diff --git a/server/storage.c b/server/storage.c
index fc232ba..c472961 100644
--- a/server/storage.c
+++ b/server/storage.c
@@ -91,8 +91,8 @@ static void stor_read_event(int fd, short events, void
*userdata)
struct open_chunk *cep = userdata;
cep-r_armed = false; /* no EV_PERSIST */
- if (cep-rcb)
- (*cep-rcb)(cep);
+ if (cep-ocb)
+ (*cep-ocb)(cep);
}
static void stor_write_event(int fd, short events, void *userdata)
@@ -100,8 +100,8 @@ static void stor_write_event(int fd, short events, void
*userdata)
struct open_chunk *cep = userdata;
cep-w_armed = false; /* no EV_PERSIST */
- if (cep-wcb)
- (*cep-wcb)(cep);
+ if (cep-ocb)
+ (*cep-ocb)(cep);
}
/*
@@ -131,6 +131,8 @@ int stor_put_start(struct open_chunk *cep, void
(*cb)(struct open_chunk *),
{
char stckey[STOR_KEY_SLEN+1];
+ if (cep-key)
+ return -EBUSY;
if (!cep-stc)
return -EINVAL;
@@ -145,9 +147,10 @@ int stor_put_start(struct open_chunk *cep, void
(*cb)(struct open_chunk *),
cep-node-id, stckey, (long long) size);
return -EIO;
}
- cep-wtogo = size;
- cep-wkey = key;
- cep-wcb = cb;
+ cep-size = size;
+ cep-done = 0;
+ cep-key = key;
+ cep-ocb = cb;
event_set(cep-wevt, cep-wfd, EV_WRITE, stor_write_event, cep);
event_base_set(cep-evbase, cep-wevt);
@@ -167,13 +170,15 @@ int stor_open_read(struct open_chunk *cep, void
(*cb)(struct open_chunk *),
char stckey[STOR_KEY_SLEN+1];
uint64_t size;
+ if (cep-key)
+ return -EBUSY;
if (!cep-stc)
return -EINVAL;
- if (cep-rsize cep-roff != cep-rsize) {
+ if (cep-size cep-done != cep-size) {
applog(LOG_ERR, Unfinished Get (%ld,%ld),
- (long)cep-roff, (long)cep-rsize);
- cep-rsize = 0;
+ (long)cep-done, (long)cep-size);
+ cep-size = 0;
}
sprintf(stckey, stor_key_fmt, (unsigned long long) key);
@@ -184,9 +189,10 @@ int stor_open_read(struct open_chunk *cep, void
(*cb)(struct open_chunk *),
return -EIO;
}
*psize = size;
- cep-rsize = size;
- cep-roff = 0;
- cep-rcb = cb;
+ cep-size = size;
+ cep-done = 0;
+ cep-key = key;
+ cep-ocb = cb;
event_set(cep-revt, cep-rfd, EV_READ, stor_read_event, cep);
event_base_set(cep-evbase, cep-revt);
@@ -213,12 +219,14 @@ void stor_close(struct open_chunk *cep)
event_del(cep-revt);
cep-r_armed = false;
}
- cep-rsize = 0;
+ cep-size = 0;
if (cep-w_armed) {
event_del(cep-wevt);
cep-w_armed = false;
}
+
+ cep-key = 0;
}
/*
@@ -251,40 +259,44 @@ void stor_abort(struct open_chunk *cep)
if (debugging)
applog(LOG_INFO, Failed to reopen Chunk nid %u (%d),
cep-node-id, rc);
+
+ cep-size = 0;
+ cep-done = 0;
+ cep-key = 0;
return;
}
- if (cep-wtogo) {
- sprintf(stckey, stor_key_fmt, (unsigned long long) cep-wkey);
+ if (cep-done != cep-size) {
+ sprintf(stckey, stor_key_fmt, (unsigned long long) cep-key);
stc_delz(cep-stc, stckey);
- cep-wtogo = 0;
}
if (cep-r_armed) {
event_del(cep-revt);
cep-r_armed = false;
}
- cep-rsize = 0;
if (cep-w_armed) {
event_del(cep-wevt);
cep-w_armed = false;
}
+
+ cep-size = 0;
+ cep-done = 0;
+
+ cep-key = 0;
}
ssize_t stor_put_buf(struct open_chunk *cep, void *data, size_t len
[patch tabled 2/8] Comment storage.c
I have a vague memory that It should be ok to return meant something
related to the way our event dispatch worked, but I cannot recall any
details. Ergo, useless comment.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/storage.c |6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
commit 8c4bed7bd56a5bb2c3a654a494a6bb545f0d7db2
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Nov 11 15:46:59 2010 -0700
Update comments.
diff --git a/server/storage.c b/server/storage.c
index c472961..5c6e3a7 100644
--- a/server/storage.c
+++ b/server/storage.c
@@ -161,9 +161,6 @@ int stor_put_start(struct open_chunk *cep, void
(*cb)(struct open_chunk *),
return 0;
}
-/*
- * It should be ok to return
- */
int stor_open_read(struct open_chunk *cep, void (*cb)(struct open_chunk *),
uint64_t key, uint64_t *psize)
{
@@ -379,6 +376,9 @@ int stor_obj_del(struct storage_node *stn, uint64_t key)
return rc;
}
+/*
+ * XXX WTF?! This accidentially tests a node instead of object! FIXME
+ */
bool stor_obj_test(struct open_chunk *cep, uint64_t key)
{
struct st_keylist *klist;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch tabled 5/8] Rename in_storage to in_socket
The clause is called Socket, so the old name was confusing.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/storparse.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
commit 6536a193f1befcf3eed8b18535990e8566518479
Author: Pete Zaitcev zait...@yahoo.com
Date: Sun Nov 28 16:29:54 2010 -0700
Rename in_storage to in_socket, because Socket.
diff --git a/server/storparse.c b/server/storparse.c
index 5ed15ea..cbffe0f 100644
--- a/server/storparse.c
+++ b/server/storparse.c
@@ -36,7 +36,7 @@ struct config_context {
boolin_chunk;
boolin_chunk_reported;
- boolin_storage;
+ boolin_socket;
boolstor_encrypt;
char*stor_port;
char*stor_host;
@@ -82,8 +82,8 @@ static void cfg_elm_start (GMarkupParseContext *context,
}
if (!strcmp(element_name, Socket)) {
- if (!cc-in_storage)
- cc-in_storage = true;
+ if (!cc-in_socket)
+ cc-in_socket = true;
else
applog(LOG_ERR, %s: Nested Socket, cc-fname);
}
@@ -95,7 +95,7 @@ static void cfg_elm_start (GMarkupParseContext *context,
}
}
-static void cfg_elm_end_storage(struct config_context *cc)
+static void cfg_elm_end_socket(struct config_context *cc)
{
if (cc-text) {
applog(LOG_WARNING, %s: Extra text in Socket element: \%s\,
@@ -206,8 +206,8 @@ static void cfg_elm_end (GMarkupParseContext *context,
}
else if (!strcmp(element_name, Socket)) {
- cfg_elm_end_storage(cc);
- cc-in_storage = false;
+ cfg_elm_end_socket(cc);
+ cc-in_socket = false;
}
else if (!strcmp(element_name, Geo)) {
@@ -272,7 +272,7 @@ static void cfg_elm_end (GMarkupParseContext *context,
return;
}
- if (cc-in_storage) {
+ if (cc-in_socket) {
n = strtol(cc-text, NULL, 10);
if (n 0 n 65536) {
free(cc-stor_port);
@@ -296,7 +296,7 @@ static void cfg_elm_end (GMarkupParseContext *context,
return;
}
- if (cc-in_storage) {
+ if (cc-in_socket) {
free(cc-stor_host);
cc-stor_host = cc-text;
cc-text = NULL;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch tabled 4/8] Split out chunk back-end
This patch is careful not to change anything but the strictly necessary
parts in the actual code.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/Makefile.am |3
server/stor_chunk.c | 409 ++
server/storage.c| 377 --
server/tabled.h |3
4 files changed, 414 insertions(+), 378 deletions(-)
commit 1eaf18ff243a2d4e291d0af2f13ec425ab0757c0
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Nov 11 16:30:52 2010 -0700
Split out the chunk-specific part, byte for byte, no changes.
diff --git a/server/Makefile.am b/server/Makefile.am
index 5b53a0a..52beec4 100644
--- a/server/Makefile.am
+++ b/server/Makefile.am
@@ -5,7 +5,8 @@ sbin_PROGRAMS = tabled tdbadm
tabled_SOURCES = tabled.h \
bucket.c cldu.c config.c metarep.c object.c replica.c \
- server.c status.c storage.c storparse.c util.c
+ server.c status.c storage.c storparse.c \
+ stor_chunk.c util.c
tabled_LDADD = ../lib/libtdb.a \
@HAIL_LIBS@ @PCRE_LIBS@ @GLIB_LIBS@ \
@CRYPTO_LIBS@ @DB4_LIBS@ @EVENT_LIBS@ @SSL_LIBS@
diff --git a/server/stor_chunk.c b/server/stor_chunk.c
new file mode 100644
index 000..815adcf
--- /dev/null
+++ b/server/stor_chunk.c
@@ -0,0 +1,409 @@
+
+/*
+ * Copyright 2010 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING. If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ */
+
+#define _GNU_SOURCE
+#include tabled-config.h
+
+#include sys/types.h
+// #include sys/socket.h
+#include errno.h
+#include syslog.h
+#include string.h
+#include glib.h
+#include event.h
+#include chunkc.h
+#include netdb.h
+#include tabled.h
+
+static const char stor_key_fmt[] = %016llx;
+#define STOR_KEY_SLEN 16
+
+static int stor_new_stc(struct storage_node *stn, struct st_client **stcp)
+{
+ struct st_client *stc;
+ struct sockaddr_in *a4;
+ struct sockaddr_in6 *a6;
+ unsigned short port;
+
+ if (stn-addr.sin6_family == AF_INET) {
+ a4 = (struct sockaddr_in *) stn-addr;
+ port = ntohs(a4-sin_port);
+ } else if (stn-addr.sin6_family == AF_INET6) {
+ a6 = stn-addr;
+ port = ntohs(a6-sin6_port);
+ } else {
+ return -EINVAL;
+ }
+
+ stc = stc_new(stn-hostname, port,
+ tabled_srv.chunk_user, tabled_srv.chunk_key,
+ false);
+ if (!stc)
+ return -EDOM;
+
+ if (!stc_table_openz(stc, tabled, CHF_TBL_CREAT)) {
+ stc_free(stc);
+ return -EDOM;
+ }
+
+ *stcp = stc;
+ return 0;
+}
+
+static void stor_read_event(int fd, short events, void *userdata)
+{
+ struct open_chunk *cep = userdata;
+
+ cep-r_armed = false; /* no EV_PERSIST */
+ if (cep-ocb)
+ (*cep-ocb)(cep);
+}
+
+static void stor_write_event(int fd, short events, void *userdata)
+{
+ struct open_chunk *cep = userdata;
+
+ cep-w_armed = false; /* no EV_PERSIST */
+ if (cep-ocb)
+ (*cep-ocb)(cep);
+}
+
+/*
+ * Open *cep using stn, set up chunk session if needed.
+ */
+static int chunk_open(struct open_chunk *cep, struct storage_node *stn,
+ struct event_base *ev_base)
+{
+ int rc;
+
+ if (cep-stc)
+ return 0;
+
+ if ((rc = stor_new_stc(stn, cep-stc)) 0)
+ return rc;
+
+ cep-evbase = ev_base;
+ cep-node = stor_node_get(stn);
+
+ /* cep-stc-verbose = 1; */
+
+ return 0;
+}
+
+static int chunk_put_start(struct open_chunk *cep,
+ void (*cb)(struct open_chunk *),
+ uint64_t key, uint64_t size)
+{
+ char stckey[STOR_KEY_SLEN+1];
+
+ if (cep-key)
+ return -EBUSY;
+ if (!cep-stc)
+ return -EINVAL;
+
+ /*
+* Set up the putting.
+*/
+ sprintf(stckey, stor_key_fmt, (unsigned long long) key);
+ if (!stc_put_startz(cep-stc, stckey, size, cep-wfd, 0)) {
+ if (debugging)
+ applog(LOG_INFO,
+ stor nid %u put %s new for %lld error,
+ cep-node-id, stckey, (long long) size);
+ return
[patch tabled 6/8] Add filesystem back-end
This patch adds the first new back-end and makes some changes to the way
nodes are added, to make the invariants of storage_node more sensible.
The filesystem back-end itself is not intended for production use,
so it makes no attempt to run any asynchronous transfers.
We also add a test. Note that this differs from the preliminary versions
of this patch. We used to add both chunk and fs back-ends, so that tabled
replicates to both. This makes sense as a test of store path, but on
retrieval tabled selects any one of available storage nodes with the
object, randomly. It creates gaps in test coverage in any given run.
Therefore, we test two back-end types sequentially now.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/Makefile.am |2
server/stor_chunk.c | 21 -
server/stor_fs.c | 498 +
server/storage.c | 157 ++--
server/storparse.c | 97 +++
server/tabled.h | 31 ++
test/Makefile.am |3
test/be_fs-test.conf |5
test/combo-redux | 74 ++
test/prep-db |4
test/start-daemon|1
test/stop-daemon |9
12 files changed, 835 insertions(+), 67 deletions(-)
commit bccedeedabbe713e4053afa185314b3f57f3d204
Author: Pete Zaitcev zait...@yahoo.com
Date: Sun Nov 28 17:58:05 2010 -0700
Add fs back-end, with a test.
diff --git a/server/Makefile.am b/server/Makefile.am
index 52beec4..71bcb35 100644
--- a/server/Makefile.am
+++ b/server/Makefile.am
@@ -6,7 +6,7 @@ sbin_PROGRAMS = tabled tdbadm
tabled_SOURCES = tabled.h \
bucket.c cldu.c config.c metarep.c object.c replica.c \
server.c status.c storage.c storparse.c \
- stor_chunk.c util.c
+ stor_chunk.c stor_fs.c util.c
tabled_LDADD = ../lib/libtdb.a \
@HAIL_LIBS@ @PCRE_LIBS@ @GLIB_LIBS@ \
@CRYPTO_LIBS@ @DB4_LIBS@ @EVENT_LIBS@ @SSL_LIBS@
diff --git a/server/stor_chunk.c b/server/stor_chunk.c
index 815adcf..7462a9c 100644
--- a/server/stor_chunk.c
+++ b/server/stor_chunk.c
@@ -31,8 +31,7 @@
#include netdb.h
#include tabled.h
-static const char stor_key_fmt[] = %016llx;
-#define STOR_KEY_SLEN 16
+static const char stor_key_fmt[] = STOR_KEY_FMT;
static int stor_new_stc(struct storage_node *stn, struct st_client **stcp)
{
@@ -66,24 +65,6 @@ static int stor_new_stc(struct storage_node *stn, struct
st_client **stcp)
return 0;
}
-static void stor_read_event(int fd, short events, void *userdata)
-{
- struct open_chunk *cep = userdata;
-
- cep-r_armed = false; /* no EV_PERSIST */
- if (cep-ocb)
- (*cep-ocb)(cep);
-}
-
-static void stor_write_event(int fd, short events, void *userdata)
-{
- struct open_chunk *cep = userdata;
-
- cep-w_armed = false; /* no EV_PERSIST */
- if (cep-ocb)
- (*cep-ocb)(cep);
-}
-
/*
* Open *cep using stn, set up chunk session if needed.
*/
diff --git a/server/stor_fs.c b/server/stor_fs.c
new file mode 100644
index 000..b433a67
--- /dev/null
+++ b/server/stor_fs.c
@@ -0,0 +1,498 @@
+
+/*
+ * Copyright 2010 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING. If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ */
+
+#define _GNU_SOURCE
+#include tabled-config.h
+
+#include sys/types.h
+#include sys/stat.h
+#include errno.h
+#include fcntl.h
+#include syslog.h
+#include string.h
+#include glib.h
+#include event.h
+#include tabled.h
+
+static const char stor_key_fmt[] = STOR_KEY_FMT;
+
+static char *fs_obj_pathname(const char *base, uint64_t key)
+{
+ enum { PREFIX_LEN = 3 };
+ char prefix[PREFIX_LEN + 1];
+ char stckey[STOR_KEY_SLEN+1];
+ char *s;
+ int rc;
+
+ /* we know that stckey is going to be longer than PREFIX_LEN */
+ sprintf(stckey, stor_key_fmt, (unsigned long long) key);
+ memcpy(prefix, stckey, PREFIX_LEN);
+ prefix[PREFIX_LEN] = 0;
+
+ rc = asprintf(s, %s/%s/%s, base, prefix, stckey + PREFIX_LEN);
+ if (rc 0)
+ goto err_out;
+
+ return s;
+
+err_out:
+ return NULL;
+}
+
+static char *fs_ctl_pathname(const char *base, const char *file)
+{
+ char *s;
+ int rc;
+
+ rc = asprintf(s, %s/%s, base, file);
+ if (rc 0)
+ return NULL;
+ return s
[patch tabled 8/8] Add Swift back-end
This patch allows to use tabled with OpenStack Swift object store as if it
were our chunkserver, with some extra tricks. The configuration has to be
entred manually into CLD, just like in case of filesystem back-end.
The code is fairly experimental, so it retains extra messages.
Also, since Swift authorizes by plaintext passwords, support for SSL is
essential, but is currently missing.
There is no build-time test for this, because it would require us to
depend on OpenStack, which is untenable.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
doc/etc.tabled.conf |5
server/Makefile.am |7
server/object.c |2
server/replica.c |4
server/server.c |8
server/stor_swift.c | 1420 +
server/storage.c |3
server/storparse.c |7
server/tabled.h | 30
test/swift-test.conf |8
10 files changed, 1487 insertions(+), 7 deletions(-)
commit f24e2bd00d10f074ca9c6f62794b0838bc2de9b5
Author: Pete Zaitcev zait...@yahoo.com
Date: Sun Nov 28 18:07:44 2010 -0700
Add Swift back-end. Sadly, no test.
diff --git a/doc/etc.tabled.conf b/doc/etc.tabled.conf
index 5112e8a..6e82fe4 100644
--- a/doc/etc.tabled.conf
+++ b/doc/etc.tabled.conf
@@ -43,8 +43,9 @@
/CLD
--
-ChunkUsertest/ChunkUser
-ChunkKeytest/ChunkKey
+!-- swift-auth-add-user -K devauth -a tabled tester1 tester1 --
+ChunkUsertester1/ChunkUser
+ChunkKeytester1/ChunkKey
!-- Monitoring interface. Best is not to expose this to Internet. --
StatusPort8084/StatusPort
diff --git a/server/Makefile.am b/server/Makefile.am
index 71bcb35..5651ec2 100644
--- a/server/Makefile.am
+++ b/server/Makefile.am
@@ -1,15 +1,16 @@
-INCLUDES = -I$(top_srcdir)/include @GLIB_CFLAGS@ @HAIL_CFLAGS@
+INCLUDES = -I$(top_srcdir)/include \
+ @LIBCURL_CPPFLAGS@ @GLIB_CFLAGS@ @HAIL_CFLAGS@
sbin_PROGRAMS = tabled tdbadm
tabled_SOURCES = tabled.h \
bucket.c cldu.c config.c metarep.c object.c replica.c \
server.c status.c storage.c storparse.c \
- stor_chunk.c stor_fs.c util.c
+ stor_chunk.c stor_fs.c stor_swift.c util.c
tabled_LDADD = ../lib/libtdb.a \
@HAIL_LIBS@ @PCRE_LIBS@ @GLIB_LIBS@ \
- @CRYPTO_LIBS@ @DB4_LIBS@ @EVENT_LIBS@ @SSL_LIBS@
+ @CRYPTO_LIBS@ @DB4_LIBS@ @EVENT_LIBS@ @LIBCURL@ @SSL_LIBS@
tdbadm_SOURCES = tdbadm.c
tdbadm_LDADD = ../lib/libtdb.a @GLIB_LIBS@ @DB4_LIBS@
diff --git a/server/object.c b/server/object.c
index 2920811..44f8d5c 100644
--- a/server/object.c
+++ b/server/object.c
@@ -700,6 +700,8 @@ static struct open_chunk *open_chunk1(struct storage_node
*stnode,
applog(LOG_ERR, OOM);
goto err_alloc;
}
+ INIT_LIST_HEAD(ochunk-evt_list);
+ INIT_LIST_HEAD(ochunk-buf_list);
rc = stor_open(ochunk, stnode, tabled_srv.evbase_main);
if (rc != 0) {
diff --git a/server/replica.c b/server/replica.c
index 7c31112..0d3f7a2 100644
--- a/server/replica.c
+++ b/server/replica.c
@@ -90,6 +90,10 @@ static struct rep_job *job_alloc(size_t klen, struct
db_obj_key *key)
job = malloc(len);
if (job) {
memset(job, 0, sizeof(struct rep_job));
+ INIT_LIST_HEAD(job-in_ce.evt_list);
+ INIT_LIST_HEAD(job-in_ce.buf_list);
+ INIT_LIST_HEAD(job-out_ce.evt_list);
+ INIT_LIST_HEAD(job-out_ce.buf_list);
memcpy(job+1, key, klen);
job-klen = klen;
job-key = (struct db_obj_key *)(job+1);
diff --git a/server/server.c b/server/server.c
index ba3e00a..de6975f 100644
--- a/server/server.c
+++ b/server/server.c
@@ -46,6 +46,7 @@
#include openssl/md5.h
#include openssl/hmac.h
#include openssl/ssl.h
+#include curl/curl.h
#include elist.h
#include chunkc.h
#include cldc.h
@@ -1245,6 +1246,8 @@ static struct client *cli_alloc(bool is_status)
applog(LOG_ERR, out of memory);
return NULL;
}
+ INIT_LIST_HEAD(cli-in_ce.evt_list);
+ INIT_LIST_HEAD(cli-in_ce.buf_list);
atcp_wr_init(cli-wst, libevent_wr_ops, cli-write_ev, cli);
@@ -2079,6 +2082,11 @@ int main (int argc, char *argv[])
SSL_library_init();
SSL_load_error_strings();
+ if (curl_global_init(CURL_GLOBAL_ALL)) {
+ fprintf(stderr, curl_global_init failed\n);
+ return 1;
+ }
+
stc_init();
cld_init();
diff --git a/server/stor_swift.c b/server/stor_swift.c
new file mode 100644
index 000..253c92c
--- /dev/null
+++ b/server/stor_swift.c
@@ -0,0 +1,1420 @@
+
+/*
+ * Copyright 2010 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed
[patch tabled 7/8] Clenaup stor_chunk.c
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/stor_chunk.c |6 ++
1 file changed, 2 insertions(+), 4 deletions(-)
commit ca0920cfe5978839ee1a35d6096754a87db6f9ac
Author: Pete Zaitcev zait...@yahoo.com
Date: Sun Nov 28 18:01:24 2010 -0700
Cleanup stor_chunk.
diff --git a/server/stor_chunk.c b/server/stor_chunk.c
index 7462a9c..9111750 100644
--- a/server/stor_chunk.c
+++ b/server/stor_chunk.c
@@ -21,14 +21,12 @@
#include tabled-config.h
#include sys/types.h
-// #include sys/socket.h
#include errno.h
#include syslog.h
#include string.h
#include glib.h
#include event.h
#include chunkc.h
-#include netdb.h
#include tabled.h
static const char stor_key_fmt[] = STOR_KEY_FMT;
@@ -117,7 +115,7 @@ static int chunk_put_start(struct open_chunk *cep,
event_base_set(cep-evbase, cep-wevt);
if (debugging)
- applog(LOG_INFO, stor nid %u put %s new for %lld,
+ applog(LOG_INFO, stor nid %u put %s size %lld,
cep-node-id, stckey, (long long) size);
return 0;
@@ -249,7 +247,7 @@ static ssize_t chunk_put_buf(struct open_chunk *cep, void
*data, size_t len)
int rc;
if (cep-done + len cep-size) {
- applog(LOG_ERR, Put size %ld remaining %ld,
+ /* P3 */ applog(LOG_ERR, Put length %ld remaining %ld,
(long) len, (long) (cep-size - cep-done));
if (cep-done == cep-size)
return -EIO;/* will spin otherwise, better error */
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch hail 1/2] use hail_log.h
I think forward declarations are not needed if we have a nice leaf header.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
include/cld_common.h |3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
commit 12f0fd5e0c6925299617082d834220396fae252e
Author: Pete Zaitcev zait...@yahoo.com
Date: Wed Nov 10 12:38:14 2010 -0700
Cleanup for hail_log.
diff --git a/include/cld_common.h b/include/cld_common.h
index 3ac18c6..84b1ec6 100644
--- a/include/cld_common.h
+++ b/include/cld_common.h
@@ -27,11 +27,10 @@
#include glib.h
#include openssl/sha.h
#include cld_msg_rpc.h
+#include hail_log.h
#define CLD_ALIGN8(n) ((8 - ((n) 7)) 7)
-struct hail_log;
-
struct cld_timer {
boolfired;
boolon_list;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch hail 2/2] add -i to cldcli
This is purely cosmetic so that tabled build logs do not show useless
prompts. The way -i works copies common shells: one cannot force
non-interactive session.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
tools/cldcli.c | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)
commit ee2364add99af233a8fa5e9773a31087eb9caf00
Author: Pete Zaitcev zait...@yahoo.com
Date: Wed Nov 10 13:08:33 2010 -0700
Suppress prompts in cldcli when piped.
diff --git a/tools/cldcli.c b/tools/cldcli.c
index ff668ab..d347bf3 100644
--- a/tools/cldcli.c
+++ b/tools/cldcli.c
@@ -42,6 +42,8 @@ static struct argp_option options[] = {
Set debug output to LEVEL (0 = off, 2 = max verbose) },
{ host, 'h', HOST:PORT, 0,
Connect to remote CLD at specified HOST:PORT },
+ { interactive, 'i', NULL, 0,
+ Force an interactive session },
{ user, 'u', USER, 0,
Set username to USER },
{ verbose, 'v', NULL, 0,
@@ -75,6 +77,7 @@ struct cldcli_lock_info {
static GList *host_list;
static char clicwd[CLD_PATH_MAX + 1] = /;
static char our_user[CLD_MAX_USERNAME + 1] = cli_user;
+static int interactive = -1;
/* globals only for use in thread */
static struct ncld_sess *nsess;
@@ -646,6 +649,9 @@ static error_t parse_opt (int key, char *arg, struct
argp_state *state)
if (!push_host(arg))
argp_usage(state);
break;
+ case 'i':
+ interactive = 1;
+ break;
case 'u':
if (strlen(arg) = CLD_MAX_USERNAME) {
fprintf(stderr, TAG : invalid user: '%s'\n, arg);
@@ -667,6 +673,8 @@ static error_t parse_opt (int key, char *arg, struct
argp_state *state)
static void prompt(void)
{
+ if (!interactive)
+ return;
printf([%s %s]$ , our_user, clicwd);
fflush(stdout);
}
@@ -691,6 +699,9 @@ int main (int argc, char *argv[])
return 1;
}
+ if (interactive 0)
+ interactive = isatty(0);
+
if (!host_list) {
enum { hostsz = 64 };
char hostb[hostsz];
@@ -724,7 +735,8 @@ int main (int argc, char *argv[])
return 1;
}
- printf(Type 'help' at the prompt to list commands.\n);
+ if (interactive)
+ printf(Type 'help' at the prompt to list commands.\n);
prompt();
while (fgets(linebuf, sizeof(linebuf), stdin) != NULL) {
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[patch hail] remove duplicated stc_readport
Now that we have a common library for Hail, an opportunity opens to trim
some duplication, such as stc_readport. It even had a comment about it.
Note that we leave cld_readport in the API for a few weeks, while I get
my tabled trees and RPMs in order. Unfortunately we routinely neglect
to set specific version in RPM headers (e.g. no Requires: cld = 0.8.2).
Also, get rid of g_file_get_contents. Talk about pointless: it requires
caller to free memory, and it's not like code is any more compact or
easier to understand.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
chunkd/config.c |4 +--
include/chunkc.h |2 -
include/cld_common.h |3 +-
lib/chunkdc.c| 29 --
lib/common.c | 36 +++--
test/chunkd/auth.c |3 +-
test/chunkd/basic-object.c |3 +-
test/chunkd/cp.c |3 +-
test/chunkd/get-part.c |3 +-
test/chunkd/it-works.c |3 +-
test/chunkd/large-object.c |3 +-
test/chunkd/lotsa-objects.c |3 +-
test/chunkd/nop.c|3 +-
test/chunkd/selfcheck-unit.c |3 +-
test/cld/basic-io.c |2 -
test/cld/basic-session.c |2 -
test/cld/lock-file.c |2 -
17 files changed, 46 insertions(+), 61 deletions(-)
commit 56536e377f30a130f621beaa22dd90cb10e29d45
Author: Pete Zaitcev zait...@yahoo.com
Date: Tue Oct 26 13:39:50 2010 -0600
hail_readport.
diff --git a/chunkd/config.c b/chunkd/config.c
index 4c58768..c74e641 100644
--- a/chunkd/config.c
+++ b/chunkd/config.c
@@ -239,11 +239,11 @@ static void cfg_elm_end_cld(struct config_context *cc)
*/
if (cc-cld_port_file) {
int port;
- if ((port = cld_readport(cc-cld_port_file)) = 0) {
+ if ((port = hail_readport(cc-cld_port_file)) = 0) {
applog(LOG_INFO, Waiting for CLD PortFile %s,
cc-cld_port_file);
sleep(2);
- while ((port = cld_readport(cc-cld_port_file)) = 0)
+ while ((port = hail_readport(cc-cld_port_file)) = 0)
sleep(3);
applog(LOG_INFO, Using CLD port %u, port);
}
diff --git a/include/chunkc.h b/include/chunkc.h
index e3c2bb7..e9b6003 100644
--- a/include/chunkc.h
+++ b/include/chunkc.h
@@ -112,8 +112,6 @@ extern bool stc_check_status(struct st_client *stc,
extern struct st_keylist *stc_keys(struct st_client *stc);
-extern int stc_readport(const char *fname);
-
static inline void *stc_get_inlinez(struct st_client *stc,
const char *key,
size_t *len)
diff --git a/include/cld_common.h b/include/cld_common.h
index 8c11601..3ac18c6 100644
--- a/include/cld_common.h
+++ b/include/cld_common.h
@@ -64,7 +64,8 @@ static inline void cld_timer_init(struct cld_timer *timer,
const char *name,
extern unsigned long long cld_sid2llu(const uint8_t *sid);
extern void cld_rand64(void *p);
extern const char *cld_errstr(enum cle_err_codes ecode);
-extern int cld_readport(const char *fname);
+extern int cld_readport(const char *fname);/* deprecated */
+extern int hail_readport(const char *fname);
/*** Validate the HMAC signature of a byte buffer.
*
diff --git a/lib/chunkdc.c b/lib/chunkdc.c
index bebf87b..02cf850 100644
--- a/lib/chunkdc.c
+++ b/lib/chunkdc.c
@@ -1303,32 +1303,3 @@ void stc_init(void)
srand(time(NULL) ^ getpid()); // for cld_rand64 et.al.
}
-/*
- * Read a port number from a port file, return the value or negative error.
- * A 100% copy of cld_readport for now, but permits us not to link libcldc.
- */
-int stc_readport(const char *fname)
-{
- enum { LEN = 11 };
- char buf[LEN+1];
- long port;
- int fd;
- int rc;
-
- if ((fd = open(fname, O_RDONLY)) == -1)
- return -errno;
- rc = read(fd, buf, LEN);
- close(fd);
- if (rc 0)
- return -errno;
- if (rc == 0)
- return -EPIPE;
- buf[rc] = 0;
-
- port = strtol(buf, NULL, 10);
- if (port = 0 || port = 65636)
- return -EDOM;
-
- return (int)port;
-}
-
diff --git a/lib/common.c b/lib/common.c
index 30bc6dd..60b120d 100644
--- a/lib/common.c
+++ b/lib/common.c
@@ -81,27 +81,33 @@ const char *cld_errstr(enum cle_err_codes ecode)
/*
* Read a port number from a port file, return the value or negative error.
*/
-int cld_readport(const char *fname)
+int hail_readport(const char *fname)
{
+ enum { LEN = 11 };
+ char buf[LEN+1];
long port;
- gchar *buf;
- GError *err = NULL;
- gsize len;
-
- if (!g_file_get_contents(fname, buf, len, err)) {
- int ret = -1000 - err-code;
- g_error_free(err);
- return ret
[patch tabled] distribute the sample tabled.conf
The doc/etc.tabled.conf is a vital piece of documentation but we forgot
to include it into the actual distribution. It was only available for
those pulling the git tree, not tarballs.
Also update the sample spec.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
doc/Makefile.am |1 +
pkg/tabled.spec |2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
commit 049a3242bd8a20837fbc7d8f091e19b447c364f1
Author: Pete Zaitcev zait...@yahoo.com
Date: Fri Oct 15 16:45:34 2010 -0600
Add the sample conf for distribution.
diff --git a/doc/Makefile.am b/doc/Makefile.am
index e344e04..d79a860 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -1,6 +1,7 @@
EXTRA_DIST = \
contributions.txt \
+ etc.tabled.conf \
setup.txt \
spec-todo.txt \
states.txt
diff --git a/pkg/tabled.spec b/pkg/tabled.spec
index ac93fb9..98004e6 100644
--- a/pkg/tabled.spec
+++ b/pkg/tabled.spec
@@ -83,7 +83,7 @@ fi
%files
%defattr(-,root,root,-)
-%doc AUTHORS COPYING LICENSE README NEWS doc/*.txt
+%doc AUTHORS COPYING LICENSE README NEWS doc/*.txt doc/etc.tabled.conf
%{_sbindir}/tabled
%{_sbindir}/tdbadm
%attr(0755,root,root) %{_initddir}/tabled
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 1/2] Change cfgfile.txt into a real config file
Not many people read Recipies page at the wiki, so let's change the in-tree cfgfile.txt so that it can be copied into /etc. Signed-off-by: Pete Zaitcev zait...@redhat.com --- doc/cfgfile.txt | 57 +++--- 1 file changed, 19 insertions(+), 38 deletions(-) diff --git a/doc/cfgfile.txt b/doc/cfgfile.txt index 6c55c9b..deb1507 100644 --- a/doc/cfgfile.txt +++ b/doc/cfgfile.txt @@ -1,8 +1,10 @@ - +!-- chunkd's configuration file is XML-like, but not strictly XML. +In particular, clause tags are case-sensitive. +Anyhow, just copy this file into /etc/chunkd.conf and supply it to -C option. +-- -listen addresses - +!-- You may specify multiple port, and/or multiple (address,port) combinations using Listen, Node, Port, and Encrypt: @@ -28,47 +30,26 @@ using Listen, Node, Port, and Encrypt: Port18080/Port Encrypttrue/Encrypt /Listen +-- +Listen Port8082/Port /Listen +!-- The default is usually acceptable -- +!-- PID/var/run/chunkd.pid/PID -- -Listen Record delimiter -Node Node address to which the TCP server will bind - (exclude Node for wildcard address) -Port Numeric port or service, to which TCP server will bind -Encrypt TLS is enabled on this port - - -volume served -= -You specify the volume served by this daemon using Path. - -Your basic setup, an empty directory: - Path/disk/massive/data/Path +Path/q/chunk-vega/Path !-- any /home directory will do -- +!-- Anything unique works: digits of IP address, time_t of creation. -- +NID13/NID -Path Path to volume +!-- + If you run tabled, vega is a Group tag where looks for chunks. + Otherwise, any path in CLD namespace works. + -- +InfoPath/chunk-vega/13/InfoPath - -SSL settings - -You specify server SSL attributes using SSL, PrivateKey, and Cert. - -Your basic setup: +!-- SSL works, although very few people/programs use it. Tabled doesn't. SSL PrivateKey/etc/pki/chunkd.pem/PrivateKey Cert/etc/pkt/cert.pem/Cert /SSL - - -SSL Record delimiter -PrivateKey Pathname of SSL private key PEM file -Cert Pathname of Certificate public key PEM file - - -miscellaneous settings -== - -PID file contained the process id of the daemon after forking, normally -written somewhere in /var/run: - - PID/var/run/chunkd.pid/PID - +-- -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Autostart
Jeff et.al.: I am working on a feature called autostart in Image Warehouse. It amounts to iwhd starting with a certain default configuration, but is a bit more: it also launches services necessary to support said default, e.g. tabled. Naturally, tabled needs cld and chunkd. An interesting question is what to do when iwhd exits. I decided not to kill what was started. So, we have a little self-contained cell of tabled, chunkd, S3, based off a certain local directory or other namespace anchor. Therefore, when iwhd restarts, it tests if the cell is still there, and uses that. It also tests if the service started successfuly, using the same method. As we see, for each service iwhd starts, it needs to verify that it's available (either before spawning it, or after). This would be done best by talking to the service. But iwhd only has S3 client, and no CLD client, so it cannot talk to cld (or chunkd). I had an idea: add an autostart feature to tabled. Tabled knows how to talk to both chunkd and cld, so it can verify that they are running. It would not be that much code. The downside is that it's clearly a special case, encoding of a policy. So I am asking how objectionable it would be (including do we want tabled -a for tests... they kinda run ok as they are). Cheers, -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam
On Mon, 27 Sep 2010 10:53:06 +0200 Jim Meyering j...@meyering.net wrote: - stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK); + v = strdup(val); + stmp = huri_field_escape(v, QUERY_ESCAPE_MASK); str = g_string_append(str, stmp); free(stmp); + free(v); I think you may be fooled by the ridiculous calling convention of huri_field_escape(). It takes a pointer to heap, then either returns its argument, or reallocates it, frees the argument, and returns the reallocated area. It frees with g_free, so it assumes its equivalence with free(), haha. The end result, it either returns what strdup returned of frees it. Therefore if you free what strudup returned, you double-free it. I honestly think this madness must stop and huri_field_escape must allocate a new buffer every time. Then we would not need the strdup there at all. It only exists to satisfy the requirement to pass a pointer to heap in case val is a const or whatnot. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH hail] lib/hstor.c: avoid an unconditional leak in append_qparam
On Mon, 27 Sep 2010 12:53:48 -0400 Jeff Garzik j...@garzik.org wrote: - stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK); + v = strdup(val); + stmp = huri_field_escape(v, QUERY_ESCAPE_MASK); str = g_string_append(str, stmp); free(stmp); + free(v); applied I'm going to post a patch that undoes the damage. It's in testing. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: On licensing and libhail
On Tue, 21 Sep 2010 17:04:41 -0400 Jeff Garzik j...@garzik.org wrote: Occasionally I receive a ping about libhail licensing. GPL is probably less convenient than LGPL, so I was wondering what existing copyright holders thought about changing libhail to LGPL, while leaving the remaining bits GPL? Looks like Colin is the only one. I did my work on Red Hat's money, so it's their call. Just need to ask whoever replaced Mark Webbink. And we didn't have any other recorded contributors. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: stor_obj_test
On Wed, 07 Jul 2010 00:02:02 -0400 Jeff Garzik j...@garzik.org wrote: This function seems to be missing the meat. It retrieves then disposes of a keylist. bool stor_obj_test(struct open_chunk *cep, uint64_t key) I do not understand what kind of meat you expect stor_obj_test to have. In the filesystem backend this was a stat() that verified if the just-stored object existed and that was it. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[tabled patch 3/5] cleanup a call to closelog()
Since we're on it, only call closelog() if openlog() was called. There is no crash if we do that, but still it's not very correct. Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/server.c |3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit ae8ac067adde81a34c9d6114cfbaa1f95c9b48da Author: Pete Zaitcev zait...@yahoo.com Date: Thu Aug 12 12:33:39 2010 -0600 Syslog cleanup. diff --git a/server/server.c b/server/server.c index 829d2db..7a9fb7a 100644 --- a/server/server.c +++ b/server/server.c @@ -2344,7 +2344,8 @@ err_evpipe: unlink(tabled_srv.pid_file); close(tabled_srv.pid_fd); err_out: - closelog(); + if (use_syslog) + closelog(); return rc; } -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[tabled patch 4/5] Support auto replicaton port
Allow random ports for replication master to listen on.
The patch is somewhat larger than expected, because before we had
the MASTER file written right after locking. Now we may have it
written without listening parameters, and the slaves must be
ready to deal with it.
Unlike the auto client port, we do not need to write any accessor
files, because we already report the host and port through CLD.
Listening on random ports has security implications.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
doc/etc.tabled.conf | 16 ++-
server/cldu.c | 166 ++--
server/config.c | 22 +++--
server/metarep.c | 34
server/tabled.h |1
test/tabled-test.conf |2
6 files changed, 173 insertions(+), 68 deletions(-)
commit 7ca8a587348e315ab7c6c9e32476d8fa387718d5
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Aug 12 12:37:27 2010 -0600
TDBRepPort auto.
diff --git a/doc/etc.tabled.conf b/doc/etc.tabled.conf
index c3b1d1d..5112e8a 100644
--- a/doc/etc.tabled.conf
+++ b/doc/etc.tabled.conf
@@ -12,14 +12,22 @@
/Listen
!--
- One group per DB, don't skimp on groups. Also, make sure the replication
- ports do not conflict when you make boxes to host several groups or use
- replication instances iwth TDBRepName.
+ One group per DB, don't skimp on groups.
--
Groupultracart2/Group
+
TDB/path/tabled-uc2//TDB!-- mkdir -p /path/tabled-uc2 --
-!-- TDBRepName12345.my_local_node_name.example.com/TDBRepName --
+
+!--
+ The usual practice is to set a fixed TDBRepPort (8083) because this
+ permits to configure a firewall easily. Remember that replication
+ has no authentication and authorization whatsoever for now!
+ When running two test instances on the same host, you may use auto.
+ But if so, do not forget to set replication instances with TDBRepName.
+ By default, a hostname serves fine as an instance name, port is auto.
+ --
TDBRepPort8083/TDBRepPort
+!-- TDBRepNameinst-b.my_local_node_name.example.com/TDBRepName --
!--
The clause CLD is not to be used in production configurations.
diff --git a/server/cldu.c b/server/cldu.c
index 45a6a83..57d486e 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -67,7 +67,6 @@ struct cld_session {
char *thisname;
char *thisgroup;
- char *thishost;
char *cfname; /* /tabled-group directory */
struct ncld_fh *cfh;/* /tabled-group directory, keep open for scan
*/
char *ffname; /* /tabled-group/thisname */
@@ -119,24 +118,17 @@ static int cldu_nextactive(struct cld_session *sp)
* chunkservers that it uses, so this function only takes one group argument.
*/
static int cldu_setgroup(struct cld_session *sp,
-const char *thisgroup, const char *thishost,
-const char *thisname)
+const char *thisgroup, const char *thisname)
{
char *mem;
if (thisgroup == NULL) {
thisgroup = default;
}
- if (thisname == NULL) {
- thisname = thishost;
- }
sp-thisgroup = strdup(thisgroup);
if (!sp-thisgroup)
goto err_oom;
- sp-thishost = strdup(thishost);
- if (!sp-thishost)
- goto err_oom;
sp-thisname = strdup(thisname);
if (!sp-thisname)
goto err_oom;
@@ -256,59 +248,82 @@ static void cldu_parse_master(const char *mfname, const
char *mfile, long len)
applog(LOG_DEBUG, %s: No name, mfname);
return;
}
+ if (namelen = sizeof(namebuf)) {
+ applog(LOG_ERR, Long master name);
+ return;
+ }
+ memcpy(namebuf, name, namelen);
+ namebuf[namelen] = 0;
+
if (!host || !hostlen) {
if (debugging)
applog(LOG_DEBUG, %s: No host, mfname);
- return;
+ hostlen = 0;
}
if (!port || !portlen) {
if (debugging)
applog(LOG_DEBUG, %s: No port, mfname);
- return;
+ portlen = 0;
}
- if (namelen = sizeof(namebuf)) {
- applog(LOG_ERR, Long master name);
- return;
- }
- memcpy(namebuf, name, namelen);
- namebuf[namelen] = 0;
+ if (hostlen != 0 portlen != 0) {
- if (hostlen = sizeof(hostbuf)) {
- applog(LOG_ERR, Long host);
- return;
- }
- memcpy(hostbuf, host, hostlen);
- hostbuf[hostlen] = 0;
+ if (hostlen = sizeof(hostbuf)) {
+ applog(LOG_ERR, Long host);
+ return;
+ }
+ memcpy(hostbuf, host, hostlen);
+ hostbuf[hostlen] = 0;
- if (portlen = sizeof(portbuf)) {
- applog(LOG_ERR, Long port);
- return
Re: [tabled patch 3/3] Fix metadata replication
On Tue, 10 Aug 2010 18:43:37 -0400 Jeff Garzik j...@garzik.org wrote: On 08/10/2010 05:19 PM, Pete Zaitcev wrote: Unfortunately this is getting a little behind, because I started working on the tests and they require some changes (e.g. the listening host and port may be unknown at the time MASTER file is locked). What does that mean -- I just applied the patch, should I wait for further updates before rolling a tarball? No, please don't. It's going to take all of today and probably tomorrow too... Perhaps even more. I meant to say, if you would not take this, I'll have updates soon. But with git pull it really does not matter to me, I can easily base any new patches on the current origin HEAD. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [hail patch 2/3] fix 32/64 wire interoperability
On Thu, 05 Aug 2010 17:25:30 -0400 Jeff Garzik j...@garzik.org wrote: I wonder if we shouldn't switch to attribute(packed) for safety, though. I dunno, as long as we test this... I irrationally hate the gcc attributes, although probably any viable competitor will have to support them. Talk about embrace-and-extend. Fine when FSF does it, not fine when Microsoft does, huh. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[tabled patch 2/3] Fix the flood of read obj(N)
Sometimes during operation tabled starts flooding a message read obj.
I thought it was something about replication thread fetching incomplete
objects, but actually it is caused by attempts to read from a closed
object. It is purely dictated by load levels.
This fix closes the obvious avenue, but leaves any potential ones.
I want to see if they ever hit.
Also, fix up messages a bit. In a redundant environment it is necessary
to know NIDs so that a broken Chunk can be isolated, so add that.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/object.c | 32
1 file changed, 24 insertions(+), 8 deletions(-)
commit 146b9efd8628343b7ecbfe879ce2c1b09e797bba
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Aug 5 20:40:29 2010 -0600
Fix floods of read obj(4FF1).
diff --git a/server/object.c b/server/object.c
index d495d6e..f8e7b12 100644
--- a/server/object.c
+++ b/server/object.c
@@ -356,10 +356,14 @@ static bool object_put_end(struct client *cli)
if (debugging) {
/* FIXME how do we test for inline objects
here? */
if (!stor_obj_test(ochunk, cli-out_objid))
- applog(LOG_ERR, Stat (%llX) failed,
+ applog(LOG_ERR,
+ Stat failed nid %u oid %llX,
+ ochunk-node-id,
(unsigned long long)
cli-out_objid);
else
- applog(LOG_DEBUG, STORED %llX, size -,
+ applog(LOG_DEBUG,
+ STORED nid %u oid %llX, size -,
+ ochunk-node-id,
(unsigned long long)
cli-out_objid);
}
obj_addr.nidv[nok] = GUINT32_TO_LE(ochunk-node-id);
@@ -705,8 +709,9 @@ static struct open_chunk *open_chunk1(struct storage_node
*stnode,
rc = stor_put_start(ochunk, object_put_event, objid, content_len);
if (rc != 0) {
- applog(LOG_WARNING, Cannot start putting for %llX (%d),
- (unsigned long long) objid, rc);
+ applog(LOG_WARNING,
+ Cannot start putting for nid %u oid %llX (%d),
+ ochunk-node-id, (unsigned long long) objid, rc);
goto err_start;
}
@@ -982,6 +987,7 @@ void cli_in_end(struct client *cli)
return;
stor_close(cli-in_ce);
+ cli-in_len = 0;
}
static bool object_get_more(struct client *cli, void *cb_data, bool done);
@@ -995,6 +1001,13 @@ static bool object_get_poke(struct client *cli)
char *buf;
ssize_t bytes;
+ /* The checks for in_len in caller should protect us, but let's see. */
+ if (!cli-in_ce.stc) {
+ applog(LOG_ERR, read on closed chunk, in_len %ld,
+ (long) cli-in_len);
+ return false;
+ }
+
/* XXX flow control - what treshold? */
/* if (cli_wqueued(cli) = 10) return false; */
@@ -1005,8 +1018,9 @@ static bool object_get_poke(struct client *cli)
bytes = stor_get_buf(cli-in_ce, buf,
MIN(cli-in_len, CLI_DATA_BUF_SZ));
if (bytes 0) {
- applog(LOG_ERR, read obj(%llX) failed,
- (unsigned long long) cli-in_objid);
+ applog(LOG_ERR, read failed nid %u oid %llX (%d),
+ cli-in_ce.node-id, (unsigned long long) cli-in_objid,
+ (int) bytes);
goto err_out;
}
if (bytes == 0) {
@@ -1048,6 +1062,8 @@ static bool object_get_more(struct client *cli, void
*cb_data, bool done)
/* do not queue more, if !completion or fd was closed early */
if (!done) /* FIXME We used to test for input errors here. */
return false;
+ if (!cli-in_len)
+ return false;
return object_get_poke(cli);/* won't hurt to try */
}
@@ -1323,8 +1339,8 @@ static bool object_get_body(struct client *cli, const
char *user,
bytes = stor_get_buf(cli-in_ce, buf, MIN(cli-in_len, sizeof(buf)));
if (bytes 0) {
- applog(LOG_ERR, read obj(%llX) failed,
- (unsigned long long) cli-in_objid);
+ applog(LOG_ERR, read failed nid %u oid %llX,
+ cli-in_ce.node-id, (unsigned long long) cli-in_objid);
goto err_out_in_end;
}
if (bytes == 0) {
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 3/3] Use a constant like elsewhere
Actually a competent compiler should replace this strlen with a
constant expression, so a performance win is unlikely, but why
bother if we already have this constant and use it in similar cases?
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
lib/chunkdc.c |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/chunkdc.c b/lib/chunkdc.c
index 3f1994e..13f5d36 100644
--- a/lib/chunkdc.c
+++ b/lib/chunkdc.c
@@ -139,9 +139,9 @@ static bool net_write(struct st_client *stc, const void
*data, size_t datalen)
static bool resp_valid(const struct chunksrv_resp *resp)
{
- if (memcmp(resp-magic, CHUNKD_MAGIC, strlen(CHUNKD_MAGIC)))
+ if (memcmp(resp-magic, CHUNKD_MAGIC, CHD_MAGIC_SZ))
return false;
-
+
return true;
}
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [hail patch 1/1] Make host, url, orig_path dynamic
On Tue, 20 Jul 2010 16:34:19 -0400 Jeff Garzik j...@garzik.org wrote: lib/hstor.c | 147 +++--- 1 file changed, 104 insertions(+), 43 deletions(-) applied It's not in the git repo. Check this URL: http://git.kernel.org/?p=daemon/distsrv/hail.git -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 2/7] Cleanup some in tdb.c
Some syntax sugar, but also make messages consistent.
I removed dbenv- prefix because it looked pretty goofy in logs.
Plus, make all messages so that they are unique and correspond
exactly to the failed function, for grepping.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
lib/tdb.c | 27 +--
1 file changed, 13 insertions(+), 14 deletions(-)
commit 55dbc1f81b0c40b8d274adb5a24f96616f9dee00
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Jul 29 19:20:53 2010 -0600
Cleanups: newlines, messages.
diff --git a/lib/tdb.c b/lib/tdb.c
index 12ff231..bc5e50a 100644
--- a/lib/tdb.c
+++ b/lib/tdb.c
@@ -75,7 +75,6 @@ retry:
env-err(env, rc, db_create);
return -EIO;
}
-
db = *db_out;
if (page_size) {
@@ -218,9 +217,9 @@ int tdb_init(struct tabledb *tdb, const char *db_home,
const char *db_password,
rc = db_env_create(tdb-env, 0);
if (rc) {
if (do_syslog)
- syslog(LOG_WARNING, tdb-env_create failed: %d, rc);
+ syslog(LOG_WARNING, db_env_create failed: %d, rc);
else
- fprintf(stderr, tdb-env_create failed: %d\n, rc);
+ fprintf(stderr, db_env_create failed: %d\n, rc);
return rc;
}
@@ -228,7 +227,6 @@ int tdb_init(struct tabledb *tdb, const char *db_home,
const char *db_password,
dbenv-app_private = tdb;
dbenv-set_errpfx(dbenv, errpfx);
-
if (do_syslog)
dbenv-set_errcall(dbenv, db4syslog);
else
@@ -247,14 +245,14 @@ int tdb_init(struct tabledb *tdb, const char *db_home,
const char *db_password,
*/
rc = dbenv-log_set_config(dbenv, DB_LOG_AUTO_REMOVE, 1);
if (rc) {
- dbenv-err(dbenv, rc, log_set_config);
+ dbenv-err(dbenv, rc, log_set_config(AUTO_REMOVE));
goto err_out;
}
if (db_password) {
rc = dbenv-set_encrypt(dbenv, db_password, DB_ENCRYPT_AES);
if (rc) {
- dbenv-err(dbenv, rc, dbenv-set_encrypt);
+ dbenv-err(dbenv, rc, set_encrypt);
goto err_out;
}
tdb-keyed = true;
@@ -262,32 +260,32 @@ int tdb_init(struct tabledb *tdb, const char *db_home,
const char *db_password,
rc = dbenv-repmgr_set_local_site(dbenv, rep_host, rep_port, 0);
if (rc) {
- dbenv-err(dbenv, rc, dbenv-set_local_site);
+ dbenv-err(dbenv, rc, repmgr_set_local_site);
goto err_out;
}
rc = dbenv-set_event_notify(dbenv, db4_event);
if (rc) {
- dbenv-err(dbenv, rc, dbenv-set_event_notify);
+ dbenv-err(dbenv, rc, set_event_notify);
goto err_out;
}
// rc = dbenv-rep_set_timeout(dbenv, DB_REP_LEASE_TIMEOUT, 1700);
// if (rc) {
- // dbenv-err(dbenv, rc, dbenv-rep_set_timeout(LEASE));
+ // dbenv-err(dbenv, rc, rep_set_timeout(LEASE));
// goto err_out;
// }
// Comment this out due to nsites must be zero if leases configured
// rc = dbenv-rep_set_config(dbenv, DB_REP_CONF_LEASE, 1);
// if (rc) {
- // dbenv-err(dbenv, rc, dbenv-rep_set_config);
+ // dbenv-err(dbenv, rc, rep_set_config(LEASE));
// goto err_out;
// }
rc = dbenv-rep_set_priority(dbenv, 100);
if (rc) {
- dbenv-err(dbenv, rc, dbenv-rep_set_priority);
+ dbenv-err(dbenv, rc, rep_set_priority);
goto err_out;
}
@@ -296,7 +294,7 @@ int tdb_init(struct tabledb *tdb, const char *db_home,
const char *db_password,
env_flags |= DB_INIT_TXN | DB_INIT_REP;
rc = dbenv-open(dbenv, db_home, env_flags, S_IRUSR | S_IWUSR);
if (rc) {
- dbenv-err(dbenv, rc, dbenv-open);
+ dbenv-err(dbenv, rc, open(dbenv));
goto err_out;
}
@@ -306,13 +304,13 @@ int tdb_init(struct tabledb *tdb, const char *db_home,
const char *db_password,
// rc = dbenv-rep_set_nsites(dbenv, nsites + 1);
// if (rc) {
- // dbenv-err(dbenv, rc, dbenv-repmgr_set_nsites);
+ // dbenv-err(dbenv, rc, rep_set_nsites);
// goto err_out;
// }
rc = dbenv-repmgr_start(dbenv, 2, DB_REP_ELECTION);
if (rc) {
- dbenv-err(dbenv, rc, dbenv-repmgr_start);
+ dbenv-err(dbenv, rc, repmgr_start);
goto err_out;
}
@@ -416,3 +414,4 @@ void tdb_fini(struct tabledb *tdb)
tdb-env-close(tdb-env, 0);
tdb-env = NULL;
}
+
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo
[hail patch 4/7] Go to 64K chunkd buffers
It only took one definition change, but there was a couple of
coattail riders.
- The batching-up of the writes is not really relevant and everything
works just fine without, but 64K made it obvious that it wasn't
actually working. It must be at least one buffer big, or the condition
never trips.
- Not exactly sure if I ever saw a hang due to half-buffer write
and a lost notification. Maybe. But thinking about it it's obvious
that we want a notification if any write is outstanding (rc nonzero).
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/object.c |2 +-
server/storage.c |2 +-
server/tabled.h |2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
commit e19aedbe09d5cd159ec6b86be4dea9958fba2905
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Jul 29 19:32:47 2010 -0600
64K I/O.
diff --git a/server/object.c b/server/object.c
index 207bb3e..d495d6e 100644
--- a/server/object.c
+++ b/server/object.c
@@ -1027,7 +1027,7 @@ static bool object_get_poke(struct client *cli)
} else {
if (cli_writeq(cli, buf, bytes, object_get_more, buf))
goto err_out;
- if (cli_wqueued(cli) = 4000)
+ if (cli_wqueued(cli) = CLI_DATA_BUF_SZ)
cli_write_start(cli);
}
return true;
diff --git a/server/storage.c b/server/storage.c
index a63012e..cac5a96 100644
--- a/server/storage.c
+++ b/server/storage.c
@@ -290,7 +290,7 @@ ssize_t stor_put_buf(struct open_chunk *cep, void *data,
size_t len)
if (!cep-stc)
return -EPIPE;
rc = stc_put_send(cep-stc, data, len);
- if (rc == 0 !cep-w_armed) {
+ if (rc len !cep-w_armed) {
event_add(cep-wevt, NULL);
cep-w_armed = true;
}
diff --git a/server/tabled.h b/server/tabled.h
index ae67838..ff419e3 100644
--- a/server/tabled.h
+++ b/server/tabled.h
@@ -49,7 +49,7 @@ enum {
CHUNK_REBOOT_TIME = 3*60, /* secs to declare chunk dead */
CLI_REQ_BUF_SZ = 8192, /* buffer for req + hdrs */
- CLI_DATA_BUF_SZ = 8192,
+ CLI_DATA_BUF_SZ = 65536,
};
enum errcode {
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 5/7] Drop a per-key message
Not a good idea on a working server with thousands of keys.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/replica.c |7 ---
1 file changed, 7 deletions(-)
commit cc860ab6272571320f6be4651107da90141dcfb8
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Jul 29 19:38:38 2010 -0600
Drop a per-key printout.
diff --git a/server/replica.c b/server/replica.c
index e774824..ac14cb2 100644
--- a/server/replica.c
+++ b/server/replica.c
@@ -601,13 +601,6 @@ static void rep_scan_verify(struct rep_arg *arg,
oid = GUINT64_FROM_LE(obj-d.a.oid);
- applog(LOG_INFO, bucket %s key %s oid %llX n(%u,%u,%u): all %d ok %d,
- bucket_name, object_name, (long long) oid,
- GUINT32_FROM_LE(obj-d.a.nidv[0]),
- GUINT32_FROM_LE(obj-d.a.nidv[1]),
- GUINT32_FROM_LE(obj-d.a.nidv[2]),
- allcnt, redcnt);
-
if (redcnt MAXWAY) { /* maybe have MINWAY too? */
rep_job_start(arg, cp-klen, cp-key, oid,
GUINT64_FROM_LE(obj-size), redcnt, redvec);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 6/7] byteswap sin_port
Signed-off-by: Pete Zaitcev zait...@redhat.com --- server/server.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 9d8c7992b91f969930643dd9ea94b6ba4f8aa6eb Author: Pete Zaitcev zait...@yahoo.com Date: Thu Jul 29 19:51:28 2010 -0600 Byteswap port. diff --git a/server/server.c b/server/server.c index 7950693..a160716 100644 --- a/server/server.c +++ b/server/server.c @@ -1644,7 +1644,7 @@ static int net_open_any(void) addr4.sin_addr.s_addr = htonl(INADDR_ANY); /* If IPv6 worked, we must use the same port number for IPv4 */ if (port) - addr4.sin_port = port; + addr4.sin_port = htons(port); fd4 = net_open_socket(AF_INET, SOCK_STREAM, 0, sizeof(addr4), addr4, false); -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 7/7] Listen on status earlier
This way status page is useful to show the database status even
when not a master (e.g. a slave).
Unfortunately it's still late to show what's going on while we
bootstrap the CLD client, because with ncld it's synchronous, but
we need the main loop to process events before HTTP service starts
working. But this fix is better than nothing.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/server.c | 48 ++
1 file changed, 40 insertions(+), 8 deletions(-)
commit d41bf1bab507111aac01e971928f4c8a7c876b2b
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Jul 29 20:25:33 2010 -0600
Split status listening from normal client listening.
diff --git a/server/server.c b/server/server.c
index a160716..0d6316d 100644
--- a/server/server.c
+++ b/server/server.c
@@ -1747,9 +1747,6 @@ static int net_open(void)
if (rc)
return rc;
- if (tabled_srv.status_port)
- net_open_known(tabled_srv.status_port, true);
-
if (tabled_srv.port_file) {
rc = net_write_port(tabled_srv.port_file,
tabled_srv.ourhost, tabled_srv.port);
@@ -1761,7 +1758,31 @@ static int net_open(void)
return 0;
}
-static void net_listen(void)
+static void net_listen_status(void)
+{
+ GList *tmp;
+
+ for (tmp = tabled_srv.sockets; tmp; tmp = tmp-next) {
+ struct server_socket *sock = tmp-data;
+
+ if (!sock-is_status)
+ continue;
+
+ if (listen(sock-fd, 10) 0) {
+ applog(LOG_WARNING, status socket listen: %s,
+ strerror(errno));
+ continue;
+ }
+
+ if (event_add(sock-ev, NULL) 0) {
+ applog(LOG_WARNING, status socket event_add error);
+ /* FIXME: There is no unlisten other than close. */
+ continue;
+ }
+ }
+}
+
+static void net_listen_client(void)
{
GList *tmp;
@@ -1771,14 +1792,21 @@ static void net_listen(void)
for (tmp = tabled_srv.sockets; tmp; tmp = tmp-next) {
struct server_socket *sock = tmp-data;
+ if (sock-is_status)
+ continue;
+
if (listen(sock-fd, 100) 0) {
- applog(LOG_WARNING, tcp socket listen: %s,
- strerror(errno));
+ if (debugging)
+ applog(LOG_DEBUG, client socket listen: %s,
+ strerror(errno));
continue;
}
+ if (debugging)
+ applog(LOG_DEBUG, client socket listen ok);
if (event_add(sock-ev, NULL) 0) {
- applog(LOG_WARNING, tcp socket event_add);
+ applog(LOG_WARNING, client socket event_add error);
+ /* FIXME: There is no unlisten other than close. */
continue;
}
}
@@ -1824,7 +1852,7 @@ static void tdb_state_process(enum st_tdb new_state)
}
add_chkpt_timer();
rep_start();
- net_listen();
+ net_listen_client();
}
}
@@ -1964,6 +1992,10 @@ int main (int argc, char *argv[])
}
/* set up server networking */
+ if (tabled_srv.status_port) {
+ if (net_open_known(tabled_srv.status_port, true) == 0)
+ net_listen_status();
+ }
rc = net_open();
if (rc)
goto err_out_net;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Amazon bucket-policy API extensions
On Thu, 08 Jul 2010 07:17:28 -0400 Jeff Darcy jda...@redhat.com wrote: Yet another thing for the to-do list, I guess. Yeah... Versions, too. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[hail patch 1/1] fix hstor crashes
This patch addresses two bugs:
- No parameters to search worked (because character '=' was missing,
e.g. delim/ instead of delim=/), and also there were weird truncations
because prefix can easily be longer than 32 characters.
- If prefix is empty, xmlNodeListGetString returns NULL instead of ,
thus segfault in strdup. This is not easy to accomplish with Boto,
but our own httpstor can do it.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
lib/hstor.c |6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
This patch was actually tested on httpstor, I've not switched to hstor
yet, but it seems to apply 100%. There's no sense in fixing httpstor,
I'm going to delete it from tabled soon.
diff --git a/lib/hstor.c b/lib/hstor.c
index 403aca3..5dfff27 100644
--- a/lib/hstor.c
+++ b/lib/hstor.c
@@ -621,13 +621,13 @@ bool hstor_del(struct hstor_client *hstor, const char
*bucket, const char *key)
static GString *append_qparam(GString *str, const char *key, const char *val,
char *arg_char)
{
- char *stmp, s[32];
+ char *stmp;
str = g_string_append(str, arg_char);
arg_char[0] = '';
- sprintf(s, %s=, key);
str = g_string_append(str, key);
+ str = g_string_append(str, =);
stmp = huri_field_escape(strdup(val), QUERY_ESCAPE_MASK);
str = g_string_append(str, stmp);
@@ -868,7 +868,7 @@ struct hstor_keylist *hstor_keys(struct hstor_client
*hstor, const char *bucket,
}
else if (!_strcmp(node-name, Prefix)) {
xs = xmlNodeListGetString(doc, node-children, 1);
- keylist-prefix = strdup((char *)xs);
+ keylist-prefix = strdup(xs? (char *)xs: );
xmlFree(xs);
}
else if (!_strcmp(node-name, Marker)) {
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] tabled: use httpstor API from libhail
On Wed, 7 Jul 2010 21:05:09 -0400 Jeff Garzik j...@garzik.org wrote: If anybody wants to recover the pre-libhail tabled.git tree, it can easily be done with Naah, I hate git branches (except the mandatory ones, of course). -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] chunkd: add cp command, for local intra-table copies
On Tue, 6 Jul 2010 03:24:29 -0400 Jeff Garzik j...@garzik.org wrote: The following patch, against current hail.git, adds the CP command to chunkd, permitting copying from object-object inside a single table. What is it for? -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New 'hail' repository created, with major packaging rework
On Fri, 02 Jul 2010 02:59:20 -0400 Jeff Garzik j...@garzik.org wrote: 4) ./autogen.sh 5) ./autogen.sh One worked for me. Trying to pretend it's nothing :-) 11) rpmbuild -ba SPECS/hail.spec Seems working perfectly, but there's one thing: Wrote: /q/zaitcev/rpms/SRPMS/hail-0.8-0.1.z1.fc13.src.rpm Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-0.8-0.1.z1.fc13.x86_64.rpm Wrote: /q/zaitcev/rpms/RPMS/x86_64/cld-0.8-0.1.z1.fc13.x86_64.rpm Wrote: /q/zaitcev/rpms/RPMS/x86_64/chunkd-0.8-0.1.z1.fc13.x86_64.rpm Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-devel-0.8-0.1.z1.fc13.x86_64.rpm Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-debuginfo-0.8-0.1.z1.fc13.x86_64.rpm The debuginfos for cld and chunkd are not produced for some reason. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New 'hail' repository created, with major packaging rework
On Mon, 05 Jul 2010 14:37:48 -0400 Jeff Garzik j...@garzik.org wrote: Wrote: /q/zaitcev/rpms/RPMS/x86_64/hail-debuginfo-0.8-0.1.z1.fc13.x86_64.rpm The debuginfos for cld and chunkd are not produced for some reason. Not produced... or simply contained within hail-debuginfo? Typically only one *-debuginfo.rpm is produced per SRPM, AFAIK. Now I'm curious how abrtd finds them. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New 'hail' repository created, with major packaging rework
On Fri, 02 Jul 2010 02:59:20 -0400 Jeff Garzik j...@garzik.org wrote: git://git.kernel.org/pub/scm/daemon/distsrv/hail.git libhail is a single shared library binary, linking together cldc, ncld, libtimer, and chunkdc modules. In other words, libhail at present is a simplistic combination of cld/lib and chunkd/lib. [zait...@lembas hail-tip]$ ls lib include include: chunkc.h chunksrv.hcld-private.h Makefile ncld.h chunk_msg.h cldc.helist.hMakefile.am objcache.h chunk-private.h cld_common.h hail_log.h Makefile.in lib: chunkdc.c cldc-udp.c libhail.pc.in Makefile chunksrv.c cld_msg_rpc.x libhail-uninstalled.pc Makefile.am cldc.c common.c libhail-uninstalled.pc.in Makefile.in cldc-dns.c libhail.pc libtimer.c pkt.c [zait...@lembas hail-tip]$ grep httpstor lib/*.c [zait...@lembas hail-tip]$ What has happened to the plan to include httpstor into libhail? -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New 'hail' repository created, with major packaging rework
On Mon, 05 Jul 2010 15:22:40 -0400 Jeff Garzik j...@garzik.org wrote: What has happened to the plan to include httpstor into libhail? Still planned, and can easily be done. Important first step was getting the foundation laid -- creating hail.git, and synchronizing hail.git and tabled.git, and associated RPM packaging. Thanks, I understand. I can release a hail 0.7.1 and tabled 0.5.1 with this change, if you feel versioning and pushing out this libhttpstor change is highly important. Well, I dunno, Darcy is the user of it but I don't know if he uses released RPMs or pulls from git. If he's fine with git, we may continue to procrastinate about releases. I rebuilt all the stuff I have here and it seems to work. I even blew out chunkds one at a time to account for the format incompatibility that you mentioned and let tabled to rebuild them. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] new structure: hail pkg instead of cld, chunkd
On Thu, 1 Jul 2010 18:58:18 -0400 Jeff Garzik jgar...@pobox.com wrote: To clarify my proposal, and taking into account your comments and some refinements, I think the following should occur at the git repository + source tarball level:   cld/* - hail   chunkd/* - hail Thus you would download and build hail-0.1.tar.gz. hail-0.1.tar.gz would be packaged as hail-0.1.src.rpm.  Building hail-0.1.src.rpm produces the following rpms:   libhail (httpstor, cldc, chunkdc, tcp_write, timers, other?)   libhail-devel (*.h, api docs)   cld   chunkd Sounds fine to me. We already have an unofficial unification with timers, so might as well take advantage of it. Also it addresses Darcy's request to make httpstor available. Although purely aesthetically it seems strange to have the library but not its server, it reflects the nature of tabled as a siamese twin of Hail core. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] new structure: hail pkg instead of cld, chunkd
On Tue, 29 Jun 2010 16:56:52 -0400 Jeff Garzik j...@garzik.org wrote: I've been thinking about a new structure for the projects, namely having a single hail or hail-core package, that includes cld and chunkd services, and associated client libraries inside a new libhail. It seems a little pointless to me to carry cld on all chunk nodes, of which we may have thousands. But of course this is exactly the situation today, because cld clients need the libraries, so at least you aren't making it worse. If you had hail-libs, it would make more sense, IMHO. In real terms, it would look like this: cld - hail libcldc - libhail, libhail-devel chunkd - hail libchunkdc - libhail, libhail-devel tabled - tabled (no change) libhttpstor - libhail, libhail-devel itd - itd (no change) nfs4d - nfs4d (no change) So I would rather see this: cld - cld (or hail, whatever) libcldc - hail-libs ncld.h - hail-devel libchunkdc - hail-libs chunkd - chunkd or hail chunkc.h- chunkd-devel or hail-devel etc. I think this new organization will be more useful to both developers and future users. For developers, changing the core services, and packaging commonly reused routines is easier. For users, the core services and application separation is more clear, IMO easier to understand at a glance. Right, we had a few common things, that are replicated needlessly. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Metadata replication in tabled
On Mon, 28 Jun 2010 08:37:51 -0400 Jeff Darcy jda...@redhat.com wrote: First, it seems like trying to do stuff under BDB replication, letting them control the flow, is proving to be rather painful - over a thousand lines in metarep.c plus other bits elsewhere, all constrained by their expectations wrt blocking, dropping messages, etc. Might it not be simpler to handle the replication *above* BDB instead, shipping the operations ourselves to single-node BDB instances? Simpler still might be to let a general framework like Gizzard handle the N-way replication and failover, or switch to a data store that's designed from the ground up around that need. [] I thought of it a little and decided to try the base API approach first, for a couple of reasons. First, I am ignorant of things like Gizzard, so when I started imagining how the update forwarding and leases would actually work, it started looking way longer than a 1000 lines of C. Second, I am afraid that people will point and ask why didn't you use rep_start(). We already reap NIH critique with Zookeeper. Now if I tried, found bugs in db4/BDB, and documented that, it would be different and my conscience would be clear. Getting all of the replication exposed in tabled is really tempting. For one thing, if we do it, we can replace db4 with TC or anything else. But it's just... too much. I don't have the balls to tackle it now. Honestly I expected to finish it all in 1 week, but actually took 3+. The roll-my-own replication would take me forever (how about 6 months?). Do you want tabled working for you or always in progress? (a) The minor problem is that if the second (inner) check for nrp-length 3 fails, then we return directly - leaking *nrp. Perhaps we should jump to the ncld_read_free at the end instead. Awww, that was silly. Thanks. (b) I'd also question whether checking nrp-length this way is necessary at all, since cldu_parse_master should fail in those cases anyway. Why not just rearrange the loop to catch such errors that way? The idea was to special-case the empty so I can see a printout. A syntax error is different - maybe a version mismatch. I even wanted the would-be masters try and truncate the MASTER file before trying to lock it. (c) Lastly, regarding the comment about the gap between lock and write, I think single retry of only the read doesn't buy us much. [...] At the other end of the scale, it's also not hard to imagine a node managing to take the lock and then itself aborting before the write, again causing other nodes to fail. What should happen in this second case, I'd argue, is that CLD should eventually detect the failure and break the lock, which would allow another waiting node to take it. Well, yeah... I guess I was too lazy and reluctant to create yet another state machine for this. Maybe I should just bite the bullet and make tabled fully multi-threaded. It was likely to come next anyway since you complained about the abysmal performance (I do not know yet what the issues with performance are, but threads are likely to participate). But if so, a thread may just easily loop, as ncld API intends. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [tabled patch 2/4] fix the selection of chunk
On Tue, 25 May 2010 18:44:11 -0400
Jeff Garzik j...@garzik.org wrote:
On 05/21/2010 10:19 PM, Pete Zaitcev wrote:
+static int object_node_select(int *nx, struct db_obj_ent *obj)
+{
+ struct storage_node *stnode;
+ if (stnode-up)
+ return stnode;
Did you compile or test this???
I suppose I deserved the tripple question mark for this. I tested it,
of course. The funny thing, everything is mapped below 4GB, way below
actually. As for warnings, I managed to miss them, because I started
running with -Wpointer-arith -Wsign-compare recently and that floods.
There's a lesson in it somewhere.
-- Pete
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[tabled patch 3/4] remove a per-key printout
We still have a few other per-key printouts in replication code that
a) are clearly marked as temporary, and b) only appear when keys are
replicated. This one was popping on every rescan and thus filling
the logs with useless garbage.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/replica.c |7 ---
1 file changed, 7 deletions(-)
commit ef3d146c043d7501aec4722980acc9b884553c7d
Author: Pete Zaitcev zait...@yahoo.com
Date: Fri May 21 19:56:38 2010 -0600
Remove per-key printout from replica.c.
diff --git a/server/replica.c b/server/replica.c
index e774824..ac14cb2 100644
--- a/server/replica.c
+++ b/server/replica.c
@@ -601,13 +601,6 @@ static void rep_scan_verify(struct rep_arg *arg,
oid = GUINT64_FROM_LE(obj-d.a.oid);
- applog(LOG_INFO, bucket %s key %s oid %llX n(%u,%u,%u): all %d ok %d,
- bucket_name, object_name, (long long) oid,
- GUINT32_FROM_LE(obj-d.a.nidv[0]),
- GUINT32_FROM_LE(obj-d.a.nidv[1]),
- GUINT32_FROM_LE(obj-d.a.nidv[2]),
- allcnt, redcnt);
-
if (redcnt MAXWAY) { /* maybe have MINWAY too? */
rep_job_start(arg, cp-klen, cp-key, oid,
GUINT64_FROM_LE(obj-size), redcnt, redvec);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[chunkd patch 4/6] Print client port
The originating port is interesting when the same client contacts the
chunkserver several times and we, for example, look at that with tcpdump.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/chunkd.h |1 +
server/server.c | 11 ---
2 files changed, 9 insertions(+), 3 deletions(-)
commit 7e2cc1ad8b1de3e33402147841d794e13b4f65f9
Author: Master zait...@lembas.zaitcev.lan
Date: Thu May 20 21:32:43 2010 -0600
Print port too (to match against tcpdump, for example).
diff --git a/server/chunkd.h b/server/chunkd.h
index e5721cd..47e6f95 100644
--- a/server/chunkd.h
+++ b/server/chunkd.h
@@ -84,6 +84,7 @@ struct client {
struct sockaddr_in6 addr; /* inet address */
charaddr_host[64]; /* ASCII version of inet addr */
+ charaddr_port[16]; /* ASCII version of port */
int fd; /* socket */
charuser[CHD_USER_SZ + 1];
diff --git a/server/server.c b/server/server.c
index de9e6a8..2d2f6c3 100644
--- a/server/server.c
+++ b/server/server.c
@@ -343,7 +343,8 @@ static void cli_write_free_all(struct client *cli)
static void cli_free(struct client *cli)
{
- applog(LOG_INFO, client %s disconnected, cli-addr_host);
+ applog(LOG_INFO, client host %s port %s disconnected,
+ cli-addr_host, cli-addr_port);
cli_write_free_all(cli);
@@ -1341,6 +1342,7 @@ static bool tcp_srv_event(int fd, short events, void
*userdata)
socklen_t addrlen = sizeof(struct sockaddr_in6);
struct client *cli;
char host[64];
+ char port[16];
int on = 1;
struct server_poll *sp;
@@ -1381,13 +1383,16 @@ static bool tcp_srv_event(int fd, short events, void
*userdata)
/* pretty-print incoming cxn info */
memset(host, 0, sizeof(host));
+ memset(port, 0, sizeof(port));
getnameinfo((struct sockaddr *) cli-addr, addrlen,
- host, sizeof(host), NULL, 0, NI_NUMERICHOST);
+ host, sizeof(host), port, sizeof(port), NI_NUMERICHOST);
host[sizeof(host) - 1] = 0;
- applog(LOG_INFO, client %s connected%s, host,
+ host[sizeof(port) - 1] = 0;
+ applog(LOG_INFO, client host %s port %s connected%s, host, port,
cli-ssl ? via SSL : );
strcpy(cli-addr_host, host);
+ strcpy(cli-addr_port, port);
if (!srv_poll_ready(fd))
applog(LOG_ERR, unable to ready srv fd for polling);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[chunkd patch 5/6] Drop fd from struct server_poll
This field seems completely unnecessary.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/chunkd.h |1 -
server/cldu.c |7 ---
server/server.c |4 +---
3 files changed, 5 insertions(+), 7 deletions(-)
commit 0f030f2d37a7d868752429632b98041821e1f57d
Author: Master zait...@lembas.zaitcev.lan
Date: Thu May 20 21:36:02 2010 -0600
Drop fd from server_poll, it's unnecessary.
diff --git a/server/chunkd.h b/server/chunkd.h
index 47e6f95..228ece3 100644
--- a/server/chunkd.h
+++ b/server/chunkd.h
@@ -170,7 +170,6 @@ struct server_stats {
};
struct server_poll {
- int fd; /* fd to poll for events */
short events; /* POLL* from poll.h */
boolbusy; /* if true, do not poll us */
diff --git a/server/cldu.c b/server/cldu.c
index a57c2d5..3d59063 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -474,6 +474,7 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
{
static struct cld_session *cs = ses;
struct server_poll *sp;
+ int rfd;
int retry_cnt;
int newactive;
@@ -525,6 +526,7 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
applog(LOG_ERR, Cannot open pipe: %s, strerror(errno));
goto err_pipe;
}
+ rfd = cs-event_pipe[0];
sp = calloc(1, sizeof(*sp));
if (!sp) {
@@ -532,12 +534,11 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
goto err_sp;
}
- sp-fd = cs-event_pipe[0];
sp-events = POLLIN;
sp-cb = cldu_pipe_event;
sp-userdata = cs;
- g_hash_table_insert(chunkd_srv.fd_info, GINT_TO_POINTER(sp-fd), sp);
+ g_hash_table_insert(chunkd_srv.fd_info, GINT_TO_POINTER(rfd), sp);
/*
* FIXME: We should find next suitable host according to
@@ -559,7 +560,7 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
return 0;
err_net:
- g_hash_table_remove(chunkd_srv.fd_info, GINT_TO_POINTER(sp-fd));
+ g_hash_table_remove(chunkd_srv.fd_info, GINT_TO_POINTER(rfd));
err_sp:
close(cs-event_pipe[0]);
close(cs-event_pipe[1]);
diff --git a/server/server.c b/server/server.c
index 2d2f6c3..801f26c 100644
--- a/server/server.c
+++ b/server/server.c
@@ -1374,7 +1374,6 @@ static bool tcp_srv_event(int fd, short events, void
*userdata)
if (!sp)
goto err_out_fd;
- sp-fd = cli-fd;
sp-events = POLLIN;
sp-cb = tcp_cli_event;
sp-userdata = cli;
@@ -1476,7 +1475,6 @@ static int net_open_socket(const struct listen_cfg *cfg,
goto err_out_fd;
}
- sp-fd = fd;
sp-events = POLLIN;
sp-cb = tcp_srv_event;
sp-userdata = sock;
@@ -1710,7 +1708,7 @@ static int main_loop(void)
sp-busy = true;
/* call callback, shutting down server if requested */
- runrunrun = sp-cb(sp-fd, pfd-revents, sp-userdata);
+ runrunrun = sp-cb(pfd-fd, pfd-revents, sp-userdata);
if (!runrunrun) {
server_running = false;
break;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[cld patch 1/1] fix crash in cldc_close due to array reallocations
A long-running tabled is virtually guaranteed to crash when it's shut
down, with the traceback like this:
Program terminated with signal 11, Segmentation fault.
#0 cldc_close (fh=0x10a71c8, copts=0x7fff2fdef780) at cldc.c:1049
1049if (sess-expired) {
(gdb) where
#0 cldc_close (fh=0x10a71c8, copts=0x7fff2fdef780) at cldc.c:1049
#1 0x7f8a878ed7c7 in ncld_close (fh=0x10a7790) at cldc.c:2215
#2 0x7f8a878ed918 in ncld_sess_close (nsess=0x10a6f10) at cldc.c:2247
#3 0x00406252 in cld_end () at cldu.c:701
#4 0x0040d6c4 in main (argc=value optimized out,
argv=value optimized out) at server.c:1980
(gdb) print *fh
$3 = {fh = 140232956496032, sess = 0x0, valid = 176}
Obviously the boolean value should not be 176.
The reason this happens is array reallocation. The API for libcldc
(used internally by ncld) returns a pointer to struct cldc_fh that
the user is expected to save. However, this structure is placed
inside a GArray. As array expands, it can be relocated in memory
and the file handle pointer is left hanging. Tabled is vulnerable
to this because 1) it keeps an open handle for the CLD file that
it locked and 2) it periodically opens and closes CLD files for
chunkservers, causing array to grow.
We observed that the indexes into the array are not the same
as file handles. Therefore, we never access elements by index
and thus it is not necessary to use an array structure.
This patch replaces array with a list, so that pointers to handle
structures can be stable.
Also, the patch frees the handles when they are closed, thus
ending the endless growth of the file handle table. This is
done in a compatible way, so handle is available to callers
of cldc_close (ncld does not need it, but it may be useful).
Although we change globally visible structures, we only replace
one ostensibly opaque pointer type (GArray) with another one
(GList), so applications need not be rebuilt.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
include/cldc.h |2 -
lib/cldc.c | 56 +++
2 files changed, 38 insertions(+), 20 deletions(-)
commit 93c91d8ce452f03a774055f156ada967d1b6b0a5
Author: Pete Zaitcev zait...@yahoo.com
Date: Sat May 15 12:21:43 2010 -0600
Switch from array to list. Fixes reallocation and unbounded growth.
diff --git a/include/cldc.h b/include/cldc.h
index 12acd32..c64eef9 100644
--- a/include/cldc.h
+++ b/include/cldc.h
@@ -110,7 +110,7 @@ struct cldc_session {
uint8_t addr[64]; /* server address */
size_t addr_len;
- GArray *fh;/* file handle table */
+ GList *cfh; /* cldc_fh table */
GList *out_msg;
time_t msg_scan_time;
diff --git a/lib/cldc.c b/lib/cldc.c
index 70e765f..72472f2 100644
--- a/lib/cldc.c
+++ b/lib/cldc.c
@@ -280,7 +280,7 @@ static int rxmsg_event(struct cldc_session *sess,
XDR xdrs;
struct cld_msg_event ev;
struct cldc_fh *fh = NULL;
- int i;
+ GList *tmp;
xdrmem_create(xdrs, sess-msg_buf, sess-msg_buf_len, XDR_DECODE);
if (!xdr_cld_msg_event(xdrs, ev)) {
@@ -291,8 +291,8 @@ static int rxmsg_event(struct cldc_session *sess,
}
xdr_destroy(xdrs);
- for (i = 0; i sess-fh-len; i++) {
- fh = g_array_index(sess-fh, struct cldc_fh, i);
+ for (tmp = sess-cfh; tmp; tmp = tmp-next) {
+ fh = tmp-data;
if (fh-fh == ev.fh)
break;
else
@@ -765,8 +765,11 @@ static void sess_free(struct cldc_session *sess)
if (!sess)
return;
- if (sess-fh)
- g_array_free(sess-fh, TRUE);
+ if (sess-cfh) {
+ for (tmp = sess-cfh; tmp; tmp = tmp-next)
+ free(tmp-data);
+ g_list_free(sess-cfh);
+ }
tmp = sess-out_msg;
while (tmp) {
@@ -775,7 +778,7 @@ static void sess_free(struct cldc_session *sess)
}
g_list_free(sess-out_msg);
- memset(sess, 0, sizeof(*sess));
+ memset(sess, 0x55, sizeof(*sess));
free(sess);
}
@@ -846,7 +849,6 @@ static int cldc_new_sess_log(const struct cldc_ops *ops,
sess-private = private;
sess-ops = ops;
sess-log = *log; /* save off caller's stack */
- sess-fh = g_array_sized_new(FALSE, TRUE, sizeof(struct cldc_fh), 16);
strcpy(sess-user, user);
strcpy(sess-secret_key, secret_key);
@@ -994,9 +996,8 @@ int cldc_open(struct cldc_session *sess,
{
struct cldc_msg *msg;
struct cld_msg_open open;
- struct cldc_fh fh, *fhtmp;
+ struct cldc_fh *fh;
size_t plen;
- int fh_idx;
*fh_out = NULL;
@@ -1020,22 +1021,38 @@ int cldc_open(struct cldc_session *sess,
if (!msg)
return -ENOMEM
Re: iSCSI front-end for Hail
On Sat, 01 May 2010 18:28:42 -0400 Jeff Garzik j...@garzik.org wrote: As I write this email, I am borrowing a lot of networking code from tabled, to convert from GNet over to the more-flexible TCP server codebase found in tabled -- notably the asynchronous background TCP writing code in tabled. Hopefully will finish and commit this by the end of the weekend. This seems crying for a common repository or something like libhail, not sure what. Remember the timer case. Eventually we'll make changes to tabled that itd will need to copy. But I don't know what course is best. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[chunkd patch 2/2] cleanup: use srv_poll_lookup
No, we did not mean anything in particular by using a naked hash
lookup here, just forgot to convert.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/server.c |3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/server/server.c b/server/server.c
index f367676..014575e 100644
--- a/server/server.c
+++ b/server/server.c
@@ -1681,8 +1681,7 @@ static int main_loop(void)
fired++;
- sp = g_hash_table_lookup(chunkd_srv.fd_info,
- GINT_TO_POINTER(pfd-fd));
+ sp = srv_poll_lookup(pfd-fd);
if (G_UNLIKELY(!sp)) {
/* BUG! */
continue;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[chunkd patch 1/2] eradicate last vestiges of libevent
We stopped using libevent in Chunk a while ago, but for some reason
not all references were removed. I tested this patch by building
on a fresh Fedora 13 system without libevent.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
configure.ac |3 ---
pkg/chunkd.spec|2 +-
server/Makefile.am |2 +-
3 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/configure.ac b/configure.ac
index bd142e8..5afea93 100644
--- a/configure.ac
+++ b/configure.ac
@@ -74,8 +74,6 @@ dnl Checks for required libraries
dnl -
AC_CHECK_LIB(crypto, MD5_Init, CRYPTO_LIBS=-lcrypto)
AC_CHECK_LIB(ssl, SSL_new, SSL_LIBS=-lssl)
-AC_CHECK_LIB(event, event_base_new, EVENT_LIBS=-levent,
- [AC_MSG_ERROR([Missing required libevent])])
AC_SEARCH_LIBS(argp_parse, argp)
AC_SEARCH_LIBS(bind, socket)
PKG_CHECK_MODULES(CLDC, libcldc)
@@ -119,7 +117,6 @@ AM_PATH_XML2(2.6.0, , exit 1)
AC_SUBST(CRYPTO_LIBS)
AC_SUBST(SSL_LIBS)
-AC_SUBST(EVENT_LIBS)
AC_CONFIG_FILES([doc/Makefile
lib/Makefile
diff --git a/pkg/chunkd.spec b/pkg/chunkd.spec
index 7de0f1d..032f184 100644
--- a/pkg/chunkd.spec
+++ b/pkg/chunkd.spec
@@ -15,7 +15,7 @@ Source3: chunkd.sysconf
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
# N.B. We need cld to build, because our make check spawns a private copy.
-BuildRequires: libevent-devel glib2-devel openssl-devel zlib-devel
+BuildRequires: glib2-devel openssl-devel zlib-devel
BuildRequires: libxml2-devel procps tokyocabinet-devel
BuildRequires: cld = 0.2.1
BuildRequires: cld-devel = 0.2.1
diff --git a/server/Makefile.am b/server/Makefile.am
index f2a66a1..9ab26fc 100644
--- a/server/Makefile.am
+++ b/server/Makefile.am
@@ -10,4 +10,4 @@ chunkd_SOURCES= chunkd.h \
objcache.c
chunkd_LDADD = \
@CLDC_LIBS@ @GLIB_LIBS@ @CRYPTO_LIBS@ \
- @SSL_LIBS@ @EVENT_LIBS@ @TOKYOCABINET_LIBS@
+ @SSL_LIBS@ @TOKYOCABINET_LIBS@
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 01/12] CLD: fix crash in retransmissions
On Sun, 18 Apr 2010 23:46:07 -0400 Jeff Garzik j...@garzik.org wrote: [tabled patch 1/1] update Makefile.am or [cld patch 1/1] libcldc: add nncld API, the new new CLD API That looks simple enough. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Trivial Q about chunkd's main_loop
Is there a reason why the main_loop in chunkd uses a naked
g_hash_table_lookup instead of srv_poll_lookup? Performance?
@@ -1681,8 +1681,7 @@ static int main_loop(void)
fired++;
- sp = g_hash_table_lookup(chunkd_srv.fd_info,
- GINT_TO_POINTER(pfd-fd));
+ sp = srv_poll_lookup(pfd-fd);
if (G_UNLIKELY(!sp)) {
/* BUG! */
-- Pete
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 01/12] CLD: fix crash in retransmissions
For a longest time I was plagued by (very infrequent) crashes like this:
Program received signal SIGSEGV, Segmentation fault.
sess_retry_output (timer=0x92070c0) at session.c:532
532 if (!next_retry || (op-next_retry next_retry))
(gdb) info threads
* 1 Thread 0xb72f96c0 (LWP 22417) sess_retry_output (timer=0x92070c0) at
session.c:532
(gdb) where
#0 sess_retry_output (timer=0x92070c0) at session.c:532
#1 session_retry (timer=0x92070c0) at session.c:565
#2 0x08049aee in cld_timers_run (tlist=0x8056630) at ../lib/libtimer.c:95
#3 0x0804e9cc in main_loop (argc=5, argv=0xbff70bd4) at server.c:983
#4 main (argc=5, argv=0xbff70bd4) at server.c:1138
The crash happens because op is NULL. As it turned out, this happens
if a packet retransmit and a session expiration occur simultaneously
(in the same pass of timers_run). The scenario is:
- timers_run collects expired timers at exec list
- timers_run expires session
- two timer_del are called, but one of them is on exec list already,
so it's ineffective
- session is freed, this zeroes -data in lists (later op)
- timers_run continues along the exec list, invokes the retransmission
callback, and that crashes with NULL op.
The proposed solution is to rework the timers_run, again. But this
time, we'll make it simpler by observing that timers are ordered by
expiration time. Therefore, we can pull next timer off the list,
expire it, and loop until expiration time is greater than the current
time. No execution list is kept. The integrity of the main list
is assured by never walking it and always referring to the head
anew at each iteration.
This patch appears to fix the problem and stands up to use that
crashed the old code.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
include/cld_common.h | 10 ++
lib/libtimer.c | 41 -
2 files changed, 26 insertions(+), 25 deletions(-)
I am absurdly proud of this. But the long history of timer problem
in CLD makes me cautious. I already thought that we fixed everything
in the past.
commit c1b4a0c4dda14d9fbc4d5896e9e0dc139c31798b
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 19:02:35 2010 -0600
Fix persistent CLD crashes in retransmits (op == NULL).
diff --git a/include/cld_common.h b/include/cld_common.h
index d269e6c..379bd9e 100644
--- a/include/cld_common.h
+++ b/include/cld_common.h
@@ -24,6 +24,7 @@
#include stdbool.h
#include string.h
#include time.h
+#include glib.h
#include openssl/sha.h
#include cld_msg_rpc.h
@@ -31,10 +32,6 @@
struct hail_log;
-struct cld_timer_list {
- void *list;
-};
-
struct cld_timer {
boolfired;
boolon_list;
@@ -44,6 +41,11 @@ struct cld_timer {
charname[32];
};
+struct cld_timer_list {
+ GList *list;/* of struct cld_timer */
+ time_t runmark;
+};
+
extern void cld_timer_add(struct cld_timer_list *tlist, struct cld_timer
*timer,
time_t expires);
extern void cld_timer_del(struct cld_timer_list *tlist, struct cld_timer
*timer);
diff --git a/lib/libtimer.c b/lib/libtimer.c
index c6f5241..75514f0 100644
--- a/lib/libtimer.c
+++ b/lib/libtimer.c
@@ -44,6 +44,17 @@ void cld_timer_add(struct cld_timer_list *tlist, struct
cld_timer *timer,
if (timer-on_list)
timer_list = g_list_remove(timer_list, timer);
+ /*
+* This additional resiliency is required by the invocations from
+* session_retry(). For some reason the computations in it result
+* in attempts to add timers in the past sometimes, and then we loop
+* when trying to run those. FIXME: maybe fix that one day.
+*
+* Even if we fix the callers, we probably should keep this.
+*/
+ if (expires tlist-runmark + 1)
+ expires = tlist-runmark + 1;
+
timer-on_list = true;
timer-fired = false;
timer-expires = expires;
@@ -66,38 +77,26 @@ time_t cld_timers_run(struct cld_timer_list *tlist)
struct cld_timer *timer;
time_t now = time(NULL);
time_t next_timeout = 0;
- GList *tmp, *cur;
- GList *timer_list = tlist-list;
- GList *exec_list = NULL;
-
- tmp = timer_list;
- while (tmp) {
- timer = tmp-data;
- cur = tmp;
- tmp = tmp-next;
+ GList *cur;
+ tlist-runmark = now;
+ for (;;) {
+ cur = tlist-list;
+ if (!cur)
+ break;
+ timer = cur-data;
if (timer-expires now)
break;
- timer_list = g_list_remove_link(timer_list, cur);
- exec_list = g_list_concat(exec_list, cur);
-
+ tlist-list = g_list_delete_link(tlist-list, cur);
timer-on_list = false
[Patch 03/12] CLD: fix commentary
Add and fix some comments regarding the reasons behind the pipe etc.
No code changes.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
lib/cldc.c | 27 ---
1 file changed, 24 insertions(+), 3 deletions(-)
commit e675f2f316bbb24ca84c1bc23e4d1c6d53b029de
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 19:14:30 2010 -0600
Fix commentary.
diff --git a/lib/cldc.c b/lib/cldc.c
index c1e8993..6ab2fe4 100644
--- a/lib/cldc.c
+++ b/lib/cldc.c
@@ -1372,7 +1372,7 @@ enum {
/*
* All the error printouts are likely to be lost for daemons, but it's
- * not a big deal. We abort instead of exist to indicate that something
+ * not a big deal. We abort instead in order to indicate that something
* went wrong, so system features should report it (usualy as a core).
* When debugging, strace or -F mode will capture the output.
*/
@@ -1503,13 +1503,29 @@ static void ncld_p_event(void *priv, struct
cldc_session *csp,
if (!nsess-is_up)
return;
nsess-is_up = false;
- /* XXX wake up all I/O waiters here */
+
+ /*
+* The cldc layer must deliver the callbacks for all pending
+* CLD operations of the failed session. If we force-wake their
+* waiters, all sorts of funny things happen with the lifetimes
+* of related structures.
+*/
+ //
+ // g_cond_broadcast(nsess-cond);
+
/*
* This is a trick. As a direct callback from clcd layer,
* we are running under nsess-mutex, so we cannot call back
* into a user of ncld. If we do, it may invoke another
* ncld operation and deadlock. So, bump session callbacks
* into the part of the helper thread that runs unlocked.
+*
+* Notice that we are already running on the context of the
+* thread that will deliver the event, so pipe really is not
+* needed: could as well set a flag and test it right after
+* the call to cldc_udp_receive_pkt(). But pipe also provides
+* a queue of events, just in case. It's not like these events
+* are super-performance critical.
*/
cmd = NCLD_CMD_SESEV;
write(nsess-to_thread[1], cmd, 1);
@@ -1820,7 +1836,12 @@ int ncld_del(struct ncld_sess *nsess, const char *fname)
g_mutex_unlock(nsess-mutex);
return -rc;
}
- /* XXX A delete operation is not accounted for end-session */
+ /*
+* A delete operation is not accounted for end-session (e.g. nios).
+* This means: do not call ncld_del and ncld_close together.
+* The ncld_close can be invoked by ncld_sess_close, so don't
+* do that either.
+*/
g_mutex_unlock(nsess-mutex);
rc = ncld_wait_del(dpb);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 06/12] Chunk: do not call timer_add from event context
No matter if timer or cld_timer is used, this was not valid.
Obviously, locking is missing, so only one thread can access a
certain tlist. But the actual hang was more interesting than a
race and crash. Suppose that we add the first timer. In that
case, main thread invokes poll() with no timeout. If we add
the timer from the event callback, nobody wakes up the poll,
so the newly-added timer is not checked for expiration and never
fires, causing the hang.
Note that if we look at the whole stack, the session failure event
is bounced through two pipes: one in ncld (loopback), and another
here. The pipe in ncld is not really needed for this, but we use
it for convenience. Seems weird and inefficient, but it's short
code and session events are not performance-critical.
P.S. There's also an important bugfix in this patch: the is_dead
should not be cleared if a retry timeout is scheduled.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/cldu.c | 77 +++-
1 file changed, 69 insertions(+), 8 deletions(-)
commit d30028b4c681e99f7934ca264a175548c827ff04
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 19:46:35 2010 -0600
Calling timer_add from event thread is invalid. Use a pipe instead.
diff --git a/server/cldu.c b/server/cldu.c
index 957bd81..fafcc3b 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -47,6 +47,7 @@ struct cld_session {
struct cld_host cldv[N_CLD];
struct cld_timer timer;
+ int event_pipe[2];
char *ffname;
struct ncld_fh *ffh;/* keep open for lock */
@@ -95,9 +96,8 @@ static void cldu_saveargs(struct cld_session *sp, char
*infopath,
sp-ploc = loc;
}
-static void cldu_timer_event(struct cld_timer *timer)
+static void cldu_sess_proc(struct cld_session *cs)
{
- struct cld_session *cs = timer-userdata;
int newactive;
if (cs-is_dead) {
@@ -106,29 +106,58 @@ static void cldu_timer_event(struct cld_timer *timer)
if (debugging)
applog(LOG_DEBUG, Reopening Chunk in %s, cs-ffname);
- ncld_sess_close(cs-nsess);
- cs-nsess = NULL;
+ if (cs-nsess) {
+ ncld_sess_close(cs-nsess);
+ cs-nsess = NULL;
+ }
cs-ffh = NULL; /* closed automatically */
- cs-is_dead = false;
newactive = cldu_nextactive(cs);
if (cldu_set_cldc(cs, newactive)) {
/* Oops, should not happen. Just loop, then... */
timer_add(cs-timer, time(NULL) + 30);
return;
}
+ cs-is_dead = false;
+ } else {
+ /*
+* We want to see if this ever happens.
+* Probably harmless, but... let's print it.
+*/
+ applog(LOG_WARNING, Event on non-dead session);
}
}
+static void cldu_timer_event(struct cld_timer *timer)
+{
+ struct cld_session *cs = timer-userdata;
+
+ cldu_sess_proc(cs);
+}
+
+static bool cldu_pipe_event(int fd, short events, void *userdata)
+{
+ struct cld_session *cs = userdata;
+ unsigned char cmd;
+ ssize_t rc;
+
+ rc = read(fd, cmd, 1);
+ if (rc 0)
+ cldu_sess_proc(cs);
+ else
+ applog(LOG_WARNING, Stray CLD event pipe poll);
+ return true;
+}
+
static void cldu_sess_event(void *priv, uint32_t what)
{
struct cld_session *cs = priv;
+ unsigned char cmd;
if (what == CE_SESS_FAILED) {
/*
* In ncld, we are not allowed to free the session structures
* from an event (it's wages of all-conquering 100% reliable
-* ncld_close_sess), so we bounce that off to a thread. Which
-* we do not have, so we steal a timer for an erzatz thread.
+* ncld_close_sess), so we bounce that off to the main thread.
*/
if (cs-nsess) {
applog(LOG_ERR, Session failed, sid SIDFMT,
@@ -137,7 +166,10 @@ static void cldu_sess_event(void *priv, uint32_t what)
applog(LOG_ERR, Session open failed);
}
cs-is_dead = true;
- timer_add(cs-timer, time(NULL) + 1);
+ cmd = 1;
+ if (write(cs-event_pipe[1], cmd, 1) 1) {
+ applog(LOG_ERR, Pipe write failed: %d, errno);
+ }
} else {
if (cs)
applog(LOG_INFO, cldc event 0x%x sid SIDFMT,
@@ -438,6 +470,7 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
struct geo *locp, void (*cb)(enum st_cld))
{
static struct cld_session *cs = ses;
+ struct server_poll *sp;
if (!nid)
return
[Patch 07/12] Chunk: retry initial CLD session open
This was an error in the conversion to ncld. In the cldc code, we
kick the state machine and the natural retries do the rest. Any
failures occure there. But in ncld the original kick can fail too.
Five retries give CLD server time to reboot. If it's down, then
clients refuse to start. This may be a bad idea, or may be not.
We may yet change the retries to be infinite, but for now it's
better if builds terminate somehow in case of unexpected problems.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/cldu.c | 12 ++--
1 file changed, 10 insertions(+), 2 deletions(-)
commit 44cdb98d2cceb2f4e081db2ee38ec60f1c1a8d8d
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 19:50:06 2010 -0600
Retry the initial connection to the CLD server.
diff --git a/server/cldu.c b/server/cldu.c
index fafcc3b..58edf4b 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -471,6 +471,8 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
{
static struct cld_session *cs = ses;
struct server_poll *sp;
+ int retry_cnt;
+ int newactive;
if (!nid)
return 0;
@@ -540,9 +542,15 @@ int cld_begin(const char *thishost, uint32_t nid, char
*infopath,
* -- Actually, it only works when recovering from CLD failure.
*Thereafter, any slave CLD redirects us to the master.
*/
- if (cldu_set_cldc(cs, 0)) {
+ newactive = 0;
+ retry_cnt = 0;
+ for (;;) {
+ if (!cldu_set_cldc(cs, newactive))
+ break;
/* Already logged error */
- goto err_net;
+ if (++retry_cnt == 5)
+ goto err_net;
+ newactive = cldu_nextactive(cs);
}
return 0;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 08/12] Chunk: fix wrong message
The message makes no sense. It was a carry-over from cldc where there
were many failure modes (fh is NULL, fh-valid false, etc.).
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/cldu.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
commit b723269c703c60560e18923ef49fdbe2a46be133
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 19:52:40 2010 -0600
Bogus message.
diff --git a/server/cldu.c b/server/cldu.c
index 58edf4b..fd7e9a1 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -384,7 +384,7 @@ static int cldu_set_cldc(struct cld_session *cs, int
newactive)
error, 0 /* CE_MASTER_FAILOVER | CE_SESS_FAILED */,
NULL, NULL);
if (cs-ffh == NULL) {
- applog(LOG_ERR, CLD open(%s) failed: NULL fh, cs-ffname);
+ applog(LOG_ERR, CLD open(%s) failed: %d, cs-ffname, error);
goto err_fopen;
}
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 09/12] tabled: drop double prefixing
On Fedora 14, the following is seen in syslog:
Apr 17 19:58:52 niphredil tabled: tabled: connecting to site
hitlain.zaitcev.lan:8083: No route to host
Apr 17 19:58:56 niphredil tabled: tabled: DB_ENV-rep_elect:WARNING:
nvotes (1) is sub-majority with nsites (2)
Drop the extra prefix, it only wastes screen space.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
lib/tdb.c |7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
commit eb60e6e5c97fe316d23b9b21ba020bca924e879e
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 20:31:56 2010 -0600
Fix double tagging in syslog.
diff --git a/lib/tdb.c b/lib/tdb.c
index cd65371..12ff231 100644
--- a/lib/tdb.c
+++ b/lib/tdb.c
@@ -38,7 +38,12 @@ enum {
static void db4syslog(const DB_ENV *dbenv, const char *errpfx, const char *msg)
{
- syslog(LOG_WARNING, %s: %s, errpfx, msg);
+ /*
+* Since we use syslog, we discard the prefix set in tdb_init,
+* because syslog adds our own prefix too. The errpfx would be
+* useful if we weren't dumping to syslog here.
+*/
+ syslog(LOG_WARNING, %s, msg);
}
static int buckets_owner_idx(DB *secondary, const DBT *pkey, const DBT *pdata,
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 11/12] tabled: check argument of -D better
The atoi() really does not cut it, as I discovered when I supplied
-D -E to tabled. Other arguments may benefit from such checking too,
but -D is unique in that nothing gets logged in case of this mistake.
So let's just add it here for now; others will at least report something.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/server.c |4
1 file changed, 4 insertions(+)
commit b340bd6bbf9d7a82b69ad620f43799b616348a45
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 20:40:12 2010 -0600
Deconfuse -D -E.
diff --git a/server/server.c b/server/server.c
index e1b0dbe..a28965c 100644
--- a/server/server.c
+++ b/server/server.c
@@ -184,6 +184,10 @@ static error_t parse_opt (int key, char *arg, struct
argp_state *state)
tabled_srv.config = arg;
break;
case 'D':
+ if (arg[0] == '-') {
+ fprintf(stderr, Option -D requires an argument\n);
+ argp_usage(state);
+ }
v = atoi(arg);
if (v 0 || v 2) {
fprintf(stderr, invalid debug level: '%s'\n, arg);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 12/12] tabled: print hostname always
This code clearly was obsolete and wishful thinking. Let's just be
simple. Most importantly print something that tells the sysadmin
what node is the problem.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/storage.c | 19 +++
server/tabled.h |2 +-
2 files changed, 4 insertions(+), 17 deletions(-)
commit 31dc52b7d177bd18a765a9fc407c2afdd103613e
Author: Master zait...@lembas.zaitcev.lan
Date: Sat Apr 17 20:42:24 2010 -0600
Print host name in storage retries.
diff --git a/server/storage.c b/server/storage.c
index 1793fa0..a63012e 100644
--- a/server/storage.c
+++ b/server/storage.c
@@ -489,26 +489,13 @@ void stor_add_node(uint32_t nid, const char *hostname,
const char *portstr,
int stor_node_check(struct storage_node *stn)
{
struct st_client *stc;
- char host[41];
- char port[6];
int rc;
rc = stor_new_stc(stn, stc);
if (rc 0) {
- if (rc == -EINVAL) {
- if (getnameinfo((struct sockaddr *) stn-addr,
- stn-alen, host, sizeof(host),
- port, sizeof(port),
- NI_NUMERICHOST|NI_NUMERICSERV) == 0) {
- applog(LOG_INFO, Error connecting to chunkd
- on host %s port %s,
- host, port);
- } else {
- applog(LOG_INFO, Error connecting to chunkd);
- }
- } else {
- applog(LOG_INFO, Error %d connecting to chunkd, rc);
- }
+ applog(LOG_INFO,
+ Error %d connecting to chunkd on host %s,
+ rc, stn-hostname);
return -1;
}
diff --git a/server/tabled.h b/server/tabled.h
index eced4b6..75fa147 100644
--- a/server/tabled.h
+++ b/server/tabled.h
@@ -96,7 +96,7 @@ struct storage_node {
unsignedalen;
struct sockaddr_in6 addr;
- char *hostname; /* Only used because stc_new is overly smart. */
+ char*hostname;
int ref;/* number of open_chunk or other */
};
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: tabled RPM build fails before it succeeds
On Fri, 16 Apr 2010 13:16:56 -0400 Jeff Garzik j...@garzik.org wrote: Build #1 (fails on x86_64): http://koji.fedoraproject.org/koji/taskinfo?taskID=2119825 PASS: prep-db chunkd[17774]: Waiting for CLD PortFile cld.port cld[17773]: databases up cld[17773]: Listening on port 34671 cld[17773]: initialized: nodebug chunkd[17774]: Using CLD port 34671 chunkd[17775]: Listening on auto port 57521 tabled[1]: Listening on port 36667 chunkd[17775]: New CLD session created, sid 4D81504D6021608C chunkd[17775]: initialized PASS: start-daemon PASS: pid-exists PASS: daemon-running server is not up after 100 s Build #2 (fails on i686): http://koji.fedoraproject.org/koji/taskinfo?taskID=2120174 Same. I think it's time to figure it out. Going for a 100s delay kinda sorta helped, but on second thought: why would any packets get lost at all, on a loopback? I'll try to see if I can reproduce this by overloading the test system. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 4/8] CLD: cleanup: add #include
A prototype is not included again. Signed-off-by: Pete Zaitcev zait...@redhat.com --- lib/common.c |1 + 1 file changed, 1 insertion(+) diff --git a/lib/common.c b/lib/common.c index fb0aae6..e399ec9 100644 --- a/lib/common.c +++ b/lib/common.c @@ -26,6 +26,7 @@ #include errno.h #include glib.h #include cld-private.h +#include cld_common.h #include cld_msg_rpc.h /* duplicated from tools/cldcli.c; put in common header somewhere? */ -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 6/8] CLD: cleanup: add a event stub just to test
Tests do not really need this kind of thing, in fact if everything
goes well, we should NOT receive any events while test is running.
Still, I am curious about being sure that we do not. This will also
pop if we change something in the library implementation and start
leaking some events.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
test/lock-file.c | 11 ++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/test/lock-file.c b/test/lock-file.c
index 36cc62c..3bbe49b 100644
--- a/test/lock-file.c
+++ b/test/lock-file.c
@@ -31,6 +31,15 @@
#include ncld.h
#include test.h
+static void sess_event(void *priv, unsigned int what)
+{
+ if (what == CE_SESS_FAILED) {
+ fprintf(stderr, Session failed\n);
+ exit(1);
+ }
+ fprintf(stderr, Unknown event %d\n, what);
+}
+
int main (int argc, char *argv[])
{
struct ncld_sess *nsess;
@@ -49,7 +58,7 @@ int main (int argc, char *argv[])
if (port == 0)
return -1;
- nsess = ncld_sess_open(TEST_HOST, port, error, NULL, NULL,
+ nsess = ncld_sess_open(TEST_HOST, port, error, sess_event, NULL,
TEST_USER, TEST_USER_KEY, NULL);
if (!nsess) {
fprintf(stderr, ncld_sess_open(host %s port %u) failed: %d\n,
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 5/8] CLD: cleanup: wrap a line
Signed-off-by: Pete Zaitcev zait...@redhat.com --- lib/cldc.c |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) Looks like I confused myself with patch numbering. This one is out of order, or maybe a duplicate? Sending on the premise that it's easier to drop. diff --git b/lib/cldc.c a/lib/cldc.c index 0eb7ad8..305e05d 100644 --- b/lib/cldc.c +++ a/lib/cldc.c @@ -1595,8 +1595,8 @@ struct ncld_sess *ncld_sess_open(const char *host, int port, int *error, if (!nsess) goto out_sesalloc; memset(nsess, 0, sizeof(struct ncld_sess)); - cld_timer_init(nsess-udp_timer, nsess-udp-timer, ncld_udp_timer_event, - nsess); + cld_timer_init(nsess-udp_timer, nsess-udp-timer, + ncld_udp_timer_event, nsess); nsess-mutex = g_mutex_new(); if (!nsess-mutex) goto out_mutex; -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 0/3] End-to-end verbosity v3
Here's yet another take on the precice verbosity control, this time with -D 2 replacing -v. Everywhere, not just in CLD. Uniformity! BTW, someone forgot to update all the examples when -D with an argument was introduced, so this patch does it now. These patches have an API change, so as usual CLD must be built first. But surely this ought to be the last logging API change ever, right? Most importantly, this thing is DEAD: /* nsess-udp-sess-log.verbose = 1; */ For some reason it peeved me a lot. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 1/7] tabled: make two dump displays uniform
On Tue, 06 Apr 2010 12:57:57 -0400 Jeff Garzik j...@garzik.org wrote: applied, thanks. I will endeavor to make the stats dump more like nfs4d in the future, FWIW. I was going to look into it too, but there's now some stuff I urgently need to address in Darcy's report on tabled's faults. So I only wrote nfs4 into the endless TODO. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 1/3] CLD: End-to-end verbosity
On Wed, 07 Apr 2010 00:36:32 -0400 Jeff Garzik j...@garzik.org wrote: On 04/06/2010 11:32 PM, Pete Zaitcev wrote: On Tue, 06 Apr 2010 10:40:33 -0400 Jeff Garzikj...@garzik.org wrote: The debug levels are 0: key messages affecting server operation, only 1: debugging output enabled, sans per-packet output 2: debugging output enabled, including per-packet output The previous patch did just that: Why did you reject it? That's a damned good question. I have no idea. Did I ever reply to that patch? It looks like I fscked up and missed it? Apparently no reply (by e-mail): http://marc.info/?l=hail-develm=126575343714155w=2 I recall you said that users hate bitmasks on IRC or in other thread, so I thought that -D and -v would be right (effectively a bitmask but no hexadecimal at least). -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: CLD doesn't build on db-4.3
On Thu, 01 Apr 2010 19:01:59 +0800 Samba - BoYang boy...@samba.org wrote: CLD doesn't build on db-4.3 on suse 11, since db-4.3 uses deprecated structure members DBC-c_xxx(c_close(), etc) instead of DBC-xxx. :-) That's unexpected. Jeff is our portability expert, but he's busy with the baby, so I guess I'll have to look if it's something easily fixable. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 1/7] tabled: make two dump displays uniform
From: Jeff Garzik jgar...@pobox.com
Subject: Re: Tabled issues
Date: Mon, 29 Mar 2010 15:32:33 -0400
I asserted that the standard stats dump facility must dump
all available statistics. That does not exclude other methods
of stat(us) dumping. Your patch added new stats to the HTML-pretty
version of output, but failed to add the new stats to the standard
stat dump facility.
Your wish is my command.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/replica.c | 28 +
server/server.c | 47 ++
server/status.c | 22 +--
server/storage.c | 50 +
server/tabled.h |3 ++
5 files changed, 117 insertions(+), 33 deletions(-)
commit 6bba495d750dcc7ebe63938b91d4bb22740879fe
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Apr 1 18:58:08 2010 -0600
Make signal+applog and HTTP displays uniform.
diff --git a/server/replica.c b/server/replica.c
index abdd647..e774824 100644
--- a/server/replica.c
+++ b/server/replica.c
@@ -871,6 +871,34 @@ void rep_start()
kscan_enabled = true;
}
+void rep_stats()
+{
+ bool running;
+ unsigned long kcnt;
+ time_t last;
+
+ applog(LOG_INFO, REP: Jobs: queued %d active %d done %d,
+ queue.njobs, active.njobs, done.njobs);
+
+ g_mutex_lock(kscan_mutex);
+ running = kscan_running;
+ last = kscan_last;
+ kcnt = kscan_cnt;
+ g_mutex_unlock(kscan_mutex);
+
+ if (running) {
+ applog(LOG_INFO, REP: run Active started %lu scanned %lu,
+ (long) last, kcnt);
+ } else {
+ if (last)
+ applog(LOG_INFO,
+ REP: run Done started %lu scanned %lu,
+ (long) last, kcnt);
+ else
+ applog(LOG_INFO, REP: run None);
+ }
+}
+
bool rep_status(struct client *cli, GList *content)
{
time_t now;
diff --git a/server/server.c b/server/server.c
index e0d785c..72db151 100644
--- a/server/server.c
+++ b/server/server.c
@@ -370,21 +370,50 @@ static void stats_signal(int signo)
write(tabled_srv.ev_pipe[1], cmd, 1);
}
-#define X(stat) \
- applog(LOG_INFO, STAT %s %lu, #stat, tabled_srv.stats.stat)
-
static void stats_dump(void)
{
- X(poll);
- X(event);
- X(tcp_accept);
- X(opt_write);
- applog(LOG_INFO, State: TDB %s,
+ applog(LOG_INFO, STATE: TDB %s,
state_name_tdb[tabled_srv.state_tdb]);
+ applog(LOG_INFO,
+ STATS: poll %lu event %lu tcp_accept %lu opt_write %lu,
+ tabled_srv.stats.poll,
+ tabled_srv.stats.event,
+ tabled_srv.stats.tcp_accept,
+ tabled_srv.stats.opt_write);
stor_stats();
+ rep_stats();
}
-#undef X
+bool stat_status(struct client *cli, GList *content)
+{
+ char *str;
+
+ /*
+* The loadavg is system dependent, we'll figure it out later.
+* On Linux, applications read from /proc/loadavg.
+*/
+ if (asprintf(str,
+h1Status/h1
+pHost %s port %s/p\r\n,
+tabled_srv.ourhost, tabled_srv.port) 0)
+ return false;
+ content = g_list_append(content, str);
+ if (asprintf(str,
+pState: TDB %s/p\r\n,
+state_name_tdb[tabled_srv.state_tdb]) 0)
+ return false;
+ content = g_list_append(content, str);
+ if (asprintf(str,
+pStats:
+poll %lu event %lu tcp_accept %lu opt_write %lu/p\r\n,
+tabled_srv.stats.poll,
+tabled_srv.stats.event,
+tabled_srv.stats.tcp_accept,
+tabled_srv.stats.opt_write) 0)
+ return false;
+ content = g_list_append(content, str);
+ return true;
+}
static bool cli_write_free(struct client *cli, struct client_write *tmp,
bool done)
diff --git a/server/status.c b/server/status.c
index bb67ac9..2029c09 100644
--- a/server/status.c
+++ b/server/status.c
@@ -114,25 +114,6 @@ out:
return false;
}
-static bool stat_status(struct client *cli, GList *content)
-{
- char *str;
-
- /*
-* The loadavg is system dependent, we'll figure it out later.
-* On Linux, applications read from /proc/loadavg.
-*/
- if (asprintf(str,
-h1Status/h1
-pHost %s port %s/p\r\n
-pStats: poll %lu event %lu/p\r\n,
-tabled_srv.ourhost, tabled_srv.port,
-tabled_srv.stats.poll, tabled_srv.stats.event) 0)
- return false;
- content = g_list_append(content, str);
- return true;
-}
-
static bool stat_root(struct client *cli
[Patch 2/7] tabled: fix the endless recusion when reading long objects
At certain network and disk speeds, tabled can blow its stack by
filling it with (essentially) endless recursion:
#2 0x0040c077 in cli_write_free (cli=value optimized out, tmp=
0x7bb910, done=value optimized out) at server.c:397
#3 0x0040ca55 in cli_writable (cli=0x686e90) at server.c:525
#4 0x0040da65 in cli_write_start (cli=0x686e90) at server.c:561
#5 0x00408ad5 in object_get_poke (cli=0x686e90) at object.c:1039
#6 0x0040c077 in cli_write_free (cli=value optimized out, tmp=
0x7bb8d0, done=value optimized out) at server.c:397
#7 0x0040ca55 in cli_writable (cli=0x686e90) at server.c:525
#8 0x0040da65 in cli_write_start (cli=0x686e90) at server.c:561
#9 0x00408ad5 in object_get_poke (cli=0x686e90) at object.c:1039
#10 0x0040c077 in cli_write_free (cli=value optimized out, tmp=
0x7bb890, done=value optimized out) at server.c:397
The fix is to deliver callbacks only from the top level.
Callbacks must be delivered every time a send is completed,
which amounts to every call to is_writeable(). Since there
is a large number of callers to it, we found it advantageous
to run callbacks from every source of events. In other words,
every function that is passed to event_set must invoke
cli_write_run_compl. Mind that storage.c contains calls to
event_set.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/object.c |4 +++
server/server.c | 52 +++---
server/tabled.h |6 +
3 files changed, 50 insertions(+), 12 deletions(-)
commit 88959fb8c4cc8232f92805618c5bcaeee9099390
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Apr 1 19:04:13 2010 -0600
Fix the endless recursion when reading long objects.
diff --git a/server/object.c b/server/object.c
index d61a446..6d32316 100644
--- a/server/object.c
+++ b/server/object.c
@@ -994,6 +994,9 @@ static bool object_get_poke(struct client *cli)
char *buf;
ssize_t bytes;
+ /* XXX flow control - what treshold? */
+ /* if (cli_wqueued(cli) = 10) return false; */
+
buf = malloc(CLI_DATA_BUF_SZ);
if (!buf)
return false;
@@ -1052,6 +1055,7 @@ static bool object_get_more(struct client *cli, void
*cb_data, bool done)
static void object_get_event(struct open_chunk *ochunk)
{
object_get_poke(ochunk-cli);
+ cli_write_run_compl(ochunk-cli);
}
bool object_get_body(struct client *cli, const char *user, const char *bucket,
diff --git a/server/server.c b/server/server.c
index 72db151..600c8de 100644
--- a/server/server.c
+++ b/server/server.c
@@ -380,6 +380,8 @@ static void stats_dump(void)
tabled_srv.stats.event,
tabled_srv.stats.tcp_accept,
tabled_srv.stats.opt_write);
+ applog(LOG_INFO, DEBUG: max_write_buf %lu,
+ tabled_srv.stats.max_write_buf);
stor_stats();
rep_stats();
}
@@ -405,16 +407,24 @@ bool stat_status(struct client *cli, GList *content)
content = g_list_append(content, str);
if (asprintf(str,
pStats:
-poll %lu event %lu tcp_accept %lu opt_write %lu/p\r\n,
+poll %lu event %lu tcp_accept %lu opt_write %lu/p\r\n
+pDebug: max_write_buf %lu/p\r\n,
tabled_srv.stats.poll,
tabled_srv.stats.event,
tabled_srv.stats.tcp_accept,
-tabled_srv.stats.opt_write) 0)
+tabled_srv.stats.opt_write,
+tabled_srv.stats.max_write_buf) 0)
return false;
content = g_list_append(content, str);
return true;
}
+static void cli_write_complete(struct client *cli, struct client_write *tmp)
+{
+ list_del(tmp-node);
+ list_add_tail(tmp-node, cli-write_compl_q);
+}
+
static bool cli_write_free(struct client *cli, struct client_write *tmp,
bool done)
{
@@ -433,11 +443,28 @@ static void cli_write_free_all(struct client *cli)
{
struct client_write *wr, *tmp;
+ list_for_each_entry_safe(wr, tmp, cli-write_compl_q, node) {
+ cli_write_free(cli, wr, true);
+ }
list_for_each_entry_safe(wr, tmp, cli-write_q, node) {
cli_write_free(cli, wr, false);
}
}
+bool cli_write_run_compl(struct client *cli)
+{
+ struct client_write *wr;
+ bool do_loop;
+
+ do_loop = false;
+ while (!list_empty(cli-write_compl_q)) {
+ wr = list_entry(cli-write_compl_q.next, struct client_write,
+ node);
+ do_loop |= cli_write_free(cli, wr, true);
+ }
+ return do_loop;
+}
+
static void cli_free(struct client *cli)
{
cli_write_free_all(cli);
@@ -455,6 +482,9 @@ static void cli_free(struct client *cli)
req_free(cli-req);
+ if (cli
[Patch 3/7] tabled: Stub NULL session in events
Currently ncld has a bug that causes it to report a session failure
when session was not open yet. So, if a session fails to open,
application crashes while trying to print the SID. This problem
made this crash occur easily.
Unfortunately for us, there's more to it. If the library opened
session, but is pre-empted before ncld_sess_open returns, the
same situation may occur legitimately. So this check is still
needed, unless we radically change the locking strategy.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/cldu.c |7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
commit 446e8ca93ce624797a69841f38081faa369fee4a
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Apr 1 19:09:34 2010 -0600
Stub the problem with failure to open.
diff --git a/server/cldu.c b/server/cldu.c
index aecf336..fadc3d8 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -176,8 +176,11 @@ static void cldu_sess_event(void *priv, uint32_t what)
struct cld_session *sp = priv;
if (what == CE_SESS_FAILED) {
- applog(LOG_ERR, Session failed, sid SIDFMT,
- SIDARG(sp-nsp-udp-sess-sid));
+ if (sp-nsp)
+ applog(LOG_ERR, Session failed, sid SIDFMT,
+ SIDARG(sp-nsp-udp-sess-sid));
+ else
+ applog(LOG_ERR, Session open failed);
sp-is_dead = true;
} else {
if (sp-nsp)
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 5/7] tabled: make unnecesserily global functions static
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/object.c |9 +
server/server.c |4 ++--
server/status.c |2 +-
server/tdbadm.c |2 +-
4 files changed, 9 insertions(+), 8 deletions(-)
commit f21ca8985bc7ba2b6e02fb0e58c79bcb96ab844f
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Apr 1 19:17:38 2010 -0600
Add a bunch of statics.
diff --git a/server/object.c b/server/object.c
index 6d32316..d4d5b8d 100644
--- a/server/object.c
+++ b/server/object.c
@@ -765,8 +765,9 @@ err:
return -1;
}
-bool object_put_body(struct client *cli, const char *user, const char *bucket,
- const char *key, long content_len, bool expect_cont)
+static bool object_put_body(struct client *cli, const char *user,
+ const char *bucket, const char *key,
+ long content_len, bool expect_cont)
{
long avail;
uint64_t objid;
@@ -1058,8 +1059,8 @@ static void object_get_event(struct open_chunk *ochunk)
cli_write_run_compl(ochunk-cli);
}
-bool object_get_body(struct client *cli, const char *user, const char *bucket,
- const char *key, bool want_body)
+static bool object_get_body(struct client *cli, const char *user,
+ const char *bucket, const char *key, bool want_body)
{
char *md5;
char timestr[64], modstr[64], *hdr, *tmp;
diff --git a/server/server.c b/server/server.c
index 600c8de..18b3359 100644
--- a/server/server.c
+++ b/server/server.c
@@ -796,8 +796,8 @@ bool cli_err_write(struct client *cli, char *hdr, char
*content)
return cli_write_start(cli);
}
-bool cli_resp(struct client *cli, int http_status, const char *content_type,
- GList *content)
+static bool cli_resp(struct client *cli, int http_status,
+const char *content_type, GList *content)
{
int rc;
char *hdr, timestr[50];
diff --git a/server/status.c b/server/status.c
index 2029c09..65bfd1e 100644
--- a/server/status.c
+++ b/server/status.c
@@ -62,7 +62,7 @@ static const char stat_err_fmt1[] = {
/html\r\n
};
-bool stat_err(struct client *cli, enum errcode code)
+static bool stat_err(struct client *cli, enum errcode code)
{
char *hdr = NULL, *content = NULL;
int err_status;
diff --git a/server/tdbadm.c b/server/tdbadm.c
index 2ab537b..86fa4b3 100644
--- a/server/tdbadm.c
+++ b/server/tdbadm.c
@@ -189,7 +189,7 @@ static const GMarkupParser cfg_parse_ops = {
.text = cfg_elm_text,
};
-void read_config(void)
+static void read_config(void)
{
GMarkupParseContext* parser;
char *text;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 6/7] tabled: unsigned 0
Enums and sizeofs are unsigned. V. annoying because of int i;.
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
server/status.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff -urp -X dontdiff tabled-m/server/status.c tabled-tip/server/status.c
--- tabled-m/server/status.c2010-04-01 19:13:05.0 -0600
+++ tabled-tip/server/status.c 2010-04-01 17:55:46.0 -0600
@@ -74,7 +74,7 @@ static bool stat_err(struct client *cli,
* We do not use all codes that the main server can return,
* so our array is smaller and it has gaps. Must check, in case.
*/
- if (code 0 || code = ARRAY_SIZE(err_info) || !err_info[code].code) {
+ if (code = ARRAY_SIZE(err_info) || !err_info[code].code) {
applog(LOG_INFO, client %s status error %d,
cli-addr_host, code);
code = InternalError;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 7/7] tabled: kill unused stor_get_enable
server/storage.c |9 -
1 file changed, 9 deletions(-)
Signed-off-by: Pete Zaitcev zait...@redhat.com
---
commit 32b1a2e9d8c481dfbf31222464aa2b6ef937ec4f
Author: Pete Zaitcev zait...@yahoo.com
Date: Thu Apr 1 19:20:09 2010 -0600
Dead function.
diff --git a/server/storage.c b/server/storage.c
index 7dab625..1793fa0 100644
--- a/server/storage.c
+++ b/server/storage.c
@@ -348,15 +348,6 @@ ssize_t stor_get_buf(struct open_chunk *cep, void *data,
size_t req_len)
return ret;
}
-void stor_get_enable(struct open_chunk *cep)
-{
- if (!cep-stc) {/* never happens */
- applog(LOG_ERR, Unopened chunk in stor_get_enable);
- return;
- }
-
-}
-
int stor_obj_del(struct storage_node *stn, uint64_t key)
{
struct st_client *stc;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 3/3] tabled: End-to-end verbosity
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
server/cldu.c |9 ++---
server/server.c | 11 ---
server/tabled.h |3 +--
3 files changed, 15 insertions(+), 8 deletions(-)
commit 820e53ea5ec85a69f6e1648e8d445a0d87ea6a72
Author: Master zait...@lembas.zaitcev.lan
Date: Wed Mar 31 18:27:00 2010 -0600
End-2-end verbosity.
diff --git a/server/cldu.c b/server/cldu.c
index aecf336..e705725 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -82,7 +82,7 @@ static void add_chunk_node(struct cld_session *sp, const char
*name);
static struct timeval cldu_rescan_delay = { 50, 0 };
-struct hail_log cldu_hail_log = {
+static struct hail_log cldu_hail_log = {
.func = applog,
};
@@ -223,7 +223,8 @@ static int cldu_set_cldc(struct cld_session *sp, int
newactive)
hp-host, hp-port);
sp-nsp = ncld_sess_open(hp-host, hp-port, error,
-cldu_sess_event, sp, tabled, tabled);
+cldu_sess_event, sp, tabled, tabled,
+cldu_hail_log);
if (sp-nsp == NULL) {
if (error 1000) {
applog(LOG_ERR, ncld_sess_open(%s,%u) error: %s,
@@ -581,12 +582,14 @@ void cld_init()
/*
* This initiates our sole session with a CLD instance.
*/
-int cld_begin(const char *thishost, const char *thisgroup)
+int cld_begin(const char *thishost, const char *thisgroup, int verbose)
{
static struct cld_session *sp = ses;
struct timespec tm;
int retry_cnt;
+ cldu_hail_log.verbose = verbose;
+
evtimer_set(ses.tm_rescan, cldu_tm_rescan, ses);
if (cldu_setgroup(sp, thisgroup, thishost)) {
diff --git a/server/server.c b/server/server.c
index e0d785c..8d03c15 100644
--- a/server/server.c
+++ b/server/server.c
@@ -78,6 +78,8 @@ static struct argp_option options[] = {
Write daemon process id to FILE },
{ foreground, 'F', NULL, 0,
Run in foreground, do not fork },
+ { verbose, 'v', NULL, 0,
+ Enable verbosity in libcldc },
{ }
};
@@ -90,6 +92,7 @@ static const struct argp argp = { options, parse_opt, NULL,
doc };
static bool server_running = true;
static bool use_syslog = true;
+static int verbose = 0;
int debugging = 0;
struct server tabled_srv = {
@@ -192,6 +195,9 @@ static error_t parse_opt (int key, char *arg, struct
argp_state *state)
case 'P':
tabled_srv.pid_file = strdup(arg);
break;
+ case 'v':
+ verbose = 1;
+ break;
case ARGP_KEY_ARG:
argp_usage(state); /* too many args */
break;
@@ -1829,8 +1835,7 @@ int main (int argc, char *argv[])
if (use_syslog)
openlog(PROGRAM_NAME, LOG_PID, LOG_LOCAL3);
if (debugging)
- applog(LOG_INFO, Verbose debug output enabled);
- cldu_hail_log.verbose = debugging;
+ applog(LOG_INFO, Debug output enabled);
/*
* now we can parse the configuration, errors to applog
@@ -1889,7 +1894,7 @@ int main (int argc, char *argv[])
if (rc)
goto err_out_net;
- if (cld_begin(tabled_srv.ourhost, tabled_srv.group) != 0) {
+ if (cld_begin(tabled_srv.ourhost, tabled_srv.group, verbose) != 0) {
rc = 1;
goto err_cld_session;
}
diff --git a/server/tabled.h b/server/tabled.h
index 72bf20d..73714f0 100644
--- a/server/tabled.h
+++ b/server/tabled.h
@@ -290,10 +290,9 @@ extern void cli_in_end(struct client *cli);
/* cldu.c */
extern void cld_init(void);
-extern int cld_begin(const char *fqdn, const char *group);
+extern int cld_begin(const char *fqdn, const char *group, int verbose);
extern void cldu_add_host(const char *host, unsigned int port);
extern void cld_end(void);
-extern struct hail_log cldu_hail_log;
/* util.c */
extern size_t strlist_len(GList *l);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCHv2] cld: use XDR for all messages
On Tue, 09 Feb 2010 04:38:21 -0500 Jeff Garzik j...@garzik.org wrote: On 02/09/2010 04:20 AM, Jeff Garzik wrote: Useful per-packet cld [the daemon] verbose output seems to have disappeared? Or did I miss a knob somewhere? cldcli no longers dumps per-packet debug info at maximum verbosity, either. Our built-in debug facilities have definitely regressed. The cldcli may be ncld's fault, let me look. I left the assignment of -verbose at least in one place. I did not touch anything at all on the daemon side though. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Does this work on BSD too?
When I run autogen.sh, this message is printed: lib/Makefile.am:13: `%'-style pattern rules are a GNU make extension It's because of this: %_rpc.h: %_rpc.x rpcgen -h $ $@ Is the above a concern? -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 1/1] CLD: Introduce the New CLD API
On Mon, 08 Feb 2010 21:06:17 -0500 Jeff Garzik j...@garzik.org wrote: - if (cldc_getaddr(host_list, hostb, NULL)) + if (cldc_getaddr(host_list, hostb, ncld_log)) return 1001; Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that. I did it before, but then every client, without exception, needed to include stdarg.h and provide the log. And what for? This is STUPID and proof is in the pudding: your getsrvinfo does not do anything of the sort. Might as well replace cldc_getadd with getsrvinfo and then everyone becomes happy. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 1/1] tabled: switch to ncld
No new function just yet, only a switch-over.
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
server/cldu.c | 789 +---
1 file changed, 215 insertions(+), 574 deletions(-)
diff --git a/server/cldu.c b/server/cldu.c
index 7e176d4..aecf336 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -29,8 +29,8 @@
#include unistd.h
#include event.h
#include errno.h
-#include cldc.h
#include elist.h
+#include ncld.h
#include tabled.h
#define ALIGN8(n) ((8 - ((n) 7)) 7)
@@ -49,60 +49,38 @@ struct cld_host {
struct cld_session {
bool forced_hosts; /* Administrator overrode default CLD */
- bool sess_open;
- struct cldc_udp *lib; /* library state */
- struct event lib_timer;
- int retry_cnt;
- int last_recv_err;
+ bool is_dead;
+ struct ncld_sess *nsp; /* library state */
/*
* For code sanity and being isomorphic with conventional programming
* using sleep(), neither of the timers must ever be active simultane-
* ously with any other. But using one timer structure is too annoying.
*/
- struct event tm_retry;
+ // struct event tm_relock;
struct event tm_rescan;
- struct event tm_reopen;
int actx; /* Active host cldv[actx] */
struct cld_host cldv[N_CLD];
char *thisgroup;
char *thishost;
- struct event ev;/* Associated with fd */
char *cfname; /* /tabled-group directory */
- struct cldc_fh *cfh;/* /tabled-group directory, keep open for scan
*/
+ struct ncld_fh *cfh;/* /tabled-group directory, keep open for scan
*/
char *ffname; /* /tabled-group/thishost */
- struct cldc_fh *ffh;/* /tabled-group/thishost, keep open for lock */
+ struct ncld_fh *ffh;/* /tabled-group/thishost, keep open for lock */
char *xfname; /* /chunk-GROUP directory */
- struct cldc_fh *xfh;/* /chunk-GROUP directory */
- char *yfname; /* /chunk-GROUP/NID file */
- struct cldc_fh *yfh;/* /chunk-GROUP/NID file */
struct list_head chunks;/* found in xfname, struct chunk_node */
};
static int cldu_set_cldc(struct cld_session *sp, int newactive);
-static int cldu_new_sess(struct cldc_call_opts *carg, enum cle_err_codes errc);
-static int cldu_open_c_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
-static int cldu_open_f_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
-static int cldu_lock_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
-static int cldu_put_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
-static int cldu_get_1_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
-static void try_open_x(struct cld_session *sp);
-static int cldu_open_x_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
-static int cldu_get_x_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
-static int cldu_close_x_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
-static void next_chunk(struct cld_session *sp);
-static int cldu_open_y_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
-static int cldu_get_y_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
-static int cldu_close_y_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
+static int scan_chunks(struct cld_session *sp);
+static void next_chunk(struct cld_session *sp, struct chunk_node *np);
static void add_remote(const char *name);
static void add_chunk_node(struct cld_session *sp, const char *name);
-static struct timeval cldu_retry_delay = { 5, 0 };
static struct timeval cldu_rescan_delay = { 50, 0 };
-static struct timeval cldu_reopen_delay = { 3, 0 };
struct hail_log cldu_hail_log = {
.func = applog,
@@ -169,170 +147,68 @@ err_oom:
return 0;
}
-static void cldu_tm_retry(int fd, short events, void *userdata)
-{
- struct cld_session *sp = userdata;
-
- if (++sp-retry_cnt = 5) {
- applog(LOG_INFO, Out of retries for %s, bailing, sp-xfname);
- exit(1);
- }
- if (debugging)
- applog(LOG_DEBUG, Trying to open %s, sp-xfname);
- try_open_x(sp);
-}
-
static void cldu_tm_rescan(int fd, short events, void *userdata)
{
struct cld_session *sp = userdata;
+ int newactive;
/* Add rescanning for tabled nodes as well. FIXME */
if (debugging)
applog(LOG_DEBUG, Rescanning for Chunks in %s, sp-xfname);
- try_open_x(sp);
-}
-
-static void cldu_tm_reopen(int fd, short events, void *userdata)
-{
- struct cld_session *sp = userdata;
-
- if (debugging)
- applog(LOG_DEBUG, Trying to reopen %d storage nodes,
- tabled_srv.num_stor);
- if (stor_update_cb() 1)
- evtimer_add(sp-tm_reopen
Re: [Patch 1/1] CLD: Introduce the New CLD API
On Mon, 08 Feb 2010 23:55:03 -0500 Jeff Garzik j...@garzik.org wrote: - if (cldc_getaddr(host_list, hostb, NULL)) + if (cldc_getaddr(host_list, hostb,ncld_log)) return 1001; Logging pointer should be supplied by the caller... that is the larger bug. Needing a no-op log function is just an indication of that. I did it before, but then every client, without exception, needed to includestdarg.h and provide the log. And what for? This is STUPID and proof is in the pudding: your getsrvinfo does not do anything of the sort. Possibly true -- but it is worse to introduce internal inconsistencies. It is far better to remove debugging statements completely than to create inconsistent usage. Hmm, you may be right about that. Might as well replace cldc_getadd with getsrvinfo and then everyone becomes happy. Seems a good fit :) How far along were you with getsrvinfo? I only have the header that you sent to the list just before Christmas. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Post-XDR CLD cannot keep session up
Hi, Jeff Colin: It looks like you broke something in CLD, not sure if server or client. There are two possibly related bugs. But first, here's the messages (The chunkd is run with -D). Note that I have 2 servers listed in DNS (both on port 4499), but only one is up. Feb 6 23:36:10 hitlain cld[1934]: databases up Feb 6 23:36:10 hitlain cld[1934]: Listening on :: port 4499 Feb 6 23:36:10 hitlain cld[1934]: initialized: verbose 0 Feb 6 23:37:10 hitlain chunkd[1967]: Verbose debug output enabled Feb 6 23:37:10 hitlain chunkd[1968]: cldc_saveaddr: found CLD host hitlain.zaitcev.lan prio 10 weight 50 Feb 6 23:37:10 hitlain chunkd[1968]: cldc_saveaddr: found CLD host elanor.zaitcev.lan prio 10 weight 50 Feb 6 23:37:10 hitlain chunkd[1968]: Selected CLD host hitlain.zaitcev.lan port 4499 Feb 6 23:37:10 hitlain chunkd[1968]: Listening on host :: port 8082 Feb 6 23:37:10 hitlain chunkd[1968]: initialized Feb 6 23:37:10 hitlain chunkd[1968]: New CLD session created, sid 05B521BF4071EBA2 Feb 6 23:37:10 hitlain chunkd[1968]: CLD file /chunk-default/2 created Feb 6 23:37:10 hitlain chunkd[1968]: CLD file /chunk-default/2 written Feb 6 23:39:45 hitlain chunkd[1968]: Session failed, sid 05B521BF4071EBA2 Feb 6 23:39:45 hitlain chunkd[1968]: Selected CLD host elanor.zaitcev.lan port 4499 Feb 6 23:39:45 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:39:50 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:39:55 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:00 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:05 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:10 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:15 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:21 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:26 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:31 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:36 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:41 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:46 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:51 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:40:56 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:01 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:06 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:11 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:16 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:21 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:26 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:31 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:36 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:41 hitlain chunkd[1968]: cldc_udp_receive_pkt failed: -111 Feb 6 23:41:46 hitlain chunkd[1968]: New CLD session creation failed: 17 Feb 6 23:41:46 hitlain chunkd[1968]: Session failed, sid 6C5A5E5D4D8F2270 Feb 6 23:41:46 hitlain chunkd[1968]: Selected CLD host hitlain.zaitcev.lan port 4499 Feb 6 23:41:46 hitlain chunkd[1968]: New CLD session created, sid 4E2A8ED73878F038 Feb 6 23:41:46 hitlain chunkd[1968]: CLD file /chunk-default/2 created Feb 6 23:41:46 hitlain chunkd[1968]: CLD lock(/chunk-default/2) failed: 11 So, first regression: session ALWAYS fails, for no reason I can see. It takes 2 minutes 35 seconds, as you can observe from the Session failed message. Second regression: locks of failed session are not removed (this is what code 11 is). Once the original session fails, CLD client cannot re-acquire the lock, ever, until the daemon is restarted. This definitely used work before the XDR, and it only takes 3 minutes to fail. Do you guys run and use chunkd or you just do make check and consider it done? I thought we talked about having virtually permanent cells and long-living CLD clients, because this sort of thing keeps cropping up. Cheers, -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: the evils of the CLD api
On Sat, 16 Jan 2010 17:54:36 -0800 Colin McCabe cmcc...@alumni.cmu.edu wrote: On Sat, Jan 16, 2010 at 3:21 PM, Pete Zaitcev zait...@redhat.com wrote: Shouldn't ncld_unlock and ncld_close return an error code rather than just eating all errors? The server certainly does sometimes send back error responses for these requests. This is an old bone of contention. The problem is, suppose the application is cleaning up for exit. What is it supposed to do with the error code? It cannot even retry sanely. If it's important enough for you to print out an error (*not* debug) message in the library, it should be important enough to pass along to the library user. After all, not everyone might be using a command-line app. Some people might want to see the error reflected in a dialog box, etc. I'm going to try and explain again why the tear-down calls must not return operation errors (they are free to return programmatic errors). This is an important point for API design. Suppose for a moment that ncld_end_sess returns an error. What is your application going to do about it? You seem to be suggesting to pop a dialog. But what is in the dialog for the user? What can a user do? The answer is: a) retry, b) resign to the loss of data and terminate the application forcefuly in the OS. Or, actually, first (a) and then (b). And for the (b) to work above OS you'll have a forceful tead-down method anyway (or rely on OS). What's more, some applications do not have a way to interact with a human (e.g. tabled). And this is where you got the logics completely wrong. When I pointed that requirements of such applications dictate a call that unconditionally ends the life of a structure or handle, you (correctly) pointed out that interactive applications exist too. But what of it?! The API must satisfy both kinds! The correct approach -- and only if you _really_ have no alternative to have some data outstanding or have other server interaction on close -- is to provide a separate flush method, which can fail. Then, the retry dialogs or other methods (e.g. copying of data into other handle) can be used sanely. -- Pete P.S. ncld does not have a flush method because its writes are synchronous. And if we cannot contact a server to close a session, it does not matter. So the whole discussion is moot really. Nonetheless I think it's important to have it, in general sense. -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 1/2] CLD: export a common timer implementation
Although we share the libtimer.a among the parts that can link it
statically, it turned out impossible to use from inside of libcldc
as it was. So, this patch makes it available to library code by
linking it in the same way we use libcommon.c already.
But if we do that, timer_add and friends become visible to all
users of libcldc, so rename them with cld_ prefix in order to
reduce namespace pollution.
Also, make the list of timers anchored outside of the library.
Not the we know of anyone who needs two in one program, but
it costs us almost nothing and is safer for the future.
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
include/libtimer.h | 19 ---
lib/Makefile.am|4 ++--
lib/libtimer.c | 25 ++---
server/Makefile.am |4 ++--
server/cld.h |8
server/server.c| 20 +---
server/session.c | 32 ++--
test/Makefile.am |1 -
test/it-works.c| 13 +++--
test/load-file-event.c | 13 +++--
test/lock-file-event.c | 21 +++--
test/save-file-event.c | 13 +++--
test/test.h|3 ++-
test/util.c|4 ++--
14 files changed, 97 insertions(+), 83 deletions(-)
commit d4eb73e875da95b7accca505eb8bd37c05e7c13d
Author: Master zait...@lembas.zaitcev.lan
Date: Fri Jan 15 23:27:46 2010 -0700
Move libtimer body inside libcldc, so it can be a shared library.
Add cld_ prefix to reduce namespace pollution.
This flips over all existing users inside CLD.
diff --git a/include/libtimer.h b/include/libtimer.h
index c10f993..ed74e68 100644
--- a/include/libtimer.h
+++ b/include/libtimer.h
@@ -6,21 +6,26 @@
#include string.h
#include time.h
-struct timer {
+struct cld_timer_list {
+ void *list;
+};
+
+struct cld_timer {
boolfired;
boolon_list;
- void(*cb)(struct timer *);
+ void(*cb)(struct cld_timer *);
void*userdata;
time_t expires;
charname[32];
};
-extern void timer_add(struct timer *timer, time_t expires);
-extern void timer_del(struct timer *timer);
-extern time_t timers_run(void);
+extern void cld_timer_add(struct cld_timer_list *tlist, struct cld_timer
*timer,
+ time_t expires);
+extern void cld_timer_del(struct cld_timer_list *tlist, struct cld_timer
*timer);
+extern time_t cld_timers_run(struct cld_timer_list *tlist);
-static inline void timer_init(struct timer *timer, const char *name,
- void (*cb)(struct timer *), void *userdata)
+static inline void cld_timer_init(struct cld_timer *timer, const char *name,
+ void (*cb)(struct cld_timer *), void *userdata)
{
memset(timer, 0, sizeof(*timer));
timer-cb = cb;
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 68be429..dfae79a 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -9,14 +9,14 @@ LINK = $(LIBTOOL) --mode=link $(CC) $(CFLAGS) $(LDFLAGS) -o $@
lib_LTLIBRARIES= libcldc.la
-libcldc_la_SOURCES = cldc.c cldc-udp.c cldc-dns.c common.c
+libcldc_la_SOURCES = cldc.c cldc-udp.c cldc-dns.c common.c libtimer.c
libcldc_la_LDFLAGS = \
-version-info $(LIBCLDC_CURRENT):$(LIBCLDC_REVISION):$(LIBCLDC_AGE) \
-no-undefined \
-export-symbols-regex ^[^_].*
-noinst_LIBRARIES = libtimer.a
+noinst_LIBRARIES =
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libcldc.pc
diff --git a/lib/libtimer.c b/lib/libtimer.c
index 5b13fbb..2c8e441 100644
--- a/lib/libtimer.c
+++ b/lib/libtimer.c
@@ -23,12 +23,10 @@
#include glib.h
#include libtimer.h
-static GList *timer_list;
-
-static gint timer_cmp(gconstpointer a_, gconstpointer b_)
+static gint cld_timer_cmp(gconstpointer a_, gconstpointer b_)
{
- const struct timer *a = a_;
- const struct timer *b = b_;
+ const struct cld_timer *a = a_;
+ const struct cld_timer *b = b_;
if (a-expires b-expires)
return 1;
@@ -37,8 +35,11 @@ static gint timer_cmp(gconstpointer a_, gconstpointer b_)
return -1;
}
-void timer_add(struct timer *timer, time_t expires)
+void cld_timer_add(struct cld_timer_list *tlist, struct cld_timer *timer,
+ time_t expires)
{
+ GList *timer_list = tlist-list;
+
if (timer-on_list)
timer_list = g_list_remove(timer_list, timer);
@@ -46,25 +47,26 @@ void timer_add(struct timer *timer, time_t expires)
timer-fired = false;
timer-expires = expires;
- timer_list = g_list_insert_sorted(timer_list, timer, timer_cmp);
+ tlist-list = g_list_insert_sorted(timer_list, timer, cld_timer_cmp);
}
-void timer_del(struct timer *timer)
+void cld_timer_del(struct cld_timer_list *tlist
[Patch 1/7] tabled: cleanup tabled.conf
The lib/ is a vestige of days when we stored files in a local filesystem, not Chunk. Just kill it. And as for CLD, it's a last resort configuration, not something we should encourage. So comment it out. Signed-Off-By: Pete Zaitcev zait...@redhat.com --- doc/etc.tabled.conf |8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) commit 9450703382d55f632a928a5aec8a414e50b2a0b1 Author: Master zait...@lembas.zaitcev.lan Date: Thu Jan 14 19:29:49 2010 -0700 Cleanup to tabled.conf example. diff --git a/doc/etc.tabled.conf b/doc/etc.tabled.conf index 9c0cb5f..b3e6d68 100644 --- a/doc/etc.tabled.conf +++ b/doc/etc.tabled.conf @@ -11,7 +11,7 @@ Port80/Port /Listen -TDB/path/tabled/lib/tdb/TDB +TDB/path/tabled/tdb/TDB TDBRepPort8083/TDBRepPort !-- @@ -21,10 +21,12 @@ a build of tabled. Consider them undocumented and a subject to change or removal at any time. -- -CLD +!-- + CLD Port8081/Port Hostlocalhost/Host -/CLD + /CLD +-- ChunkUsertest/ChunkUser ChunkKeytest/ChunkKey -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 2/7] tabled: add Cell element
Make the cell configurable. Among other things we need this for is
that you cannot migrate a tabled over when DB format changes, short
of running 2 CLDs.
We also add a few thoughts to the documentation, prompted by re-reading
of the document while adding the clause for Cell.
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
doc/etc.tabled.conf |7 +++
doc/setup.txt | 40 +---
server/cldu.c |4
server/config.c | 11 +++
server/server.c |2 +-
server/tabled.h |1 +
6 files changed, 61 insertions(+), 4 deletions(-)
commit 8c883a9b453b85d1da026f3638f074fe6d46df3f
Author: Master zait...@lembas.zaitcev.lan
Date: Thu Jan 14 19:37:59 2010 -0700
Add Cell configuration.
diff --git a/doc/etc.tabled.conf b/doc/etc.tabled.conf
index b3e6d68..f6c0bee 100644
--- a/doc/etc.tabled.conf
+++ b/doc/etc.tabled.conf
@@ -11,6 +11,13 @@
Port80/Port
/Listen
+!--
+ One cell per DB, don't skimp on cells. Also, make sure the replication
+ ports do not conflict when you make hosts to host several cells.
+ Unfortunately, the diagnostics are not very good if they do.
+ Most likely you'll see database corruption in such cases.
+ --
+Cellultracart2/Cell
TDB/path/tabled/tdb/TDB
TDBRepPort8083/TDBRepPort
diff --git a/doc/setup.txt b/doc/setup.txt
index 4338db4..0fd32da 100644
--- a/doc/setup.txt
+++ b/doc/setup.txt
@@ -10,13 +10,30 @@ _cld._udp.phx2.ex.com has SRV record 10 50 8081
pacman.phx2.ex.com.
_cld._udp.phx2.ex.com has SRV record 10 50 8081 maika.phx2.ex.com.
[r...@table1 ~]#
- If this does not work, set up CLD before proceeding.
+ If this does not work, STOP NOW. Then, talk to your DNS administrator.
Also, make sure that your hostname has a domain. We don't want to search
- for CLD in the DNS root, do we?
+ for CLD in the world-wide DNS root, do we?
+
+ Make sure CLD is up (run cldcli to verify).
+
+*) Another thing to set up in DNS is a wildcard host for the system where
+ tabled will run. Unlike the SRV records of CLD, this is optional, but
+ these days applications may presume that it exists. Here's the way it's
+ done in ISC BIND:
+
+emus3 IN A 192.168.128.9
+IN TXT Dell C8WQMC1
+IN fec0::1:219:b9ff:fe58:7ad6
+*.emus3 IN CNAME emus3
+
+ All examples on Google say FQDN is required, and most presume aliasing
+ of A and records, but BIND 9 eats the above fine.
*) Copy configuration file from doc/etc.tabled.conf to /etc/tabled.conf
- and edit to suit (see configurable items below).
+ and edit to suit (see configurable items below). Notice that the file
+ looks like XML, but is not really. In particular, names of elements are
+ case-sensitive.
*) The tabled writes its pid to /var/run/tabled.pid by default.
If you don't like the location, change it with PID tag.
@@ -24,6 +41,17 @@ _cld._udp.phx2.ex.com has SRV record 10 50 8081
maika.phx2.ex.com.
*) Create/choose TDB data directory, where Berkeley DB stores
data and metadata. Change this location with TDB tag.
+*) Assign the cell name:
+
+ Cellultracart3/Cell
+
+ Currently, a tabled process scans its cell, finds all other tabled
+ instances, and attempts to replicate its database to/from them.
+ In short, use one cell per business application.
+
+ Cell name defaults to default, so you can leave this element unset,
+ but don't do it. Any name, even qwerty, is better than the default.
+
*) Select the port to listen, if desired. This is done using the Listen
element:
@@ -31,6 +59,12 @@ _cld._udp.phx2.ex.com has SRV record 10 50 8081
maika.phx2.ex.com.
Port80/Port
/Listen
+ Default is port 80, which you should use unless the system in question
+ hosts an unrelated webserver too. This is because some libraries, such
+ as ancient Google Boto, may have bugs that prevent them from talking to
+ ports other than 80 (for unencrypted HTTP). If your target clients are
+ bug-free, use anything you like. The 8080 is a popular choice.
+
*) Initialize TDB, the metadata database. Load user/password pairs
into TDB, using tdbadm utility. The user/password pairs are
presented to tdbadm stdin in the following text format:
diff --git a/server/cldu.c b/server/cldu.c
index d02eafd..273f149 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -129,6 +129,10 @@ static int cldu_nextactive(struct cld_session *sp)
return sp-actx;
}
+/*
+ * Notice that for now we use the same cell name for both tabled and the
+ * chunkservers that it uses, so this function only takes one cell argument.
+ */
static int cldu_setcell(struct cld_session *sp,
const char *thiscell, const char *thishost)
{
diff --git a/server/config.c b/server/config.c
index 037176d..9d8ca87 100644
--- a/server/config.c
+++ b/server/config.c
@@ -317,6 +317,17 @@ static void
[Patch 4/7] tabled: retry conflicting locks
This problem was with us for a while, and even with this fix our start-up
is not reliable. But at least we will not be 100% guaranteed to hang as
before when restarting too quickly. So although the whole area needs some
serious reworking, this specific case was just too annoying to let it
continue.
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
server/cldu.c | 38 ++
1 file changed, 34 insertions(+), 4 deletions(-)
commit fa910aacff5118664177f988029cc5f8e6ef886d
Author: Master zait...@lembas.zaitcev.lan
Date: Thu Jan 14 19:56:13 2010 -0700
Retry the lock conflict.
diff --git a/server/cldu.c b/server/cldu.c
index 273f149..1d61672 100644
--- a/server/cldu.c
+++ b/server/cldu.c
@@ -59,6 +59,7 @@ struct cld_session {
* using sleep(), neither of the timers must ever be active simultane-
* ously with any other. But using one timer structure is too annoying.
*/
+ struct event tm_relock;
struct event tm_retry;
struct event tm_rescan;
struct event tm_reopen;
@@ -85,6 +86,7 @@ static int cldu_set_cldc(struct cld_session *sp, int
newactive);
static int cldu_new_sess(struct cldc_call_opts *carg, enum cle_err_codes errc);
static int cldu_open_c_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
static int cldu_open_f_cb(struct cldc_call_opts *carg, enum cle_err_codes
errc);
+static void try_lock(struct cld_session *sp);
static int cldu_lock_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
static int cldu_put_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
static int cldu_get_1_cb(struct cldc_call_opts *carg, enum cle_err_codes errc);
@@ -99,6 +101,7 @@ static int cldu_close_y_cb(struct cldc_call_opts *carg, enum
cle_err_codes errc)
static void add_remote(const char *name);
static void add_chunk_node(struct cld_session *sp, const char *name);
+static struct timeval cldu_relock_delay = { 10, 0 };
static struct timeval cldu_retry_delay = { 5, 0 };
static struct timeval cldu_rescan_delay = { 50, 0 };
static struct timeval cldu_reopen_delay = { 3, 0 };
@@ -168,6 +171,15 @@ err_oom:
return 0;
}
+static void cldu_tm_relock(int fd, short events, void *userdata)
+{
+ struct cld_session *sp = userdata;
+
+ if (debugging)
+ applog(LOG_DEBUG, Retrying locking of %s, sp-ffname);
+ try_lock(sp);
+}
+
static void cldu_tm_retry(int fd, short events, void *userdata)
{
struct cld_session *sp = userdata;
@@ -454,8 +466,6 @@ static int cldu_open_c_cb(struct cldc_call_opts *carg, enum
cle_err_codes errc)
static int cldu_open_f_cb(struct cldc_call_opts *carg, enum cle_err_codes errc)
{
struct cld_session *sp = carg-private;
- struct cldc_call_opts copts;
- int rc;
if (errc != CLE_OK) {
applog(LOG_ERR, CLD open(%s) failed: %d, sp-ffname, errc);
@@ -473,6 +483,15 @@ static int cldu_open_f_cb(struct cldc_call_opts *carg,
enum cle_err_codes errc)
if (debugging)
applog(LOG_DEBUG, CLD file \%s\ created, sp-ffname);
+ try_lock(sp);
+ return 0;
+}
+
+static void try_lock(struct cld_session *sp)
+{
+ struct cldc_call_opts copts;
+ int rc;
+
/*
* Lock the file, in case two hosts got the same hostname.
*/
@@ -483,8 +502,6 @@ static int cldu_open_f_cb(struct cldc_call_opts *carg, enum
cle_err_codes errc)
if (rc) {
applog(LOG_ERR, cldc_lock call error %d, rc);
}
-
- return 0;
}
static int cldu_lock_cb(struct cldc_call_opts *carg, enum cle_err_codes errc)
@@ -497,6 +514,18 @@ static int cldu_lock_cb(struct cldc_call_opts *carg, enum
cle_err_codes errc)
if (errc != CLE_OK) {
applog(LOG_ERR, CLD lock(%s) failed: %d, sp-ffname, errc);
+ if (errc == CLE_LOCK_CONFLICT) {
+ /*
+* The usual reason why we get a lock conflict is
+* restarting too quickly and hitting the previous lock
+* that is going to disappear soon.
+*
+* FIXME: However, it may also be that a master
+* is ok and we should become a slave, e.g. start TDB.
+* We do not support multi-node, but we should.
+*/
+ evtimer_add(sp-tm_relock, cldu_relock_delay);
+ }
return 0;
}
@@ -940,6 +969,7 @@ int cld_begin(const char *thishost, const char *thiscell)
{
static struct cld_session *sp = ses;
+ evtimer_set(ses.tm_relock, cldu_tm_relock, ses);
evtimer_set(ses.tm_retry, cldu_tm_retry, ses);
evtimer_set(ses.tm_rescan, cldu_tm_rescan, ses);
evtimer_set(ses.tm_reopen, cldu_tm_reopen, ses);
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message
[Patch 6/7] tabled: cleanup tabled.h
The structure server_socket is not even referenced anywhere in tabled.h.
Some kind of atavism, apparently.
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
server/tabled.h |1 -
1 file changed, 1 deletion(-)
commit ffcd630f41450d37493b6cad52a6ee11e5db7965
Author: Master zait...@lembas.zaitcev.lan
Date: Thu Jan 14 20:03:20 2010 -0700
Drop unused (anymore) forward declaration.
diff --git a/server/tabled.h b/server/tabled.h
index 633142a..b81e615 100644
--- a/server/tabled.h
+++ b/server/tabled.h
@@ -70,7 +70,6 @@ enum errcode {
struct client;
struct client_write;
-struct server_socket;
enum {
pat_auth,
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[Patch 7/7] tabled: cleanup config.c re. empty elements
I was changing it one by one, but honestly it was just stupid.
Let us change over to the sane checking once and for all.
Signed-Off-By: Pete Zaitcev zait...@redhat.com
---
server/config.c | 41 +
1 file changed, 33 insertions(+), 8 deletions(-)
commit 1d6ca69e3f5f2cae0d5630e785f69549d93080e5
Author: Master zait...@lembas.zaitcev.lan
Date: Thu Jan 14 20:04:27 2010 -0700
Switch cc-text checking completely to more precise variety.
diff --git a/server/config.c b/server/config.c
index 6fc7a88..301be66 100644
--- a/server/config.c
+++ b/server/config.c
@@ -158,7 +158,11 @@ static void cfg_elm_end (GMarkupParseContext *context,
struct stat statb;
long n;
- if (!strcmp(element_name, PID) cc-text) {
+ if (!strcmp(element_name, PID)) {
+ if (!cc-text) {
+ applog(LOG_WARNING, PID element empty);
+ return;
+ }
if (tabled_srv.pid_file) {
/* Silent about command line override. */
free(cc-text);
@@ -168,13 +172,22 @@ static void cfg_elm_end (GMarkupParseContext *context,
cc-text = NULL;
}
- else if (!strcmp(element_name, ForceHost) cc-text) {
+ else if (!strcmp(element_name, ForceHost)) {
+ if (!cc-text) {
+ applog(LOG_WARNING, ForceHost element empty);
+ return;
+ }
free(tabled_srv.ourhost);
tabled_srv.ourhost = cc-text;
cc-text = NULL;
}
- else if (!strcmp(element_name, TDB) cc-text) {
+ else if (!strcmp(element_name, TDB)) {
+ if (!cc-text) {
+ applog(LOG_WARNING, TDB element empty);
+ return;
+ }
+
if (stat(cc-text, statb) 0) {
applog(LOG_ERR, stat(2) on TDB '%s' failed: %s,
cc-text, strerror(errno));
@@ -192,7 +205,12 @@ static void cfg_elm_end (GMarkupParseContext *context,
cc-text = NULL;
}
- else if (!strcmp(element_name, TDBRepPort) cc-text) {
+ else if (!strcmp(element_name, TDBRepPort)) {
+ if (!cc-text) {
+ applog(LOG_WARNING, TDBRepPort element empty);
+ return;
+ }
+
n = strtol(cc-text, NULL, 10);
if (n = 0 || n = 65536) {
applog(LOG_WARNING,
@@ -244,8 +262,7 @@ static void cfg_elm_end (GMarkupParseContext *context,
if (cc-in_listen) {
n = strtol(cc-text, NULL, 10);
- if ((n 0 n 65536) ||
- !strcmp(cc-text, auto)) {
+ if ((n 0 n 65536) || !strcmp(cc-text, auto)) {
free(cc-tmp_listen.port);
cc-tmp_listen.port = cc-text;
} else {
@@ -306,13 +323,21 @@ static void cfg_elm_end (GMarkupParseContext *context,
cc-text = NULL;
}
- else if (!strcmp(element_name, ChunkUser) cc-text) {
+ else if (!strcmp(element_name, ChunkUser)) {
+ if (!cc-text) {
+ applog(LOG_WARNING, ChunkUser element empty);
+ return;
+ }
free(tabled_srv.chunk_user);
tabled_srv.chunk_user = cc-text;
cc-text = NULL;
}
- else if (!strcmp(element_name, ChunkKey) cc-text) {
+ else if (!strcmp(element_name, ChunkKey)) {
+ if (!cc-text) {
+ applog(LOG_WARNING, ChunkKey element empty);
+ return;
+ }
free(tabled_srv.chunk_key);
tabled_srv.chunk_key = cc-text;
cc-text = NULL;
--
To unsubscribe from this list: send the line unsubscribe hail-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] cld: use XDR for all messages
On Wed, 13 Jan 2010 16:03:45 -0500 Jeff Garzik j...@garzik.org wrote: Well, this definitely does not build as-is. lib/Makefile.am needs BUILT_SOURCES = cld_msg_rpc.h otherwise nothing builds at all, because cld_msg_rpc.h does not exist. test/Makefile.am and tools/Makefile.am both need -I$(top_srcdir)/lib added to their INCLUDES. [] Kernel places generated files into the top include directory, e.g. autoconf.h and asm-offsets.h for asm/. It's probably easier than piling up include paths. I dunno. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 4/4] chunkd: add self-checking
On Tue, 12 Jan 2010 09:21:24 -0500 Jeff Garzik j...@garzik.org wrote: sleep(n) self_check() algorithm seems less useful to the average admin than a slightly more complex one that solves the problem defined as guarantee an object is checked at least every N days. Because, as the dataset grows beyond a test database measures in megabytes, the dataset scan time will dwarf the self-check sleep period. The behavior becomes one of constant scanning, with a tiny period of rest in between. That's obvious. You also forgot to recall that your fat node exhacerbates the problem, but conversely, splitting them up helps. The single-threaded design is on purpose: it provides a crude method of load control. But that aside, two things about your scheme: - how do you select N? It is no different from n in its arbitrariness. - as the data set grows beyond a test database, it is going to take more work to satisfy N. What you are proposing is actually no different. It is not more adaptive. It seems like either (a) tracking the total dataset size and total dataset scan time, or (b) tracking and modifying per-object last-self-check times, would be needed. Well, that would be nice. Still, we are powerless to keep the scan time down when the dataset grows. At best, we can sacrificy the mean detection time and constrain the power that the scan consumes. I'm thinking about doing just that actually. But first I'm going to implement a reporting scheme in tabled. Also of relevance to admins is scan start time. This patch would have a scan begin at essentially random times throughout the day or night, in particular occurring during the most heavily-trafficked portion of the day. An algorithm that occurs on hour N every day (or hour N, day M of each week) is much more predictable, regardless of dataset size. So, you want the mean time to detect to be at least 12 hours. I can do that if you insist. Another option is to add an administrative command START SCAN, and permit an external utility, scheduled by cron or executed by make check, to be the entity that starts the scan thread in the background. That would permit maximum flexibility for both the admin and our testsuite. We can do that too, I guess. Just don't know when. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 1/2] tabled: fix bugs in streaming of data
On Thu, 07 Jan 2010 17:10:49 -0500 Jeff Garzik j...@garzik.org wrote: I was thinking specifically that chunkd may want the in-progress xfer codes changes in your patch. chunkd, tabled, nfs4d (and ancient storaged) all share the same basic write-in-progress and TCP data xfer code. I'll take a look, no need to worry about it yourself. At least 1 of the 3 that I fixed definitely was a corruption that I introduced to your design when I adapted it to simultaneous streaming on write. Not sure about other two, but maybe I just screwed tabled. But sure, if you have a moment to re-check chunkd, be my guest. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Patch 2/2] tabled: add a test for larger objects
On Tue, 05 Jan 2010 04:14:08 -0500 Jeff Garzik j...@garzik.org wrote: a follow-up patch that checks the checksums of the data sent/received would be nice... That would require more coding than using single check bytes because a checksum can span a block boundary. But ok, you're right, I'll do it. -- Pete -- To unsubscribe from this list: send the line unsubscribe hail-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
