Re: rbd STDIN import does not work / wip-rbd-export-stdout
Am 27.11.2012 23:50, schrieb Josh Durgin: On 11/27/2012 01:16 AM, Stefan Priebe - Profihost AG wrote: Am 26.11.2012 21:00, schrieb Dan Mick: As for fstrim: on the rbd image?sure, if it's a filesystem, it ought to work (modulo some bugs I've heard about with 32-bit vs. 64-bit offsets in qemu... :) ) it works fine with my patches i send to qemu. But right now they're no upstream an nobody of the qemu people answers anymore. So right now i'll keep my patches and be lucky others can't use RBD and fstrim. So maybe inktank should push this a bit more to the qemu people. If you could send another version with PATCHv3 instead of PATCH in the subject, and a description of the changes since the last version between the --- and the diffstat, the qemu folks would probably appreciate it. No idea how to archieve this with git send-email ;-( But still more important is also the patch for discards... scripts/checkpatch.pl in qemu.git will warn you about style issues like the conditionals without braces. Thank! 0001-rbd-block-driver-fix-race-between-aio-completition-a.patch has no obvious style problems and is ready for submission. Stefan -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Upgrade a running cluster to bobtail ? (when it will be out :p)
Hi, I'd like to know the recommended upgrade procedure when bobtail is out ? I've read in the past in the list that rolling upgrade between stable release was going to be possible, but exactly what is the way to do this ? Upgrade OSD host by host, then upgrade MON one by one ? Or do the MON first ? Cheers, Sylvain -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 04/30] ceph_mon.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/ceph_mon.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ceph_mon.cc b/src/ceph_mon.cc index da4529f..f82be18 100644 --- a/src/ceph_mon.cc +++ b/src/ceph_mon.cc @@ -16,7 +16,6 @@ #include sys/stat.h #include fcntl.h -#include sys/stat.h #include iostream #include string using namespace std; -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 03/30] ceph_mds.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/ceph_mds.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ceph_mds.cc b/src/ceph_mds.cc index 6187948..e6928fb 100644 --- a/src/ceph_mds.cc +++ b/src/ceph_mds.cc @@ -16,7 +16,6 @@ #include sys/stat.h #include fcntl.h -#include sys/stat.h #include iostream #include string using namespace std; -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 07/30] client/Client.cc: remove twice included headers
Fix includes: remove twice included common/config.h. Remove include of sys/param.h in special __FreeBSD__ section, since this file is included in general anyway Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/client/Client.cc | 6 -- 1 file changed, 6 deletions(-) diff --git a/src/client/Client.cc b/src/client/Client.cc index 35cba96..b808f20 100644 --- a/src/client/Client.cc +++ b/src/client/Client.cc @@ -25,10 +25,6 @@ #include sys/statvfs.h -#if defined(__FreeBSD__) -#include sys/param.h -#endif - #include iostream using namespace std; @@ -67,8 +63,6 @@ using namespace std; #include common/admin_socket.h #include common/errno.h -#include common/config.h - #define dout_subsys ceph_subsys_client #include include/lru.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 02/30] ceph_fuse.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/ceph_fuse.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ceph_fuse.cc b/src/ceph_fuse.cc index e645777..09aadb4 100644 --- a/src/ceph_fuse.cc +++ b/src/ceph_fuse.cc @@ -37,7 +37,6 @@ using namespace std; #endif // DARWIN #include sys/types.h -#include sys/stat.h #include fcntl.h void usage() -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 08/30] client/MetaRequest.cc: remove twice included client/Dentry.h
Fix includes: remove twice included client/Dentry.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/client/MetaRequest.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/client/MetaRequest.cc b/src/client/MetaRequest.cc index 205e201..2c50f06 100644 --- a/src/client/MetaRequest.cc +++ b/src/client/MetaRequest.cc @@ -5,7 +5,6 @@ #include client/MetaRequest.h #include client/Dentry.h #include client/Inode.h -#include client/Dentry.h #include messages/MClientReply.h #include common/Formatter.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 09/30] client/SyntheticClient.cc: remove twice included headers
Fix includes: remove twice included common/config.h and sys/types.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/client/SyntheticClient.cc | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/client/SyntheticClient.cc b/src/client/SyntheticClient.cc index f081244..b2a936f 100644 --- a/src/client/SyntheticClient.cc +++ b/src/client/SyntheticClient.cc @@ -29,12 +29,10 @@ using namespace std; #include sys/types.h #include sys/stat.h #include fcntl.h -#include sys/types.h #include utime.h #include math.h #include sys/statvfs.h -#include common/config.h #include include/assert.h #define dout_subsys ceph_subsys_client -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 05/30] ceph_osd.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/ceph_osd.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ceph_osd.cc b/src/ceph_osd.cc index 0652089..8fbeb75 100644 --- a/src/ceph_osd.cc +++ b/src/ceph_osd.cc @@ -17,7 +17,6 @@ #include fcntl.h #include uuid/uuid.h -#include sys/stat.h #include iostream #include string using namespace std; -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 06/30] ceph_syn.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/ceph_syn.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/ceph_syn.cc b/src/ceph_syn.cc index a087e9f..8ffe40a 100644 --- a/src/ceph_syn.cc +++ b/src/ceph_syn.cc @@ -36,7 +36,6 @@ using namespace std; #endif // DARWIN || __FreeBSD__ #include sys/types.h -#include sys/stat.h #include fcntl.h extern int syn_filer_flags; -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 01/30] fusetrace_ll.cc: remove twice included sys/time.h
Fix includes: remove twice included sys/time.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- fusetrace/fusetrace_ll.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/fusetrace/fusetrace_ll.cc b/fusetrace/fusetrace_ll.cc index 9a0a076..1b0d45f 100644 --- a/fusetrace/fusetrace_ll.cc +++ b/fusetrace/fusetrace_ll.cc @@ -30,7 +30,6 @@ #ifdef HAVE_SETXATTR #include sys/xattr.h #endif -#include sys/time.h #include time.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 12/30] monmaptool.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/monmaptool.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/monmaptool.cc b/src/monmaptool.cc index 4c99fdd..5870e5f 100644 --- a/src/monmaptool.cc +++ b/src/monmaptool.cc @@ -17,7 +17,6 @@ #include fcntl.h #include errno.h -#include sys/stat.h #include iostream #include string using namespace std; -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 13/30] include/buffer.h: remove twice included stdio.h
Fix includes: remove twice included stdio.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/include/buffer.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/include/buffer.h b/src/include/buffer.h index 4a9e96c..9a635bd 100644 --- a/src/include/buffer.h +++ b/src/include/buffer.h @@ -46,7 +46,6 @@ void *valloc(size_t); #endif #include stdint.h -#include stdio.h // snprintf #include string.h #ifndef __CYGWIN__ -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 14/30] librados/RadosClient.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/librados/RadosClient.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/librados/RadosClient.cc b/src/librados/RadosClient.cc index ae56c76..7e76b65 100644 --- a/src/librados/RadosClient.cc +++ b/src/librados/RadosClient.cc @@ -16,7 +16,6 @@ #include sys/stat.h #include fcntl.h -#include sys/stat.h #include iostream #include string #include pthread.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 15/30] mds/Dumper.h: remove twice included msg/Messenger.h
Fix includes: remove twice included msg/Messenger.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/mds/Dumper.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/mds/Dumper.h b/src/mds/Dumper.h index d9af3aa..066ba5c 100644 --- a/src/mds/Dumper.h +++ b/src/mds/Dumper.h @@ -19,7 +19,6 @@ #include osdc/Journaler.h #include msg/Dispatcher.h #include msg/Messenger.h -#include msg/Messenger.h #include auth/Auth.h /** -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 16/30] mds/Migrator.cc: remove twice included Migrator.h
Fix includes: remove twice included Migrator.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/mds/Migrator.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/mds/Migrator.cc b/src/mds/Migrator.cc index 9829d68..b66b54c 100644 --- a/src/mds/Migrator.cc +++ b/src/mds/Migrator.cc @@ -19,7 +19,6 @@ #include CDentry.h #include Migrator.h #include Locker.h -#include Migrator.h #include Server.h #include MDBalancer.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 17/30] mds/Resetter.h: remove twice included msg/Messenger.h
Fix includes: remove twice included msg/Messenger.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/mds/Resetter.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/mds/Resetter.h b/src/mds/Resetter.h index 663dde1..b3bc067 100644 --- a/src/mds/Resetter.h +++ b/src/mds/Resetter.h @@ -19,7 +19,6 @@ #include osdc/Journaler.h #include msg/Dispatcher.h #include msg/Messenger.h -#include msg/Messenger.h #include auth/Auth.h /** -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 30/30] tools/common.cc: remove twice included headers
Cleanup includes: Remove twice included sys/stat.h and osd/OSDMap.h.
Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de
---
src/tools/common.cc | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/tools/common.cc b/src/tools/common.cc
index c500568..9a454f7 100644
--- a/src/tools/common.cc
+++ b/src/tools/common.cc
@@ -23,7 +23,6 @@ using namespace std;
#include memory
#include sys/types.h
-#include sys/stat.h
#include fcntl.h
extern C {
@@ -81,7 +80,6 @@ float tick_interval = 3.0;
// observe (push)
#include mon/PGMap.h
-#include osd/OSDMap.h
#include mds/MDSMap.h
#include common/LogEntry.h
--
1.8.0
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 25/30] osd/OSD.h: remove twice included OpRequest.h
Fix includes: remove twice included OpRequest.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/osd/OSD.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/osd/OSD.h b/src/osd/OSD.h index ff277e1..ab6e3ac 100644 --- a/src/osd/OSD.h +++ b/src/osd/OSD.h @@ -49,7 +49,6 @@ using namespace std; #include ext/hash_set using namespace __gnu_cxx; -#include OpRequest.h #include common/shared_cache.hpp #include common/simple_cache.hpp #include common/sharedptr_registry.hpp -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 24/30] os/IndexManager.cc: remove twice included FlatIndex.h
Fix includes: remove twice included FlatIndex.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/os/IndexManager.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/os/IndexManager.cc b/src/os/IndexManager.cc index 2165036..85281c4 100644 --- a/src/os/IndexManager.cc +++ b/src/os/IndexManager.cc @@ -30,7 +30,6 @@ #include IndexManager.h #include FlatIndex.h #include HashIndex.h -#include FlatIndex.h #include CollectionIndex.h #include chain_xattr.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 26/30] osdmaptool.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/osdmaptool.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/osdmaptool.cc b/src/osdmaptool.cc index c28926b..d367c36 100644 --- a/src/osdmaptool.cc +++ b/src/osdmaptool.cc @@ -17,7 +17,6 @@ #include fcntl.h #include errno.h -#include sys/stat.h #include iostream #include string using namespace std; -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 27/30] rados.cc: remove twice included headers
Fix includes: remove twice included common/obj_bencher.h and common/errno.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/rados.cc | 4 1 file changed, 4 deletions(-) diff --git a/src/rados.cc b/src/rados.cc index b06f703..a870255 100644 --- a/src/rados.cc +++ b/src/rados.cc @@ -18,8 +18,6 @@ #include rados_sync.h using namespace librados; -#include common/obj_bencher.h - #include common/config.h #include common/ceph_argparse.h #include global/global_init.h @@ -42,8 +40,6 @@ using namespace librados; #include climits #include locale -#include common/errno.h - #include cls/lock/cls_lock_client.h int rados_tool_sync(const std::map std::string, std::string opts, -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 28/30] remove twice included linux/types.h
Cleanup includes: remove twice included linux/types.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- qa/workunits/direct_io/test_sync_io.c | 1 - 1 file changed, 1 deletion(-) diff --git a/qa/workunits/direct_io/test_sync_io.c b/qa/workunits/direct_io/test_sync_io.c index 1a84fee..613631e 100644 --- a/qa/workunits/direct_io/test_sync_io.c +++ b/qa/workunits/direct_io/test_sync_io.c @@ -12,7 +12,6 @@ //#include ../client/ioctl.h #include linux/ioctl.h -#include linux/types.h #define CEPH_IOCTL_MAGIC 0x97 #define CEPH_IOC_SYNCIO _IO(CEPH_IOCTL_MAGIC, 5) -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 22/30] os/DBObjectMap.cc: remove twice included headers
Fix includes: remove twice included string and tr1/memory Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/os/DBObjectMap.cc | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/os/DBObjectMap.cc b/src/os/DBObjectMap.cc index 83b746d..10b7b70 100644 --- a/src/os/DBObjectMap.cc +++ b/src/os/DBObjectMap.cc @@ -8,9 +8,7 @@ #include string #include tr1/memory -#include string #include vector -#include tr1/memory #include ObjectMap.h #include KeyValueDB.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 29/30] rgw/rgw_rest.cc: remove twice included rgw_formats.h
Cleanup includes: Remove twice included rgw_formats.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/rgw/rgw_rest.cc | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/rgw/rgw_rest.cc b/src/rgw/rgw_rest.cc index 99eb090..9d978d1 100644 --- a/src/rgw/rgw_rest.cc +++ b/src/rgw/rgw_rest.cc @@ -12,8 +12,6 @@ #include rgw_rest_s3.h #include rgw_swift_auth.h -#include rgw_formats.h - #include rgw_client_io.h #include rgw_resolve.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 11/30] crushtool.cc: remove twice included sys/stat.h
Fix includes: remove twice included sys/stat.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/crushtool.cc | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/crushtool.cc b/src/crushtool.cc index 271cd15..0f54852 100644 --- a/src/crushtool.cc +++ b/src/crushtool.cc @@ -17,8 +17,6 @@ #include fcntl.h #include errno.h -#include sys/stat.h - #include fstream #include common/debug.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 10/30] common/pipe.c: remove twice included unistd.h
Fix includes: remove twice included unistd.h
Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de
---
src/common/pipe.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/common/pipe.c b/src/common/pipe.c
index e35b6fe..9c01b32 100644
--- a/src/common/pipe.c
+++ b/src/common/pipe.c
@@ -17,7 +17,6 @@
#include errno.h
#include fcntl.h
#include unistd.h
-#include unistd.h
int pipe_cloexec(int pipefd[2])
{
--
1.8.0
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 21/30] mount/canonicalize.c: remove twice included string.h
Fix includes: remove twice included string.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/mount/canonicalize.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/mount/canonicalize.c b/src/mount/canonicalize.c index c3bdb38..7561e41 100644 --- a/src/mount/canonicalize.c +++ b/src/mount/canonicalize.c @@ -24,7 +24,6 @@ #include string.h #include ctype.h #include unistd.h -#include string.h #include errno.h #include stdlib.h #include limits.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 19/30] messages/MClientRequest.h: remove twice included sys/types.h
Fix includes: remove twice included sys/types.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/messages/MClientRequest.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/messages/MClientRequest.h b/src/messages/MClientRequest.h index e8ce763..9dc9383 100644 --- a/src/messages/MClientRequest.h +++ b/src/messages/MClientRequest.h @@ -40,7 +40,6 @@ #include sys/types.h #include utime.h #include sys/stat.h -#include sys/types.h #include fcntl.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 18/30] mds/journal.cc: remove twice included common/config.h
Fix includes: remove twice included common/config.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/mds/journal.cc | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/mds/journal.cc b/src/mds/journal.cc index 04b1a92..46adbf2 100644 --- a/src/mds/journal.cc +++ b/src/mds/journal.cc @@ -51,9 +51,6 @@ #include Locker.h - -#include common/config.h - #define dout_subsys ceph_subsys_mds #undef DOUT_COND #define DOUT_COND(cct, l) (l=cct-_conf-debug_mds || l = cct-_conf-debug_mds_log \ -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 20/30] mon/Monitor.cc: remove twice included osd/OSDMap.h
Fix includes: remove twice included osd/OSDMap.h Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/mon/Monitor.cc | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/mon/Monitor.cc b/src/mon/Monitor.cc index 79f2577..b819fce 100644 --- a/src/mon/Monitor.cc +++ b/src/mon/Monitor.cc @@ -65,8 +65,6 @@ #include LogMonitor.h #include AuthMonitor.h -#include osd/OSDMap.h - #include auth/AuthMethodList.h #include auth/KeyRing.h -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 23/30] os/DBObjectMap.h: remove twice included string
Fix includes: remove twice included string Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de --- src/os/DBObjectMap.h | 1 - 1 file changed, 1 deletion(-) diff --git a/src/os/DBObjectMap.h b/src/os/DBObjectMap.h index 331..18c6ce4 100644 --- a/src/os/DBObjectMap.h +++ b/src/os/DBObjectMap.h @@ -7,7 +7,6 @@ #include map #include string -#include string #include vector #include tr1/memory #include boost/scoped_ptr.hpp -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: rbd STDIN import does not work / wip-rbd-export-stdout
Hi Dan, thanks for your update / new import-export branch. The import seems to work now. What i'm wondering is that it is really slow. I see around 10% CPU Usage on importing host (host where rbd import is started) and just 4MB every 2s raising Data on ceph. This means 40GB took around 5-6 hours. Night this be due to he size update? 4MB every 2s looks like the stripe size. Greets, Stefan -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] fix build of unittest_formatter
Add CRYPTO_CXXFLAGS to unittest_formatter_CXXFLAGS to find pk11pub.h to
be included in src/common/ceph_crypto.h.
Signed-off-by: Danny Al-Gaaf danny.al-g...@bisect.de
---
src/Makefile.am | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/Makefile.am b/src/Makefile.am
index be851d6..2f6f486 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -764,7 +764,7 @@ check_PROGRAMS += unittest_heartbeatmap
unittest_formatter_SOURCES = test/formatter.cc rgw/rgw_formats.cc
unittest_formatter_LDFLAGS = $(PTHREAD_CFLAGS) ${AM_LDFLAGS}
unittest_formatter_LDADD = ${UNITTEST_LDADD} $(LIBGLOBAL_LDA)
-unittest_formatter_CXXFLAGS = ${AM_CXXFLAGS} ${UNITTEST_CXXFLAGS}
+unittest_formatter_CXXFLAGS = ${CRYPTO_CXXFLAGS} ${AM_CXXFLAGS}
${UNITTEST_CXXFLAGS}
check_PROGRAMS += unittest_formatter
unittest_libcephfs_config_SOURCES = test/libcephfs_config.cc
--
1.8.0
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: chooseleaf_descend_once
Hey Jim, Running the third test with tunable chooseleaf_descend_once 0 with no devices marked out yields the following result (999.827397, 0.48667056652539997) so chi squared value is 999 with a corresponding p value of 0.487 so that the placement distribution seems to be drawn from the uniform distribution as desired. Caleb -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: chooseleaf_descend_once
On 11/28/2012 09:11 AM, Caleb Miles wrote:
Hey Jim,
Running the third test with tunable chooseleaf_descend_once 0 with no
devices marked out yields the following result
(999.827397, 0.48667056652539997)
so chi squared value is 999 with a corresponding p value of 0.487 so that
the placement distribution seems to be drawn from the uniform distribution
as desired.
Great, thanks for doing that extra test.
Plus, I see that Sage has merged it. Cool.
Thanks -- Jim
Caleb
On Tue, Nov 27, 2012 at 1:28 PM, Jim Schuttjasc...@sandia.gov wrote:
Hi Caleb,
On 11/26/2012 07:28 PM, caleb miles wrote:
Hello all,
Here's what I've done to try and validate the new chooseleaf_descend_once
tunable first described in commit f1a53c5e80a48557e63db9c52b83f3**9391bc69b8
in the wip-crush branch of ceph.git.
First I set the new tunable to it's legacy value, disabled,
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 0
The map contains one thousand osd devices contained in one hundred hosts
with the following data rule
rule data {
ruleset 0
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
}
I then simulate the creation of one million placement groups using the
crushtool
$ crushtool -i hundred.map --test --min-x 0 --max-x 99 --num-rep 3
--output-csv --weight 120 0.0 --weight 121 0.0 --weight 122 0.0 --weight
123 0.0 --weight 124 0.0 --weight 125 0.0 --weight 125 0.0 --weight 150 0.0
--weight 151 0.0 --weight 152 0.0 --weight 153 0.0 --weight 154 0.0
--weight 155 0.0 --weight 156 0.0 --weight 180 0.0 --weight 181 0.0
--weight 182 0.0 --weight 183 0.0 --weight 184 0.0 --weight 185 0.0
--weight 186 0.0
with the majority of devices in three hosts marked out. Then in (I)Python
import scipy.stats as s
import matplotlib.mlab as m
data = m.csv2rec(data-device_**utilization.csv)
s.chisquare(data['number_of_**objects_stored'], data['number_of_objects_*
*expected'])
which will output
(122939.76474477499, 0.0)
so that the chi squared value is 122939.795 and the p value is, rounded
to, 0.0 and the observed placement distribution statistically differs from
a uniform distribution. Repeating with the new tunable set to
tunable chooseleaf_descend_once 1
I obtain the following result
(998.97643161876761, 0.32151775131589833)
so that the chi squared value is 998.976 and the p value is 0.32 and the
observed placement distribution is statistically identical to the uniform
distribution at the five and ten percent confidence levels, higher as well
of course. The p value is the probability of obtaining a chi squared value
more extreme than the statistic observed. Basically, from my rudimentary
understanding of probability theory, that if you obtain a p value p P
then reject the null hypothesis, in our case that the observed placement
distribution is drawn from the uniform distribution, at the P confidence
level.
Cool. Thanks for doing these tests.
Is there any point to doing a third test, with
tunable chooseleaf_descend_once 0
and no devices marked out, but in all other respects
the same as the above two tests?
I would expect the results for that case and the last
case you tested to be essentially identical in the degree
of uniformity, but is it worth verifying?
-- Jim
Caleb
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at
http://vger.kernel.org/**majordomo-info.htmlhttp://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: rbd STDIN import does not work / wip-rbd-export-stdout
On 11/28/2012 12:09 AM, Stefan Priebe - Profihost AG wrote: Am 27.11.2012 23:50, schrieb Josh Durgin: On 11/27/2012 01:16 AM, Stefan Priebe - Profihost AG wrote: Am 26.11.2012 21:00, schrieb Dan Mick: As for fstrim: on the rbd image?sure, if it's a filesystem, it ought to work (modulo some bugs I've heard about with 32-bit vs. 64-bit offsets in qemu... :) ) it works fine with my patches i send to qemu. But right now they're no upstream an nobody of the qemu people answers anymore. So right now i'll keep my patches and be lucky others can't use RBD and fstrim. So maybe inktank should push this a bit more to the qemu people. If you could send another version with PATCHv3 instead of PATCH in the subject, and a description of the changes since the last version between the --- and the diffstat, the qemu folks would probably appreciate it. No idea how to archieve this with git send-email ;-( But still more important is also the patch for discards... Use git format-patch, edit the patch file (it includes the basic headers already), then send it with git send-email. Your return value type change was already merged into master of qemu.git as 08448d5195aeff49bf25fb62b4a6218f079f5284. Josh -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Multiple CEPH roles on single platform
Another rookie question: What's the thinking on having the same server perform multiple CEPH roles (e.g. osd(s), and monitor roles)? I think this would mainly be a facto for smaller/cost sensitive environments. Tim -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/30] cleanup includes
Merged, thanks! sage On Wed, 28 Nov 2012, Danny Al-Gaaf wrote: This patch series removes several twice included header files from the source files. Danny Al-Gaaf (30): fusetrace_ll.cc: remove twice included sys/time.h ceph_fuse.cc: remove twice included sys/stat.h ceph_mds.cc: remove twice included sys/stat.h ceph_mon.cc: remove twice included sys/stat.h ceph_osd.cc: remove twice included sys/stat.h ceph_syn.cc: remove twice included sys/stat.h client/Client.cc: remove twice included headers client/MetaRequest.cc: remove twice included client/Dentry.h client/SyntheticClient.cc: remove twice included headers common/pipe.c: remove twice included unistd.h crushtool.cc: remove twice included sys/stat.h monmaptool.cc: remove twice included sys/stat.h include/buffer.h: remove twice included stdio.h librados/RadosClient.cc: remove twice included sys/stat.h mds/Dumper.h: remove twice included msg/Messenger.h mds/Migrator.cc: remove twice included Migrator.h mds/Resetter.h: remove twice included msg/Messenger.h mds/journal.cc: remove twice included common/config.h messages/MClientRequest.h: remove twice included sys/types.h mon/Monitor.cc: remove twice included osd/OSDMap.h mount/canonicalize.c: remove twice included string.h os/DBObjectMap.cc: remove twice included headers os/DBObjectMap.h: remove twice included string os/IndexManager.cc: remove twice included FlatIndex.h osd/OSD.h: remove twice included OpRequest.h osdmaptool.cc: remove twice included sys/stat.h rados.cc: remove twice included headers remove twice included linux/types.h rgw/rgw_rest.cc: remove twice included rgw_formats.h tools/common.cc: remove twice included headers fusetrace/fusetrace_ll.cc | 1 - qa/workunits/direct_io/test_sync_io.c | 1 - src/ceph_fuse.cc | 1 - src/ceph_mds.cc | 1 - src/ceph_mon.cc | 1 - src/ceph_osd.cc | 1 - src/ceph_syn.cc | 1 - src/client/Client.cc | 6 -- src/client/MetaRequest.cc | 1 - src/client/SyntheticClient.cc | 2 -- src/common/pipe.c | 1 - src/crushtool.cc | 2 -- src/include/buffer.h | 1 - src/librados/RadosClient.cc | 1 - src/mds/Dumper.h | 1 - src/mds/Migrator.cc | 1 - src/mds/Resetter.h| 1 - src/mds/journal.cc| 3 --- src/messages/MClientRequest.h | 1 - src/mon/Monitor.cc| 2 -- src/monmaptool.cc | 1 - src/mount/canonicalize.c | 1 - src/os/DBObjectMap.cc | 2 -- src/os/DBObjectMap.h | 1 - src/os/IndexManager.cc| 1 - src/osd/OSD.h | 1 - src/osdmaptool.cc | 1 - src/rados.cc | 4 src/rgw/rgw_rest.cc | 2 -- src/tools/common.cc | 2 -- 30 files changed, 46 deletions(-) -- 1.8.0 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OSD and MON memory usage
What replication level are you using? -Sam On Tue, Nov 27, 2012 at 9:23 AM, Cláudio Martins c...@ist.utl.pt wrote: On Fri, 23 Nov 2012 16:46:00 + Joao Eduardo Luis joao.l...@inktank.com wrote: On 11/16/2012 05:24 PM, Cláudio Martins wrote: As for the monitor daemon on this cluster (running on a dedicated machine), it is currently using 3.2GB of memory, and it got to that point again in a matter of minutes after being restarted. Would it be good if we tested with the changes from the wip-mon-leaks-fix branch? Following up on this, wip-mon-leaks-fix was merged into master a couple of days ago. If you have the chance to check if that fixes your memory consumption issues on the monitor, it would be much appreciated! Hi João, I've had a chance to test it and it does indeed seem to make a big difference on mon memory usage. As for the OSD memory usage issue, it's still looking really bad. I'm preparing to do more testing and send more info about this, but a lot of unrelated stuff crept up this week and things are going slowly on this front. I hope to talk more about this before the weekend. Thanks! Cláudio -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OSD and MON memory usage
On Wed, 28 Nov 2012 13:00:17 -0800 Samuel Just sam.j...@inktank.com wrote: What replication level are you using? Hi, The replication level is 3. Thanks Cláudio -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OSD and MON memory usage
Can you post the output of ceph -s? -Sam On Wed, Nov 28, 2012 at 1:05 PM, Cláudio Martins c...@ist.utl.pt wrote: On Wed, 28 Nov 2012 13:00:17 -0800 Samuel Just sam.j...@inktank.com wrote: What replication level are you using? Hi, The replication level is 3. Thanks Cláudio -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: endless flying slow requests
Did you observe hung io along with that error? Both sub_op_commit and
sub_op_applied have happened, so the sub_op_reply should have been
sent back to the primary. This looks more like a leak. If you also
observed hung io, then it's possible that the problem is occurring
between the sub_op_applied event and the response.
-Sam
On Tue, Nov 27, 2012 at 11:47 PM, Andrey Korolyov and...@xdel.ru wrote:
On Wed, Nov 28, 2012 at 5:51 AM, Sage Weil s...@inktank.com wrote:
Hi Stefan,
On Thu, 15 Nov 2012, Sage Weil wrote:
On Thu, 15 Nov 2012, Stefan Priebe - Profihost AG wrote:
Am 14.11.2012 15:59, schrieb Sage Weil:
Hi Stefan,
I would be nice to confirm that no clients are waiting on replies for
these requests; currently we suspect that the OSD request tracking is
the
buggy part. If you query the OSD admin socket you should be able to
dump
requests and see the client IP, and then query the client.
Is it librbd? In that case you likely need to change the config so that
it is listening on an admin socket ('admin socket = path').
Yes it is. So i have to specify admin socket at the KVM host?
Right. IIRC the disk line is a ; (or \;) separated list of key/value
pairs.
How do i query the admin socket for requests?
ceph --admin-daemon /path/to/socket help
ceph --admin-daemon /path/to/socket objecter_dump (i think)
Were you able to reproduce this?
Thanks!
sage
Meanwhile, I did. :)
Such requests will always be created if you have restarted or marked
an osd out and then back in and scrub didn`t happen in the meantime
(after such operation and before request arrival).
What is more interesting, the hangup happens not exactly at the time
of operation, but tens of minutes later.
{ description: osd_sub_op(client.1292013.0:45422 4.731
a384cf31\/rbd_data.1415fb1075f187.00a7\/head\/\/4 [] v
16444'21693 snapset=0=[]:[] snapc=0=[]),
received_at: 2012-11-28 03:54:43.094151,
age: 27812.942680,
duration: 2.676641,
flag_point: started,
events: [
{ time: 2012-11-28 03:54:43.094222,
event: waiting_for_osdmap},
{ time: 2012-11-28 03:54:43.386890,
event: reached_pg},
{ time: 2012-11-28 03:54:43.386894,
event: started},
{ time: 2012-11-28 03:54:43.386973,
event: commit_queued_for_journal_write},
{ time: 2012-11-28 03:54:45.360049,
event: write_thread_in_journal_buffer},
{ time: 2012-11-28 03:54:45.586183,
event: journaled_completion_queued},
{ time: 2012-11-28 03:54:45.586262,
event: sub_op_commit},
{ time: 2012-11-28 03:54:45.770792,
event: sub_op_applied}]}]}
sage
Stefan
On Wed, 14 Nov 2012, Stefan Priebe - Profihost AG wrote:
Hello list,
i see this several times. Endless flying slow requests. And they never
stop
until i restart the mentioned osd.
2012-11-14 10:11:57.513395 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31789.858457 secs
2012-11-14 10:11:57.513399 osd.24 [WRN] slow request 31789.858457
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:58.513584 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31790.858646 secs
2012-11-14 10:11:58.513586 osd.24 [WRN] slow request 31790.858646
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:59.513766 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31791.858827 secs
2012-11-14 10:11:59.513768 osd.24 [WRN] slow request 31791.858827
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:12:00.513909 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31792.858971 secs
2012-11-14 10:12:00.513916 osd.24 [WRN] slow request 31792.858971
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:12:01.514061 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31793.859124 secs
2012-11-14 10:12:01.514063 osd.24 [WRN] slow request 31793.859124
seconds
old,
received at 2012-11-14 01:22:07.654922:
Re: endless flying slow requests
Also, these clusters aren't mixed argonaut and next, are they? (Not
that that shouldn't work, but it would be a useful data point.)
-Sam
On Wed, Nov 28, 2012 at 1:11 PM, Samuel Just sam.j...@inktank.com wrote:
Did you observe hung io along with that error? Both sub_op_commit and
sub_op_applied have happened, so the sub_op_reply should have been
sent back to the primary. This looks more like a leak. If you also
observed hung io, then it's possible that the problem is occurring
between the sub_op_applied event and the response.
-Sam
On Tue, Nov 27, 2012 at 11:47 PM, Andrey Korolyov and...@xdel.ru wrote:
On Wed, Nov 28, 2012 at 5:51 AM, Sage Weil s...@inktank.com wrote:
Hi Stefan,
On Thu, 15 Nov 2012, Sage Weil wrote:
On Thu, 15 Nov 2012, Stefan Priebe - Profihost AG wrote:
Am 14.11.2012 15:59, schrieb Sage Weil:
Hi Stefan,
I would be nice to confirm that no clients are waiting on replies for
these requests; currently we suspect that the OSD request tracking is
the
buggy part. If you query the OSD admin socket you should be able to
dump
requests and see the client IP, and then query the client.
Is it librbd? In that case you likely need to change the config so
that
it is listening on an admin socket ('admin socket = path').
Yes it is. So i have to specify admin socket at the KVM host?
Right. IIRC the disk line is a ; (or \;) separated list of key/value
pairs.
How do i query the admin socket for requests?
ceph --admin-daemon /path/to/socket help
ceph --admin-daemon /path/to/socket objecter_dump (i think)
Were you able to reproduce this?
Thanks!
sage
Meanwhile, I did. :)
Such requests will always be created if you have restarted or marked
an osd out and then back in and scrub didn`t happen in the meantime
(after such operation and before request arrival).
What is more interesting, the hangup happens not exactly at the time
of operation, but tens of minutes later.
{ description: osd_sub_op(client.1292013.0:45422 4.731
a384cf31\/rbd_data.1415fb1075f187.00a7\/head\/\/4 [] v
16444'21693 snapset=0=[]:[] snapc=0=[]),
received_at: 2012-11-28 03:54:43.094151,
age: 27812.942680,
duration: 2.676641,
flag_point: started,
events: [
{ time: 2012-11-28 03:54:43.094222,
event: waiting_for_osdmap},
{ time: 2012-11-28 03:54:43.386890,
event: reached_pg},
{ time: 2012-11-28 03:54:43.386894,
event: started},
{ time: 2012-11-28 03:54:43.386973,
event: commit_queued_for_journal_write},
{ time: 2012-11-28 03:54:45.360049,
event: write_thread_in_journal_buffer},
{ time: 2012-11-28 03:54:45.586183,
event: journaled_completion_queued},
{ time: 2012-11-28 03:54:45.586262,
event: sub_op_commit},
{ time: 2012-11-28 03:54:45.770792,
event: sub_op_applied}]}]}
sage
Stefan
On Wed, 14 Nov 2012, Stefan Priebe - Profihost AG wrote:
Hello list,
i see this several times. Endless flying slow requests. And they
never
stop
until i restart the mentioned osd.
2012-11-14 10:11:57.513395 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31789.858457 secs
2012-11-14 10:11:57.513399 osd.24 [WRN] slow request 31789.858457
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:58.513584 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31790.858646 secs
2012-11-14 10:11:58.513586 osd.24 [WRN] slow request 31790.858646
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:59.513766 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31791.858827 secs
2012-11-14 10:11:59.513768 osd.24 [WRN] slow request 31791.858827
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:12:00.513909 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31792.858971 secs
2012-11-14 10:12:00.513916 osd.24 [WRN] slow request 31792.858971
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:12:01.514061 osd.24 [WRN] 1
Re: Multiple CEPH roles on single platform
Hi Tim, It's all up to you whether you assign more roles to a system or not. What actually happens is that you need to start a daemon for each of the ceph services. Each of the systems in your ceph cluster will most likely run at least one of those daemons, but it is not required that all system run all ceph daemons (as a matter of fact, that's highly unlikely as you will probably have most systems running the ceph-osd service since they actually contain the block storage devices...) For instance, see the example config here; http://ceph.com/docs/master/rados/configuration/ceph-conf/#example-ceph-conf You have to to be aware of a few things though. Mainly; you need at least one and preferably 3 or 5 different systems (odd number!) running the monitor service (ceph-mon). Failovers for the metadata service is also nice of course, and a cluster with one system running the ceph-osd daemon is... well... not a ceph cluster :-) It all boils down to the load (usage, size) on your cluster how many systems you want to have performing which role. Ceph doesn't constrain you to one particular setup. - Roald On Wed, Nov 28, 2012 at 8:10 PM, Snider, Tim tim.sni...@netapp.com wrote: Another rookie question: What's the thinking on having the same server perform multiple CEPH roles (e.g. osd(s), and monitor roles)? I think this would mainly be a facto for smaller/cost sensitive environments. Tim -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OSD and MON memory usage
On Wed, 28 Nov 2012 13:08:08 -0800 Samuel Just sam.j...@inktank.com wrote:
Can you post the output of ceph -s?
'ceph -s' right now gives
health HEALTH_WARN 923 pgs degraded; 8666 pgs down; 9606 pgs peering; 7 pgs
recovering; 406 pgs recovery_wait; 3769 pgs stale; 9606 pgs stuck inactive;
3769 pgs stuck stale; 11052 pgs stuck unclean; recovery 121068/902868 degraded
(13.409%); 4824/300956 unfound (1.603%); 2/18 in osds are down
monmap e1: 1 mons at {0=193.136.128.202:6789/0}, election epoch 1, quorum 0 0
osdmap e7669: 62 osds: 16 up, 18 in
pgmap v47643: 12480 pgs: 35 active, 1223 active+clean, 129 stale+active,
321 active+recovery_wait, 198 stale+active+clean, 236 peering, 2
active+remapped, 2 stale+active+recovery_wait, 6126 down+peering, 249
active+degraded, 2 stale+active+recovering+degraded, 598 stale+peering, 7
active+clean+scrubbing, 29 active+recovery_wait+remapped, 2067
stale+down+peering, 618 stale+active+degraded, 52
active+recovery_wait+degraded, 61 remapped+peering, 365 down+remapped+peering,
2 stale+active+recovery_wait+degraded, 45 stale+remapped+peering, 108
stale+down+remapped+peering, 5 active+recovering; 1175 GB data, 1794 GB used,
25969 GB / 27764 GB avail; 121068/902868 degraded (13.409%); 4824/300956
unfound (1.603%)
mdsmap e1: 0/0/1 up
The cluster has been in this state since the last attempt to get it
going. I added about 100GB of swap on each machine to avoid the OOM
killer. Running like this resulted in the machines trashing wildly and
getting to ~2000 load avg, and after a while the osds started
dying/commited suicide, but *not* from OOM. Some of the few that remain
have bloated to around 1.9GB of mem usage.
If you want, I can try to restart the whole thing tomorrow and collect
fresh log output from the dying OSDs, or any other action or debug info
that you might find useful.
Thanks!
Cláudio
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: endless flying slow requests
I actually have this exact issue with hung io during these slow requests. The
odd thing is when I restarted the osd that was being mentioned via ceph -w it
didn't resume io. A bit later (unsure on exact time) another one started being
listed in ceph -w and when I restarted that different osd io came back. I do
believe these occur when I reboot nodes. I never reboot more than one at a
time and I have 3 replicas in the cluster.
I have a pure ceph version 0.53
(commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c) cluster.
Regards,
Shawn
From: ceph-devel-ow...@vger.kernel.org [ceph-devel-ow...@vger.kernel.org] on
behalf of Samuel Just [sam.j...@inktank.com]
Sent: Wednesday, November 28, 2012 16:12
To: Andrey Korolyov
Cc: Sage Weil; Stefan Priebe - Profihost AG; ceph-devel@vger.kernel.org
Subject: Re: endless flying slow requests
Also, these clusters aren't mixed argonaut and next, are they? (Not
that that shouldn't work, but it would be a useful data point.)
-Sam
On Wed, Nov 28, 2012 at 1:11 PM, Samuel Just sam.j...@inktank.com wrote:
Did you observe hung io along with that error? Both sub_op_commit and
sub_op_applied have happened, so the sub_op_reply should have been
sent back to the primary. This looks more like a leak. If you also
observed hung io, then it's possible that the problem is occurring
between the sub_op_applied event and the response.
-Sam
On Tue, Nov 27, 2012 at 11:47 PM, Andrey Korolyov and...@xdel.ru wrote:
On Wed, Nov 28, 2012 at 5:51 AM, Sage Weil s...@inktank.com wrote:
Hi Stefan,
On Thu, 15 Nov 2012, Sage Weil wrote:
On Thu, 15 Nov 2012, Stefan Priebe - Profihost AG wrote:
Am 14.11.2012 15:59, schrieb Sage Weil:
Hi Stefan,
I would be nice to confirm that no clients are waiting on replies for
these requests; currently we suspect that the OSD request tracking is
the
buggy part. If you query the OSD admin socket you should be able to
dump
requests and see the client IP, and then query the client.
Is it librbd? In that case you likely need to change the config so
that
it is listening on an admin socket ('admin socket = path').
Yes it is. So i have to specify admin socket at the KVM host?
Right. IIRC the disk line is a ; (or \;) separated list of key/value
pairs.
How do i query the admin socket for requests?
ceph --admin-daemon /path/to/socket help
ceph --admin-daemon /path/to/socket objecter_dump (i think)
Were you able to reproduce this?
Thanks!
sage
Meanwhile, I did. :)
Such requests will always be created if you have restarted or marked
an osd out and then back in and scrub didn`t happen in the meantime
(after such operation and before request arrival).
What is more interesting, the hangup happens not exactly at the time
of operation, but tens of minutes later.
{ description: osd_sub_op(client.1292013.0:45422 4.731
a384cf31\/rbd_data.1415fb1075f187.00a7\/head\/\/4 [] v
16444'21693 snapset=0=[]:[] snapc=0=[]),
received_at: 2012-11-28 03:54:43.094151,
age: 27812.942680,
duration: 2.676641,
flag_point: started,
events: [
{ time: 2012-11-28 03:54:43.094222,
event: waiting_for_osdmap},
{ time: 2012-11-28 03:54:43.386890,
event: reached_pg},
{ time: 2012-11-28 03:54:43.386894,
event: started},
{ time: 2012-11-28 03:54:43.386973,
event: commit_queued_for_journal_write},
{ time: 2012-11-28 03:54:45.360049,
event: write_thread_in_journal_buffer},
{ time: 2012-11-28 03:54:45.586183,
event: journaled_completion_queued},
{ time: 2012-11-28 03:54:45.586262,
event: sub_op_commit},
{ time: 2012-11-28 03:54:45.770792,
event: sub_op_applied}]}]}
sage
Stefan
On Wed, 14 Nov 2012, Stefan Priebe - Profihost AG wrote:
Hello list,
i see this several times. Endless flying slow requests. And they
never
stop
until i restart the mentioned osd.
2012-11-14 10:11:57.513395 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31789.858457 secs
2012-11-14 10:11:57.513399 osd.24 [WRN] slow request 31789.858457
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:58.513584 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31790.858646 secs
2012-11-14 10:11:58.513586 osd.24 [WRN] slow request 31790.858646
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write
Re: endless flying slow requests
On Thu, Nov 29, 2012 at 1:12 AM, Samuel Just sam.j...@inktank.com wrote:
Also, these clusters aren't mixed argonaut and next, are they? (Not
that that shouldn't work, but it would be a useful data point.)
-Sam
On Wed, Nov 28, 2012 at 1:11 PM, Samuel Just sam.j...@inktank.com wrote:
Did you observe hung io along with that error? Both sub_op_commit and
sub_op_applied have happened, so the sub_op_reply should have been
sent back to the primary. This looks more like a leak. If you also
observed hung io, then it's possible that the problem is occurring
between the sub_op_applied event and the response.
-Sam
It is relatively easy to check if one of client VMs has locked one or
more cores to iowait or just hangs, so yes, these ops are related to
real commit operations and they are hanged.
I`m using all-new 0.54 cluster, without mixing of course. Does
everyone who hit that bug readjusted cluster before bug shows
itself(say, in a day-long distance)?
On Tue, Nov 27, 2012 at 11:47 PM, Andrey Korolyov and...@xdel.ru wrote:
On Wed, Nov 28, 2012 at 5:51 AM, Sage Weil s...@inktank.com wrote:
Hi Stefan,
On Thu, 15 Nov 2012, Sage Weil wrote:
On Thu, 15 Nov 2012, Stefan Priebe - Profihost AG wrote:
Am 14.11.2012 15:59, schrieb Sage Weil:
Hi Stefan,
I would be nice to confirm that no clients are waiting on replies for
these requests; currently we suspect that the OSD request tracking is
the
buggy part. If you query the OSD admin socket you should be able to
dump
requests and see the client IP, and then query the client.
Is it librbd? In that case you likely need to change the config so
that
it is listening on an admin socket ('admin socket = path').
Yes it is. So i have to specify admin socket at the KVM host?
Right. IIRC the disk line is a ; (or \;) separated list of key/value
pairs.
How do i query the admin socket for requests?
ceph --admin-daemon /path/to/socket help
ceph --admin-daemon /path/to/socket objecter_dump (i think)
Were you able to reproduce this?
Thanks!
sage
Meanwhile, I did. :)
Such requests will always be created if you have restarted or marked
an osd out and then back in and scrub didn`t happen in the meantime
(after such operation and before request arrival).
What is more interesting, the hangup happens not exactly at the time
of operation, but tens of minutes later.
{ description: osd_sub_op(client.1292013.0:45422 4.731
a384cf31\/rbd_data.1415fb1075f187.00a7\/head\/\/4 [] v
16444'21693 snapset=0=[]:[] snapc=0=[]),
received_at: 2012-11-28 03:54:43.094151,
age: 27812.942680,
duration: 2.676641,
flag_point: started,
events: [
{ time: 2012-11-28 03:54:43.094222,
event: waiting_for_osdmap},
{ time: 2012-11-28 03:54:43.386890,
event: reached_pg},
{ time: 2012-11-28 03:54:43.386894,
event: started},
{ time: 2012-11-28 03:54:43.386973,
event: commit_queued_for_journal_write},
{ time: 2012-11-28 03:54:45.360049,
event: write_thread_in_journal_buffer},
{ time: 2012-11-28 03:54:45.586183,
event: journaled_completion_queued},
{ time: 2012-11-28 03:54:45.586262,
event: sub_op_commit},
{ time: 2012-11-28 03:54:45.770792,
event: sub_op_applied}]}]}
sage
Stefan
On Wed, 14 Nov 2012, Stefan Priebe - Profihost AG wrote:
Hello list,
i see this several times. Endless flying slow requests. And they
never
stop
until i restart the mentioned osd.
2012-11-14 10:11:57.513395 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31789.858457 secs
2012-11-14 10:11:57.513399 osd.24 [WRN] slow request 31789.858457
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:58.513584 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31790.858646 secs
2012-11-14 10:11:58.513586 osd.24 [WRN] slow request 31790.858646
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14 10:11:59.513766 osd.24 [WRN] 1 slow requests, 1 included
below;
oldest blocked for 31791.858827 secs
2012-11-14 10:11:59.513768 osd.24 [WRN] slow request 31791.858827
seconds
old,
received at 2012-11-14 01:22:07.654922: osd_op(client.30286.0:6719
rbd_data.75c55bf2fdd7.1399 [write 282624~4096]
3.3f6d2373) v4
currently delayed
2012-11-14
Re: OSD and MON memory usage
Hi, If you want, I can try to restart the whole thing tomorrow and collect fresh log output from the dying OSDs, or any other action or debug info that you might find useful. Is the clock synchronized on all machines ? What you describe (growing mem, recovery that doesn't seem to end) seems pretty similar to what I experienced when clocks of OSD were off ... Cheers, Sylvain -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OSD and MON memory usage
On Thu, 29 Nov 2012 00:13:25 +0100 Sylvain Munaut s.mun...@whatever-company.com wrote: Hi, If you want, I can try to restart the whole thing tomorrow and collect fresh log output from the dying OSDs, or any other action or debug info that you might find useful. Is the clock synchronized on all machines ? Yup. All machines synched by ntp. Cláudio What you describe (growing mem, recovery that doesn't seem to end) seems pretty similar to what I experienced when clocks of OSD were off ... -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: parsing in the ceph osd subsystem
On Wednesday, November 28, 2012 at 11:45 PM, Andrey Korolyov wrote: $ ceph osd down - osd.0 is already down $ ceph osd down --- osd.0 is already down the same for ``+'', ``/'', ``%'' and so - I think that for osd subsys ceph cli should explicitly work only with positive integers plus zero, refusing all other input. Yes indeed! This is already fixed in bobtail (and v0.54, maybe?), and I believe Joao went through and audited all our input parsing pretty carefully. Certainly things are much better than they were. -Greg -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
