Hi, Sanjit,
tail of RangeServer's log :
2009-07-22 09:39:01,395 1351514432 Hypertable.RangeServer [INFO]
(RangeServer/AccessGroup.cc:372) Starting InMemory Compaction
of METADATA[0:<FF><FF>..<FF><FF>](location)
2009-07-22 09:39:01,397 1351514432 Hypertable.RangeServer [DEBUG]
create (RangeServer/CellStoreV1.cc:171): bloom-filter-mode=1
max-approx-items=1000 false-positive=0.01
2009-07-22 09:39:01,401 1351514432 Hypertable.RangeServer [DEBUG]
create_bloom_filter (RangeServer/CellStoreV1.cc:180): Creating new
BloomFilter for CellStore '/hypertable/tables/METADATA/location/
AB2A0D28DE6B77FFDD6C72AF/cs391' for estimated 416 items
2009-07-22 09:39:01,401 1351514432 Hypertable.RangeServer [DEBUG]
BasicBloomFilter (Common/BloomFilter.h:54): num funcs=6 num
bits=3987 num bytes=499 bits per element=9.58413
2009-07-22 09:39:01,402 1351514432 Hypertable.RangeServer [DEBUG]
create_bloom_filter (RangeServer/CellStoreV1.cc:193): Create
d new BloomFilter for CellStore '/hypertable/tables/METADATA/location/
AB2A0D28DE6B77FFDD6C72AF/cs391'
2009-07-22 09:39:01,412 1246615872 Hypertable.RangeServer [DEBUG]
update (RangeServer/RangeServer.cc:1136): Update: {TableIdentifier:
name='METADATA' id=0 generation=1}
2009-07-22 09:39:01,412 1246615872 Hypertable.RangeServer [DEBUG]
(RangeServer/RangeServer.cc:1371) Added 1 (0 split off) updates to
'METADATA'
2009-07-22 09:39:01,415 1246615872 Hypertable.RangeServer [DEBUG] add
(RangeServer/Range.cc:274): key=control=(REV|TS|SHARED)
row='0:<FF><FF>' family=2 qualifier='location' ts=1248226741412525001
rev=1248226741412525001 value='/hypertable/tables/M...'
2009-07-22 09:39:01,415 1351514432 Hypertable.RangeServer [INFO]
(RangeServer/AccessGroup.cc:533) Finished Compaction of METADATA
[0:<FF><FF>..<FF><FF>](location)
2009-07-22 09:39:01,415 1351514432 Hypertable.RangeServer [INFO]
(RangeServer/AccessGroup.cc:379) Starting Major Compaction of METADATA
[0:<FF><FF>..<FF><FF>](logging)
2009-07-22 09:39:01,418 1351514432 Hypertable.RangeServer [DEBUG]
create (RangeServer/CellStoreV1.cc:171): bloom-filter-mode=1 max-
approx-items=1000 false-positive=0.01
2009-07-22 09:39:01,418 1351514432 Hypertable.RangeServer [DEBUG]
matches_deleted_row (RangeServer/MergeScanner.h:67): filtering deleted
row '2:<FF><FF>' vs '3:<FF><FF>'
2009-07-22 09:39:01,418 1351514432 Hypertable.RangeServer [DEBUG]
create_bloom_filter (RangeServer/CellStoreV1.cc:180): Creating new
BloomFilter for CellStore '/hypertable/tables/METADATA/logging/
AB2A0D28DE6B77FFDD6C72AF/cs0' for 0 items
2009-07-22 09:39:01,418 1351514432 Hypertable.RangeServer [FATAL]
(Common/BloomFilter.h:47) failed expectation: m_num_bits != 0
Thanks
--kuer
On 7月22日, 上午9时44分, kuer <[email protected]> wrote:
> Hi, Sanjit,
>
> with --debug option, I get some logging message, but the file is big,
> how to share it with you?
>
> gdb backtrace of core files
>
> (gdb) bt
> #0 0x0000000000538272 in
> Hypertable::BasicBloomFilter<Hypertable::MurmurHash2>::BasicBloomFilter
> ()
> #1 0x000000000053d3be in Hypertable::CellStoreV1::create_bloom_filter
> ()
> #2 0x000000000053e10e in Hypertable::CellStoreV1::finalize ()
> #3 0x000000000051f112 in Hypertable::AccessGroup::run_compaction ()
> #4 0x0000000000504e45 in Hypertable::Range::split_compact_and_shrink
> ()
> #5 0x0000000000509310 in Hypertable::Range::split ()
> #6 0x00000000004ec693 in
> Hypertable::MaintenanceQueue::Worker::operator() ()
> #7 0x00000000006a5c40 in thread_proxy ()
> #8 0x00000038ae406367 in start_thread () from /lib64/libpthread.so.0
> #9 0x00000038ad8d2f7d in clone () from /lib64/libc.so.6
>
> -- kuer
>
> On 7月22日, 上午9时07分, Sanjit Jhala <[email protected]> wrote:
>
> > Hi Kuer,
>
> > This looks like a bug in the RangeServer code. The RangeServer is
> > trying to create a CellStore file and while creating the CellStore's
> > BloomFilter its hitting an error condition.
>
> > Can you try a couple of things to help debug this issue?
>
> > Firstly turn on the RangeServer debug logging and report RangeServer
> > logs. You can do this by adding the global option --debug to your
> > start-all-servers.sh command line. Example: <$HYPERTABLE_INSTALL_DIR>/
> > bin/start-all-servers.sh kfs --debug
>
> > Secondly, if you could compile a debug build and send the stack trace
> > that would be helpful. To do this, from your hypertable build
> > directory run
> > ccmake <$HYPERTABLE_SRC_DIR> and make sure CMAKE_BUILD_TYPE is set to
> > Debug and install the new build. After you try to bring up the
> > RangeServer and it dumps core, you can load the core file in gdb (Eg:
> > gdb gdb <$HYPERTABLE_INSTALL_DIR>/bin/Hypertable.RangeServer <
> > $CORE_FILE>). You can run bt (backtrace) in gdb to get the stack trace.
>
> > -Sanjit
>
> > On Jul 21, 2009, at 5:36 PM, kuer wrote:
>
> > > Hi, all,
>
> > > one of RangeServers hangs after coredump and restarting . here are
> > > messages in rangeserver's log :
>
> > > 2009-07-22 08:23:41,448 1295067456 Hypertable.RangeServer [WARN] (Lib/
> > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because
> > > 1246607682171649001 >= 1246607682128108001 (file='/hypertable/servers/
> > > 221.194.134.173_31060/log/root/0')
> > > 2009-07-22 08:23:41,448 1295067456 Hypertable.RangeServer [WARN] (Lib/
> > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because
> > > 1248187695757932563 >= 1247819802453791364 (file='/hypertable/servers/
> > > 221.194.134.173_31060/log/metadata/2')
> > > 2009-07-22 08:23:41,448 1295067456 Hypertable.RangeServer [WARN] (Lib/
> > > CommitLog.cc:250) clgc LOG FRAGMENT PURGE breaking because
> > > 1248193806824860161 >= 1248189458336849002 (file='/hypertable/servers/
> > > 221.194.134.173_31060/log/user/401')
> > > 2009-07-22 08:23:41,448 1295067456 Hypertable.RangeServer [INFO]
> > > (RangeServer/MaintenancePrioritizerLogCleanup.cc:103) Adding
> > > maintenance for range METADATA[0: .. ] because mid-split(1)
> > > 2009-07-22 08:23:41,449 1295067456 Hypertable.RangeServer [INFO]
> > > (RangeServer/RangeServer.cc:2032) Memory Usage: 312320288 bytes
> > > 2009-07-22 08:23:41,449 1378986304 Hypertable.RangeServer [INFO]
> > > (RangeServer/AccessGroup.cc:379) Starting Major Compaction of METADATA
> > > [0: .. ](default)
> > > 2009-07-22 08:23:41,529 1378986304 Hypertable.RangeServer [INFO]
> > > (RangeServer/AccessGroup.cc:533) Finished Compaction of METADATA
> > > [0: .. ](default)
> > > 2009-07-22 08:23:41,530 1378986304 Hypertable.RangeServer [INFO]
> > > (RangeServer/AccessGroup.cc:372) Starting InMemory Compaction of
> > > METADATA[0: .. ](location)
> > > 2009-07-22 08:23:41,549 1378986304 Hypertable.RangeServer [INFO]
> > > (RangeServer/AccessGroup.cc:533) Finished Compaction of METADATA
> > > [0: .. ](location)
> > > 2009-07-22 08:23:41,549 1378986304 Hypertable.RangeServer [INFO]
> > > (RangeServer/AccessGroup.cc:379) Starting Major Compaction of METADATA
> > > [0: .. ](logging)
> > > 2009-07-22 08:23:41,552 1378986304 Hypertable.RangeServer [FATAL]
> > > (Common/BloomFilter.h:47) failed expectation: m_num_bits != 0
>
> > > It seems that RangeServer cannot restore from log-replaying.
>
> > > What's the problem? How to fix it ?
>
> > > Thanks
>
> > > -- kuer
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Hypertable Development" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/hypertable-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
