hi Michael, terribly sorry for the very late response. i actually reverted back to using whisper as i was unable to determine what was causing the issues: you're pointer on the inode exhaustion is a good one and i'll try it out….
so i have a new machine i can test on (same as the last one) and i'm trying to run megacarbon. i'm a little confused over the unified relay/write daemons. i expected that i could set a daemon with the following in daemon.conf: PIPELINE = relay, write but i get the following error when trying to start the instance: carbon.conf.ConfigError: <ConfigError(daemon.conf: Exactly one 'write' or 'relay' must exist in PIPELINE)> however, if i try to configure just a relay (like with 0.9.10), with PIPELINE = relay however, i get the following a different error: carbon.conf.ConfigError: <ConfigError(Missing expected configuration "CACHE_SIZE_LOW_WATERMARK")> am i doing something wrong? cheers, On Wednesday, 29 August 2012 at 10:23, Michael Leinartas wrote: > Hey, sorry I missed this message over a month ago! > > If the file doesn't exist then I'd guess it's trying to write out a > new file - unlike Whisper, Ceres will continue to create new files for > a metric as time goes on (hinted at by the timestamp in the name). > > Have you been able to work through this issue? If not, my guess is > that you're running out of inodes - While it's somewhat more space > efficient than Whisper (not using space for points that aren't stored > yet) it does end up with a lot more files. You should be able to > confirm this by trying to touch a file on the same partition as a > non-root user. > > Let us know how it works out! > > thanks > michael > > On Wed, Jul 11, 2012 at 1:03 AM, Yee-Ting Li <[email protected] > (mailto:[email protected])> wrote: > > hi - great project :) > > > > i'm using mleinart's graphite-megacarbon latest branch with ceres in a vain > > attempt to reduce the disk space that my data occupies; basically i'm > > pushing a bunch of network switch counters every 15 seconds. > > > > so, i feed data into ceres and everything seems great. however, after a few > > hours, the logs give me a bunch of 'database write operation failed' > > errors. this is repeatable…. adding an exception string to the try/except > > shows: > > > > "database write operation failed: (<type 'exceptions.IOError'>) [Errno 28] > > No space left on device: > > '/opt/graphite/storage/ceres/ptolemy/<REDACTED>/<REDACTED>/port_stats/rfc2863/unicast_pkts_out/[email protected] > > (mailto:[email protected])'" > > > > (the redacted bits is just my fully qualified name of my switch/router and > > port) > > > > my disk is definitely not full. and in fact the file/slice does not exist. > > > > i haven't had too much time to look into the ceres code yet, but i did a > > quick hack to try to re-create the database (basically running the database > > creation bit of write_cached_datapoints() again) if i get this error, but > > still nothing. and no exceptions from database.create() > > > > after i stop the daemon, and start it again - all writes will report this > > same error and i get no new data into the system. a 'rm -rf' on the ceres > > store fixes things. > > > > please forgive my attachment, but as they say, a picture shows a thousand > > words. rates are relative low for now - will likely want to goto about 50 > > times that in production - as long as i can actually write data > > continuously ;) > > > > any ideas? > > > > Yee. > > > > > > > > _______________________________________________ > > Mailing list: https://launchpad.net/~graphite-dev > > Post to : [email protected] > > (mailto:[email protected]) > > Unsubscribe : https://launchpad.net/~graphite-dev > > More help : https://help.launchpad.net/ListHelp > > > > _______________________________________________ > Mailing list: https://launchpad.net/~graphite-dev > Post to : [email protected] > (mailto:[email protected]) > Unsubscribe : https://launchpad.net/~graphite-dev > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~graphite-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~graphite-dev More help : https://help.launchpad.net/ListHelp
