Hello Lars, thank you for the hint. I will try the hashed timestamp prefix.
Cheers On Mon, May 21, 2012 at 6:14 PM, Lars George <[email protected]> wrote: > Hi Mete, > > OpenTSDB uses the "natural" availability of the metrics ID to bucket the > metrics by type. After that it relies on scanner batching, and block loads. > > For your uses case you could bin by time frames, say for example hash the > start of each hour into an MD5 and concatenate it with the actual epoch > like so > > <MD5(hour(epoch))>-<epoch> > > That way you have the ability to range scan the bin but also distribute > the bins across the server evenly. > > HTH, > Lars > > On May 21, 2012, at 7:56 AM, mete wrote: > > > Hello folks, > > > > i am trying to come up with a nice key design for storing logs in the > > company. I am planning to index them and store row key in the index for > > random reads. > > > > I need to balance the writes equally between the R.S. and i could not > > understand how opentsdb does that with prefixing the metric id. (i > related > > metric id with the log type) In my log storage case a log line just has a > > type and a date and the rest of it is not really very useful information. > > > > So i think that i can create a table for every distinct log type and i > need > > a random salt to route to a different R.S. similar to this: > > <salt>-<date> > > > > But with this approach i believe i will lose the ability to do effective > > partial scans to a specific date. (if for some reason i need that) What > do > > you think? And for the salt approach do you use randomly generated salts > or > > hashes that actually mean something? (like the hash of the date) > > > > I am using random uuids at the moment but i am trying to find a better > > approach, any feedback is welcome > > > > cheers > > Mete > >
