Hey Simon, If you grep that logfile for "r_object", you'll find a tuple where that is the first atom, followed by two binaries. The first is the bucket, the second is the key.
You should then be able to request that bucket-key combination with a content-type of "text/plain" to see a list of its siblings, if the fsm doesn't crash again (which may indeed happen, because despite only asking for the siblings, the fsm is asked for the whole object). Sam -- Sam Elliott Engineer [email protected] -- On Thursday, 29 August 2013 at 3:18AM, Simon Effenberg wrote: > And here the log files when it crashes (OOM).. > > On Thu, 29 Aug 2013 09:01:35 +0200 > Simon Effenberg <[email protected] > (mailto:[email protected])> wrote: > > > At least one of my 18 nodes is crashing soonish after I started it. > > > > I attached the app.config and vm.args files. > > > > If it is a big object (or multiple of them) then I have no clue how to > > find out what object it is. Our secondary indexes are somehow b0rked > > because I can't find a new imported object even if I search with the > > exakt createdat_int index which is returned by a HEAD/GET request to > > the object itself. > > > > Any help is much appreciated.. > > Simon > > > > On Thu, 29 Aug 2013 07:26:31 +0200 > > Simon Effenberg <[email protected] > > (mailto:[email protected])> wrote: > > > > > allow_multi is on and I looked into the graphs.. we have had some > > > siblings while having big objects but its "only" a max of 35 siblings > > > found within one GET request. But I can't say if this is "one" GET > > > request having 35 siblings or 35 GET requests each having 1 > > > sibling. > > > > > > Also the question: in the erl_crash.dmp I have almost all data at the > > > end (> 500MB) which is a huge multiline but with reaaaaaallyyy long > > > lines of hexadecimal numbers like > > > > > > 36F6465223A223730303531393935227D2C7B226163636F756E74223A22313 > > > > > > can I get therein (crash dump file) any hint about the object which > > > caused the crash? > > > > > > Cheers > > > Simon > > > > > > > > > On Wed, 28 Aug 2013 19:05:23 -0400 > > > Sam Elliott <[email protected] (mailto:[email protected])> wrote: > > > > > > > Simon, > > > > > > > > This sounds like it might be some kind of sibling explosion. Do you > > > > have allow_mult=true set on any buckets? If so, are you resolving every > > > > single time you read the object? What's your regular object size > > > > (before you started seeing big objects)? > > > > > > > > There's some more info in our docs [1] - search for "siblings" for the > > > > stat names associated with them that might give you some information. > > > > > > > > Sam > > > > > > > > [1] http://docs.basho.com/riak/latest/ops/running/stats-and-monitoring/ > > > > > > > > > > > > -- > > > > Sam Elliott > > > > Engineer > > > > [email protected] (mailto:[email protected]) > > > > -- > > > > > > > > > > > > On Wednesday, 28 August 2013 at 6:50PM, Simon Effenberg wrote: > > > > > > > > > Hi, > > > > > > > > > > we have suddenly (regarding to riak stats) really big objects (100th > > > > > percentile of object size) of 400MB to 900MB. > > > > > > > > > > We have no clue from where or how this could be.. is it somehow > > > > > possible to identify them easily? > > > > > > > > > > Cheers > > > > > Simon > > > > > > > > > > _______________________________________________ > > > > > riak-users mailing list > > > > > [email protected] (mailto:[email protected]) > > > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > > > > > > > > > > > > > > -- > > > Simon Effenberg | Site Ops Engineer | mobile.international GmbH > > > Fon: + 49-(0)30-8109 - 7173 > > > Fax: + 49-(0)30-8109 - 7131 > > > > > > Mail: [email protected] (mailto:[email protected]) > > > Web: www.mobile.de (http://www.mobile.de) > > > > > > Marktplatz 1 | 14532 Europarc Dreilinden | Germany > > > > > > > > > Geschäftsführer: Malte Krüger > > > HRB Nr.: 18517 P, Amtsgericht Potsdam > > > Sitz der Gesellschaft: Kleinmachnow > > > > > > _______________________________________________ > > > riak-users mailing list > > > [email protected] (mailto:[email protected]) > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > > > > > -- > > Simon Effenberg | Site Ops Engineer | mobile.international GmbH > > Fon: + 49-(0)30-8109 - 7173 > > Fax: + 49-(0)30-8109 - 7131 > > > > Mail: [email protected] (mailto:[email protected]) > > Web: www.mobile.de (http://www.mobile.de) > > > > Marktplatz 1 | 14532 Europarc Dreilinden | Germany > > > > > > Geschäftsführer: Malte Krüger > > HRB Nr.: 18517 P, Amtsgericht Potsdam > > Sitz der Gesellschaft: Kleinmachnow > > > > > -- > Simon Effenberg | Site Ops Engineer | mobile.international GmbH > Fon: + 49-(0)30-8109 - 7173 > Fax: + 49-(0)30-8109 - 7131 > > Mail: [email protected] (mailto:[email protected]) > Web: www.mobile.de (http://www.mobile.de) > > Marktplatz 1 | 14532 Europarc Dreilinden | Germany > > > Geschäftsführer: Malte Krüger > HRB Nr.: 18517 P, Amtsgericht Potsdam > Sitz der Gesellschaft: Kleinmachnow > > > Attachments: > - console.log > > - crash.log > > - erlang.log.1 > > - error.log > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
