Hi Dipesh, I have a few questions. Are you still on S3? Which version of elasticsearch are you using? How many snapshots do you currently keep in S3? How fast is your index growing over time?
Igor On Wednesday, May 7, 2014 6:58:05 AM UTC-4, Dipesh Patel wrote: > > Hi > > We've noticed recently that our snapshot durations are increasing over > time. Our rate of flow of data going into elasticsearch has remained fairly > constant. Though we do create new indices everyday ( though this is a fixed > number that doesn't vary from day to day ). We are currently snapshoting, > or trying to snapshot every hour. However with the snapshots taking a > progressively longer time this is proving difficult. > > Here's some stats showing our time to finish: > > Name > Duration ( milli ) > snapshot_2014-05-01_01:30:00 4497010 snapshot_2014-05-01_03:30:00 > 4513037 snapshot_2014-05-01_05:30:00 4770288 > snapshot_2014-05-01_07:30:00 5413361 snapshot_2014-05-01_11:30:00 6978384 > snapshot_2014-05-01_13:30:00 6907554 snapshot_2014-05-01_15:30:00 7388500 > This is just the tail end originally the snapshots were only taking 7-8 > mins to run, they've just been getting progressively longer. > > Any help to debug etc appreciated. > > Dip > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/96ca5fa6-f366-4f9e-b3c1-a57b511a341e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
