Definitely scan and scroll API is for that. Some clients provide that feature out of the box.
That said, reading from couchbase is something super easy. So I guess you could may be replicate your data from couchbase twice using 2 XDCR configs one for the weekly index and the other one for the year index. Is that something doable? -- David Pilato | Technical Advocate | Elasticsearch.com @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 14 janv. 2015 à 14:51, Behrad <[email protected]> a écrit : > > I'll recheck elasticsearch-transport-couchbase again David, another two > related questions comes to my mind to ask, > > 1) Is there any best practice to replicate/sync/duplicate an existing index > to a new index by ES? > 2) Considering I have an existing index with data persist enabled on ES, Can > I use that to create a new index with a customer defined mapping on my data > on ES without the need of my primary database? (does sroll api help with > this?) > > 2015-01-14 17:00 GMT+03:30 David Pilato <[email protected] > <mailto:[email protected]>>: > May be you could use different mapping for the year index and the week index? > One with ttl disabled and the other one with ttl enabled? > > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-ttl-field.html > > <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-ttl-field.html> > > -- > David Pilato | Technical Advocate | Elasticsearch.com > <http://elasticsearch.com/> > @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr > <https://twitter.com/elasticsearchfr> | @scrutmydocs > <https://twitter.com/scrutmydocs> > > > >> Le 14 janv. 2015 à 14:21, Behrad <[email protected] >> <mailto:[email protected]>> a écrit : >> >> I am actually using Couchbase ES plugin David, this is my database >> requirement that data has a weekly ttl, so when data is removed in >> Couchbase, ES plugin also removes it (using Couchbase's XDCR) within ES. >> >> 2015-01-14 16:43 GMT+03:30 David Pilato <[email protected] >> <mailto:[email protected]>>: >> Don't use ttl for that kind of usage but prefer time based indices. >> Much more efficient. >> >> So the year index won't have any removal and you will end up to just remove >> weekly indices which is super fast and doesn't generate a lot of IO (but ttl >> does) >> >> HTH >> >> David >> >> Le 14 janv. 2015 à 14:01, Behrad Zari <[email protected] >> <mailto:[email protected]>> a écrit : >> >>> My data contains a weekly ttl so my ES index will loose older data, however >>> I want to have another accumulative large index containing a year of my >>> data... How can I instruct ES/sync my second index without affecting ttls >>> in the first index? ( I don't want deleted docs from first index to be >>> deleted in the second) >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/08f4979d-ccd4-431a-96b9-3e2911c9ae80%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elasticsearch/08f4979d-ccd4-431a-96b9-3e2911c9ae80%40googlegroups.com?utm_medium=email&utm_source=footer>. >>> For more options, visit https://groups.google.com/d/optout >>> <https://groups.google.com/d/optout>. >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/9Spk2d84FLk/unsubscribe >> <https://groups.google.com/d/topic/elasticsearch/9Spk2d84FLk/unsubscribe>. >> To unsubscribe from this group and all its topics, send an email to >> [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/48E15A95-E2E6-4347-9040-A7789466A378%40pilato.fr >> >> <https://groups.google.com/d/msgid/elasticsearch/48E15A95-E2E6-4347-9040-A7789466A378%40pilato.fr?utm_medium=email&utm_source=footer>. >> >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. >> >> >> >> -- >> --Behrad >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAJFrgeH_NJHp7rJWNXppQzASw6E8Xsh9trxshFaXows%2Bpr3BjQ%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elasticsearch/CAJFrgeH_NJHp7rJWNXppQzASw6E8Xsh9trxshFaXows%2Bpr3BjQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to a topic in the Google > Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/9Spk2d84FLk/unsubscribe > <https://groups.google.com/d/topic/elasticsearch/9Spk2d84FLk/unsubscribe>. > To unsubscribe from this group and all its topics, send an email to > [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/EBBC7F7B-F82F-401B-898B-84C76D6F5F80%40pilato.fr > > <https://groups.google.com/d/msgid/elasticsearch/EBBC7F7B-F82F-401B-898B-84C76D6F5F80%40pilato.fr?utm_medium=email&utm_source=footer>. > > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. > > > > -- > --Behrad > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAJFrgeEAoSg_ppRreOpOchx9By3ezD%2BYT06VR84FJ4jMs6PmUg%40mail.gmail.com > > <https://groups.google.com/d/msgid/elasticsearch/CAJFrgeEAoSg_ppRreOpOchx9By3ezD%2BYT06VR84FJ4jMs6PmUg%40mail.gmail.com?utm_medium=email&utm_source=footer>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/884A1E22-F487-4487-AC86-82FA757E171D%40pilato.fr. For more options, visit https://groups.google.com/d/optout.
