Hello Abhay, You only need to keep or merge old segments if you 'quickly' need to reindex the data, and are unable to start with a fresh crawl. If you frequently recrawl all urls, e.g. a month, then segments older than a month can safely be removed.
You can also do daily an monthly merges, like we do. This makes it possible to revisit old data for research, in case websites change layout, or are no longer customer and not being crawled anymore. Regards, Markus Op di 6 apr. 2021 om 21:54 schreef Abhay Ratnaparkhi < abhay.ratnapar...@gmail.com>: > Hello, > > I have a large number of segments occupying disk space. It is a good > strategy to delete old segments or it's better to merge them. > > Thank you > Abhay >
