On Wed, 6 May 2026 at 23:49, Roy Smith via Cloud <[email protected]> wrote:
> Is this just the opensearch that's serving production, or was the one in > Toolforge also updated? > > Cloudelastic [1] is the cluster providing access to production indices (a full indexing of all our wikis) to be used in the context of toolforge / WMCS. It is different from the general purpose Toolforge Elasticsearch cluster [2] (which is not affected by this issue). The OpenSearch cluster backing our on-wiki search is also not affected by this issue. [1] https://wikitech.wikimedia.org/wiki/Help:CirrusSearch_OpenSearch_replicas [2] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Elasticsearch > On May 5, 2026, at 11:22 AM, Peter Fischer via Cloud-announce via Cloud < > [email protected]> wrote: > > Hello everyone, > > As part of the ongoing upgrade of our Search infrastructure from > OpenSearch 1.3 → 2.19 → 3.5, we recently upgraded the first production > cluster (cloudelastic). > > Following the upgrade, the cluster has become unstable. In particular, > some indices have entered a read-only (red) state, which may impact search > functionality relying on this cluster. > > We are currently investigating the issue from multiple angles. Early > indications suggest this may be related to readahead settings in > combination with the underlying storage hardware, but this is not yet > confirmed. > > Tracking task: https://phabricator.wikimedia.org/T424852 > > Our current focus is on stabilizing the cluster and restoring full > functionality. We will share updates as we learn more. At this point, we > expect to have the cluster stabilized again by the end of the week. > > If you are observing issues that you suspect are related, please add > details to the Phabricator task or reach out to the Search team. > > Thanks for your patience, > Peter > -- > Peter Fischer (he/him) > Senior Software Engineer, Search Platform > Wikimedia Foundation > _______________________________________________ > Cloud-announce mailing list -- [email protected] > List information: > https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.org/ > _______________________________________________ > Cloud mailing list -- [email protected] > List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > > > _______________________________________________ > Cloud mailing list -- [email protected] > List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > -- *Guillaume Lederrey* (he/him) Engineering Manager Wikimedia Foundation <https://wikimediafoundation.org/>
_______________________________________________ Cloud mailing list -- [email protected] List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
