How to Improve OpenSearch Search Performance

14 tips on how to reduce OpenSearch search latency and optimize search performance:

1. Size parameter

   Assigning a huge value to size parameter causes OpenSearch to compute vast amounts of hits, which causes severe performance issues. Instead of setting a huge size, you should batch requests in small sizes.

2. Shards and replicas

   Optimize necessary index settings that play a crucial role in OpenSearch performance, like the number of shards and replicas. In many cases having more replicas helps improve search performance. Please refer to Opster’s guide on shards and replicas to learn more.

3. Deleted documents

   Having a large number of deleted documents in the OpenSearch index also causes search performance issues. Force merge API can be used to remove a large number of deleted documents and optimize the shards.

4. Search filters

   Effective use of filters in OpenSearch queries can improve search performance dramatically as the filter clauses are 1) cached, and 2) able to reduce the target documents to be searched in the query clause.

5. Wildcard queries

   Avoid wildcard, especially leading wildcard queries, which causes the entire OpenSearch index to be scanned. 

6. Regex and parent-child

   Note that Regex queries and parent-child can cause search latency.

7. Implementing features

   There are multiple ways to implement a specific feature in OpenSearch. For example, Autocomplete can be implemented in various styles. This documentation gives a 360-degree view of both functional and non-functional features.

8. Multitude of small shards

   Having many small shards could cause a lot of network calls and threads, which severely impact search performance.

9. Heavy aggregations

   Avoid heavy aggregations that involve unique IDs.

10. Timeout and terminate

    Timeout param and terminate after param can be useful when executing heavy searches, or when result data is vast.

11. Search templates

    Use search templates to achieve better abstraction, meaning without exposing your query syntax to your users. Search templates also help you transfer less data over the network, which is particularly useful when you have large OpenSearch queries.

12. Multi search API

    Use msearch whenever possible. In most of the applications it’s required to query multiple OpenSearch indices for a single transaction, and sometimes users do so in a serial order even when it’s not required. In both cases, when you need to query multiple indices for the same transaction and when the result of these queries are independent, you should always use msearch to execute the queries in parallel in OpenSearch.

13. Term queries

    Use term query when you need an exact match and on keywords fields. By default, OpenSearch generates both text and keyword fields for every field that consists of a string value if explicit mapping is not supplied. Users tend to use the match query even on keyword data types like product-ids, which is costly as match query goes through an analysis operation. Always use term query on keyword data types and wherever you need exact searches for better performance.

14. Source filtering

    _source filtering is a great way to improve the performance of OpenSearch queries when retrieving a large number of documents or documents of large sizes. By default, OpenSearch returns the complete source of matching documents. If you don’t need _source at all or need only values of specific fields, you can achieve this with _source filtering.