How to limit elastic search memory consumption

Introduction

Using the Elasticsearch Health Check-Up, you can circumvent problems with Elasticsearch’s memory use. It will assist in checking for a large number of configuration issues that are associated with the utilization of memory in your cluster and will give general improvements and ideas to your setup.

What is Garbage Collector?

The JVM handles memory management in Java automatically in the background, which also has a Garbage Collector component. With its autonomous memory management, Garbage Collector simplifies application development and reduces writing time. Again, the Garbage Collector can resolve the memory regions being lost due to faults in the programming.
The process of Garbage Collection is a kind of memory management that is performed automatically. It searches the Heap memory, finds any items that have been utilized, and then deletes those that have not been referenced. Memory is cleaned of the space held by unused or non-referenced items, which increases the amount of memory that is accessible for usage.

Configure the Elasticsearch Service Heap Size

Memory use is a resource that is heavily utilized by the Elasticsearch service. The Elasticsearch service is configured to utilize a minimum and maximum heap size of 2 gigabytes by default. This setting may be changed if necessary. The Elasticsearch Warden configuration file and the jvm. Options files are two places where you may make modifications that will allow you to alter these default settings. After making changes to the settings, you will need to restart Elasticsearch.

os. swap

The host computer is mentioned once more inside the swap section. The host system supports switching in this particular scenario. When running a container inside a host with swapping still active, this behaviour is perfectly typical and expected.

os.cgroup

Last but not least, the cgroup section that discusses the container is visible to us now. The container uses 4 Gigabytes (GB) of RAM in this particular instance.

os. mem

The first part, referred to as “mem,” discusses the host server on which the computer is now operating. In this scenario, we are using Docker; hence, the 16 GB mentioned above refers to the RAM of the host system on which the container operates. It is essential to keep in mind that it is very usual for a computer to use almost all of the memory that it has, and this does not signal that there is a problem.

Some Best Practice Notes

• It is strongly suggested that the size of the heap not exceed more than 50 percent of the entire memory. If you have 64 gigabytes of RAM, you shouldn’t use the default setting of 48 gigabytes for your heap size.
• It is not suggested that the Heap Size be more than 32 GB.
• By default, Xms1g and Xmx1g equal 1 GB of storage space.
• To modify the JVM heap size, go to the /etc/elasticsearch/jvm. Options directory, which is the default path address.

Conclusion

Memory use throughout the Elasticsearch operation is relatively high. The Java Virtual Machine, or JVM, is used by Elasticsearch, and it is recommended that about half of the RAM available on each node be dedicated to the JVM. The JVM machine uses memory because the Lucene process requires direction on where to find index data stored on the disc. The remaining fifty percent is necessary for the file system’s cache, which is responsible for storing in memory information that is often accessed