OpenSearch is an open-source, distributed search and analytics engine built for horizontal scalability, reliability, and real-time search capabilities. It’s a popular alternative to Elasticsearch, especially after its open-source fork from Elasticsearch and Kibana in 2021. OpenSearch is widely used for use cases like log analysis, full-text search, and monitoring.
If you’re ready to set up OpenSearch for your project but aren’t sure where to start, this step-by-step guide will walk you through the process. From installation to basic configuration, you’ll have your OpenSearch cluster up and running in no time.
Prerequisites
Before we dive into the setup, ensure that you meet these prerequisites:
- A Linux-based system: This guide assumes you’re using a Unix-based OS, such as Ubuntu or CentOS. While OpenSearch can also run on macOS and Windows, Linux tends to provide better performance for production environments.
- Java 11+: OpenSearch requires Java 11 or later. Make sure that you have it installed and configured.
- Sufficient system resources: OpenSearch is a heavy application, especially if you’re dealing with large volumes of data. Ensure you have at least 4GB of RAM (8GB recommended for production) and adequate disk space.
- Root or Sudo access: To install software packages and configure system settings, you’ll need administrative privileges.
Step 1: Installing OpenSearch
There are multiple ways to install OpenSearch, including via package managers (like APT or YUM) or Docker. In this section, we’ll cover two methods: using the APT repository (for Ubuntu/Debian) and using Docker.
Option 1: Installing with APT (Ubuntu/Debian)
- Update your package index: Open a terminal and run the following command to update the list of available packages:
- sudo apt update
- Install prerequisites: You’ll need to install some required packages to allow APT to work with HTTPS repositories.
- sudo apt install apt-transport-https ca-certificates curl software-properties-common
- Add the OpenSearch repository: Import the OpenSearch GPG key:
- curl -fsSL https://artifacts.opensearch.org/GPG-KEY-opensearch | sudo tee /etc/apt/trusted.gpg.d/opensearch.asc
Then, add the OpenSearch APT repository:
echo “deb https://artifacts.opensearch.org/packages/2.x/apt stable main” | sudo tee /etc/apt/sources.list.d/opensearch.list
- Install OpenSearch: Now, install OpenSearch by running:
- sudo apt update
- sudo apt install opensearch
Option 2: Installing with Docker
If you prefer using Docker for containerized applications, OpenSearch provides an official Docker image that you can use.
- Install Docker: If you don’t have Docker installed yet, follow the official Docker installation guide for your platform.
- Pull the OpenSearch Docker image: Once Docker is set up, pull the latest OpenSearch image:
- docker pull opensearchproject/opensearch:latest
- Run OpenSearch in a container: To start OpenSearch in a Docker container, use the following command:
- docker run -d –name opensearch -p 9200:9200 -p 9600:9600 opensearchproject/opensearch:latest
This command will start OpenSearch and map its HTTP port (9200) and the monitoring port (9600) to your local machine.
Step 2: Configuring OpenSearch
Once OpenSearch is installed, you’ll need to configure it to suit your needs. OpenSearch configuration files are located in the /etc/opensearch/ directory for APT installs or inside the container for Docker installs.
- Configure Java Options: OpenSearch requires Java, and you can set Java heap size and other related options in the jvm.options file located in /etc/opensearch/jvm.options.
To set the heap size, you might change these lines:
-Xms2g
-Xmx2g
This sets the minimum (-Xms) and maximum (-Xmx) heap size to 2GB. Adjust these based on your system’s memory.
- Cluster Settings: OpenSearch can run as a single node or in a multi-node cluster. For single-node setups, you’ll likely need minimal configuration. However, if you’re setting up a cluster, you’ll need to configure network and discovery settings in the opensearch.yml file.
Common cluster configurations include:
- cluster.name: The name of your OpenSearch cluster.
- node.name: The name of the current node.
- network.host: The network interface OpenSearch binds to.
Example configuration:
cluster.name: my-opensearch-cluster
node.name: node-1
network.host: 0.0.0.0 # Listen on all network interfaces
For a multi-node cluster, you’ll also need to configure discovery.seed_hosts and cluster.initial_master_nodes to let OpenSearch nodes discover each other.
- Security Settings: OpenSearch comes with built-in security plugins, but for simplicity, you can disable them in the early stages of setup by modifying the opensearch.yml file:
- plugins.security.disabled: true
It’s highly recommended to enable security for production environments, but for now, we’ll keep it disabled.
Step 3: Starting OpenSearch
After configuring OpenSearch, you can start it using the following command, depending on your installation method.
For APT Installation:
sudo systemctl start opensearch
For Docker:
If you’re using Docker, OpenSearch should already be running after executing the docker run command.
To verify the OpenSearch service is running, use:
curl -X GET “localhost:9200/”
You should see a JSON response with information about your OpenSearch instance.
Step 4: Accessing the OpenSearch Dashboard (Optional)
OpenSearch also includes a Kibana-like web UI known as OpenSearch Dashboards. It can be useful for interacting with your data and visualizing search results.
To install OpenSearch Dashboards, run the following (if you installed OpenSearch via APT):
sudo apt install opensearch-dashboards
Start the service:
sudo systemctl start opensearch-dashboards
Once the dashboard is running, you can access it at:
http://localhost:5601
Step 5: Indexing Data into OpenSearch
Now that OpenSearch is running, you can start indexing data into your cluster. You can do this through the RESTful API.
For example, to create an index:
curl -X PUT “localhost:9200/my_index”
To add a document:
curl -X POST “localhost:9200/my_index/_doc” -H ‘Content-Type: application/json’ -d’
{
“title”: “Hello OpenSearch”,
“content”: “This is a test document”
}’
Step 6: Scaling and Monitoring OpenSearch
As your needs grow, you may want to scale OpenSearch by adding more nodes to your cluster or configuring additional settings like backup strategies or enhanced security.
Monitoring your OpenSearch cluster is essential for ensuring performance. OpenSearch includes a built-in monitoring system, which you can access from the OpenSearch Dashboards or via APIs.
For more advanced features, you may also explore integration with Logstash or Beats to collect logs or data and send it to OpenSearch for indexing.
Conclusion
Setting up OpenSearch doesn’t have to be complicated. By following this guide, you’ve learned how to install and configure OpenSearch, start indexing data, and optionally set up the OpenSearch Dashboards for visualization. As you scale your usage, remember to review cluster configurations, security settings, and monitoring tools to ensure you’re getting the most out of your OpenSearch instance.