What is Elasticsearch And Why Do You Need It?

Elasticsearch logo
Reading time: 4 minutes

What is Elasticsearch?

In this article, we will fly you through a technology called Elasticsearch. I’m a big fan of this technology and I’ve been using it since the early days, even before it turned into a company. To start off let’s see what Elastic, the company behind Elasticsearch, has to say about this product first:

Elasticsearch is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on Apache Lucene and was first released in 2010 by Elasticsearch N.V. (now known as Elastic). Known for its simple REST APIs, distributed nature, speed, and scalability, Elasticsearch is the central component of the Elastic Stack, a set of open source tools for data ingestion, enrichment, storage, analysis, and visualization.

Elastic.co/what-is/elasticsearch

Wow. Some pretty concise and long sentences. Let’s break it up!

A Search and analytics engine

Elasticsearch allows you to store all kinds of data. You may think search is all about text. And yes, it is great at indexing and querying text. But that’s not all. You can also store numeric data and geospatial data like coordinates and geo shapes. Elasticsearch allows you to query your data, to summarise it, calculated averages and more!

Open-source

It’s free and open source. Elastic, the company, has turned into a very profitable and commercial company over the last years. Luckily you won’t have to spend a dime on licenses to use it in production. Obviously, Elastic adds a lot of value to the product in terms of support and extra features when you pay them. For example, you may benefit from their fully managed cloud-based hosting. They also offer X-Pack. Some features in X-Pack are free, but others require a license, like advanced authorization and authentication options, Artificial Intelligence, and graph analysis.

An entire ecosystem

Elasticsearch is the central component of a growing stack of products made by Elastic, called the “Elastic Stack”. These tools help you visualize (Kibana), ingest (Beats, Logstash) and manage data stored in Elasticsearch. Besides the official tools, there are plenty of free and commercial tools available too.

Elastic?

So what’s with the name? What’s so elastic about this search technology? I think it boils down to two key points. First of all, it will scale effortlessly from one node to more nodes than you will ever need. It also has elasticity in how easily you can get up and running with it. It’s easy to get started which you’ll find out in my other articles, where we get our hands dirty. Beyond this easy start, it offers plenty of ways to help you use it successfully in a production environment too. It can be very flexible and forgiving if you want it to be, but it can also be steady as a rock once you go to production.

Distributed

I believe one of the biggest strengths of Elasticsearch is the ease with which it scales up once you need it. Please note that, like with traditional databases, most users will get by with just a single node. But once your business starts growing, Elasticsearch will scale with you effortlessly. Physically it’s just a matter of adding machines and listing them in the configuration file. Your indexes will be automatically distributed to the other nodes once they are added!

There are lots of pro tips when it comes to scaling up Elasticsearch, but I will save these for later articles.

What is it used for?

Elasticsearch is used by many as the main data store. The advantage here is that you can both store and query your documents very well. It’s used this way for application search, enterprise search, and website search.

Another widely seen use case is log storage and indexing. With the ELK stack, it’s a breeze to get all your log data into Elasticsearch for analysis. A similar use case is the storage of security log data in order to create a threat intelligence platform.

Furthermore, the ELK stack is often used to monitor infrastructure and application performance and usage. It is also used for the storage and analysis of geospatial data. And last but not least, it can be a business intelligence platform.

Alternatives to Elasticsearch

Apache Solr

Without going into it too deep, I believe you can’t value a product to it’s full extend without knowing its competitors. And there’s only one real competitor to Elasticsearch that offers a similar set of features, which is Apache Solr. When I started using Elasticsearch in 2010, the way Solr worked was way more “clunky” than Elasticsearch. For example, Solr couldn’t even get close to the ease with which Elasticsearch scales up once you need to. It also required a lot of XML. I’m sure Solr has improved since then, but I have not found a valid reason to switch to Solr in the past 9 years.

Popularity

To give you an idea of the popularity of these two products, here’s a comparison on Google Trends.

These are Google searches. It clearly shows a huge increase in popularity for Elasticsearch that started after its launch in 2010. It surpassed Solr mid-2014. Solr has slowly been declining in popularity where Elasticsearch only seems to grow.

To wrap up

So there you have it. Now you have a basic knowledge of what Elasticsearch is, what it is used for and how it is positioned in the market. In the next articles, I will show you just how easy it is to get started with Elasticsearch. Prepare to get your hands dirty and continue reading here:

Liked this article? Please share it with others:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.