ELK Setup and Nodejs Integration with basic query types

I was experimenting with elastic search for search implementation in one of our project. I had to do setup and do a PoC. We were exploring some of the options in the market like redis search and mongodb search which is built on lucene engine.
However, considering features and market cap, we moved ahead with elastic search.

Thanks to docker for the amazing simple local setup. Refer to the repo below and yay! ELK is up on local.

GitHub - preethamslab/docker-elk: The Elastic stack (ELK) powered by Docker and Compose.

Elastic search runs on port 9600 and kibana runs on 5601.

launch up Kibana with: https://localhost:5601

login with base credentials userID(elastic) and password (changeme).

You should be on the below screen

image 1.1

Cool! You’ve just launched a Kibana workspace.

Next step is to create a data set and an index. Since we’re just getting started we’ll manually upload file and run few queries.

Head to the below repo, download news_category_dataset_v2.json file.

GitHub - preethamslab/elasticsearch-Crud-nodejs: This repository is for elastic search

and in image 1.1, click upload a file, drag and drop a file you just downloaded.

1.2

Click on import.

Let’s give a index name as “my_news_headlines”. We’ll start with the simple index. Then hit import.

1.3

It will take a while to import. Once done, Let’s head to dev tools.

<note: if you’re facing issues while importing, don’t worry, head to dev tools, experiments and queries will remain same.>

Under management section of left bottom, click on Dev tools.

In the console, create an index, with run icon on the right of the console.

Index is like a reference to your store or particular collection. You can store data to index or query like the examples below.

Now the document is inserted to index. Now you can query the document like below.

this query will give you all the documents under the index my_index_001.

Now let’s count the documents under index my_index_001

Now let’s get the specific document with match query:

Now let’s get all the documents which begins with some word, here it’s just in headline tag. Prefix query solves this problem for us.

Now let’s get the documents which begins with anything but end with “lliant”. Wildcard query solves this problem for us.

Now, let’s use some regexp query. You can enter regular expressions and search. Below is one such example.

Now, if I’ve to paginate, just type.

From refers to start of the result-set and size is number of documents.

So, now you’ve been observing, I’m always querying on single field, so how do we query on multiple fields. I was able to solve it with below query. Please refer to the documentation to understand, since lot of stuff going on behind.

All the queries are in the below gist.

https://gist.githubusercontent.com/preethamslab/50d6e75dac1a39f1d4ec99c03efd7c9e/raw/3dc472b9361c49964c5ff0a841330356abab8f2a/elkqueries.json

one last is to delete the index.

DELETE /my_index_001

Ok, that’s about the basics of ELK and queries.

Now, let’s head to nodejs implementation.

GitHub - preethamslab/elasticsearch-Crud-nodejs: This repository is for elastic search

Head to the above repository, follow the instructions on readme.

Connection file will make the connection to your ELK cluster.

ESIndexManager does all the CRUD work for you.

Blogger will call the methods for you, uncomment the method to run it.

I’ll keep you posted on my ELK journey. Until then! BBYE!