Browsed by
Category: ElasticSeach

PURE BUILDERS ELASTIC: Beat and ingest pipeline

PURE BUILDERS ELASTIC: Beat and ingest pipeline

In this fourth post in this series, I’ll describe how to use an Ingest Node Pipeline to process our incoming documents. We will use the ingest pipeline to split the JSON log data we’ve created in our Python application, so that we can use the individual fields in Elastic for further processing. As a quick reminder, the purpose of this blog series is to write about the steps I took in my journey to build a teeny tiny data pipeline…

Read More Read More

PURE BUILDERS ELASTIC: Elastic Cloud

PURE BUILDERS ELASTIC: Elastic Cloud

In this third post in this series, I’ll describe how to deploy Elastic Search on Kubernetes using Elastic Cloud. We need this Elastic deployment to be able to ingest the logfiles that are created by the Python application that I’ve created in the first part of this series. As a quick reminder, the purpose of this blog series is to write about the steps I took in my journey to build a teeny tiny data pipeline using Python, Kubernetes and…

Read More Read More

PURE BUILDERS ELASTIC: Containerise my python app

PURE BUILDERS ELASTIC: Containerise my python app

In this second post in this series, I’ll describe how to containerise the Python application that I’ve created in my previous blog in this series. In this blog series I’ll write about the steps I took in my journey to build a teeny tiny data pipeline using Python, Kubernetes and ElasticSearch. The idea of the data pipeline I’m creating, is to use Python to export data from the FlashArray and FlashBlade to a log file and then ingest this data…

Read More Read More

Pure Builders Elastic: Getting started with Python

Pure Builders Elastic: Getting started with Python

This is the first in a series of blog posts, where I’ll write about the steps I took in my journey to build a teeny tiny data pipeline using Python, Kubernetes and ElasticSearch. The idea of the data pipeline I’m creating, is to use Python to export data from the FlashArray and FlashBlade to a log file and then ingest this data into ElasticSearch. In my example, I’ll be collecting volume statistics, such as size, used space and the data…

Read More Read More