How Elastic Stack 7.2.0 will influence your DevOps, Monitoring, Analytics, and Security

In our April 2019 blog, What’s new from Elastic for DevOps and Big Data, Timmanna discussed some of the highlights of the release, such as Elasticsearch JS client (RC1); Infrastructure 7.0, and Elasticsearch for Apache Hadoop 7.0.0, Logstash 7.0.0, & Kibana 7.0.0. This time, I will be talking about the Elastic Stack 7.2.0 release, which has some amazing upgrades.

Observability is at the core of Elastic Stack 7.2.0, and you will be able to obtain insights into systems, services, and application health details. Observability has been achieved using .NET in APM, Kubernetes monitoring, and Metrics Explorer. This release is focused on automating all operational & cluster management tasks while simultaneously simplifying the Elastic on K8s experience of users.

It comes with extended support for .NET applications and brings the benefits of multi-page applications to single-page applications. It is now developed to collect agent-specific metrics through the detection of the programming language used. Additionally, it comes with Metrics Explorer, which is now embedded in the Elastic Infrastructure app.

Elastic has launched the Security Information and Event Management (SIEM) solution with this release. Through SIEM, Elastic aims toward the betterment of network and host-based data collection. It has a built-in User Interface (UI) that simplifies the visualization of data and identification of issues that may emerge. This UI is capable of handling host & network security analysis along with timeline event analysis.

And there’s more exciting news… Elastic App Search On-prem, which used to be a hosted service, is now Generally Available. Yes, you heard it right! So those of you who wish to run it in your data centers or laptops and desktops, Elastic has granted your wish. All you have to do is download and start using it.

Elastic has rolled out five important releases, Elasticsearch 7.2.0, Kibana 7.2.0, Elastic Uptime Monitoring 7.2.0, Beats 7.2.0, and Logstash 7.2.0. Now let’s have a look at these releases one by one.

Elasticsearch 7.2.0

Let us start with Elastic’s favorite release, Elasticsearch. It is very uncommon for Elastic to not release any updates for Elasticsearch and the 7.2.0 release is no exception. This release has key updates to take care of your search requirements.

When it comes to ranking considerations, you can achieve them using the distance feature query. This query is optimized to work with geo & time fields. The search_as_you_go approach is an important aspect of suggestions provided while a user is typing queries, and 7.2.0 is well-equipped to leverage typeahead search and achieve better performance. It also comes with improved resiliency and an array of improvements for Elasticsearch SQL, data frames, HTML Strip Processor, and OpenID Connect Realm.

Talking about resiliency, replication of closed indices is now possible for frozen indices. Empowerment of Snapshot Repositories with incremental snapshot mechanism further improves resiliency. The Snapshot Repositories app in Kibana supports both on-prem and cloud environments. Using this app, you can browse repositories and snapshots that you have created. Elastic is looking forward to adding more features to it in its next release.

Improvements in geographic queries through SQL statements, medium absolute deviation, and if-else statements are some of the new additions to Elasticsearch SQL. Dataframe plugins are now added so that you can transform indexed data with ease. Similarly, improvements in the HTML strip processor make the source field more readable, and improvements in OpenID Connect Realm come with new customizations to improve the stack.

Logstash 7.2.0

Now, let us talk about Logstash 7.2.0, which has a bagful of pleasant surprises for Java Developers. Logstash 7.2.0 has extended “support for Java plugins” which is now Generally Available. This will enable developers to write plugins using Java without worrying about Ruby dependencies.

Java plugins run effortlessly for Java execution engines as well as alongside Ruby plugins. The Logstash JMS input plugin supports data consumption from JMS queues into Elastic Stack. The bring-your-own-driver model opens doors to a broad spectrum of technologies that comply with JMS standards.

There’s more good news! Logstash is now integrated with the Self-Managed Elastic App Search. Thus, Logstash can be used to easily migrate your data to Elastic App Search. Additionally, Logstash 7.2.0 now offers more integrations with GCP and can read/write from the GCS blob store, along with options for real-time analytics.

Beats7.2.0

The Beats 7.2.0 release is packed with new features to meet your monitoring requirements. It has the Palo Alto Network Module to monitor PAN-OS firewall logs. Similarly, its Cisco ASA Module monitors Cisco ASA firewall logs, whereas NewFlow monitors NetFlow IPFIX flow records. Coming toward cloud-native, 7.2.0 has CoreDNS within the Filebeat and Metricbeat to monitor CoreDNS logs and metrics.

For Windows systems, the Sysmon module monitors event log records from the system, and the system & security model monitors Windows Security Event Logs. It has gone further to support the Windows XML Event Log format.

Elastic Uptime Monitoring 7.2.0

The Elastic Uptime Monitoring 7.2.0 release allows you to view and observe location-specific information conveniently. This information is passed on from Heartbeat to Elasticsearch, which is then displayed on the Uptime app. It further allows integration with other observability solutions.

Kibana 7.2.0

Kibana 7.2.0 is overflowing with new features and updates; so many that there can be a dedicated blog on this release. However, I will cover a few important ones here.

It is equipped with new modules and inputs, has several virtualization options, and is geared with feature control. It can also prohibit users from accessing certain features depending on their designation and roles in an organization. It allows importing and exporting dashboards, indexes, maps, etc., for authorized users. The ability to enhance custom elements and share these custom elements with teammates is another important aspect of this release.

Snapshot repositories store snapshots, which can be rolled back and restored whenever needed. Filtered search results related to influencers for a selected job can also be obtained in the Anomaly Explorer. Moreover, there is a provision to handle and monitor machine learning jobs using the Metricbeat System Module.

Visualizing rolled-up data is now convenient using Time Series. It allows you to create index patterns that visualize data and create multiple Y-axes in the same chart. It also comes with canvas enhancements for custom elements, autoplay, and filter groups. Moreover, maps are now integrated into dashboards consisting of newly added visualization options such as dark mode, class styling, and collapsible map legends.

Data transformation is now simplified with data frames wherein jobs have a dedicated list page displaying details and status controls to handle these jobs. You can use Filebeat to collect the logs which will be displayed along with the monitoring data. Observability is achieved with Metrics Explorer and APM. Metrics Explorer makes it easy to interact with infrastructure metrics, and APM UI displays metrics and indicators collected by API agents.

It is interesting how Elastic is continuing its legacy of making a lot of things “Generally Available” with every new release that is being rolled out. I believe this will have a positive impact on Elastic’s reach. What do you think about these updates and releases? Do let us know in the comments section.