Elastic Search Introduction

• The course aims to provide a solid foundation in search and information retrieval. Starting with fundamental concepts and covers best practices, key features, and distributed search application development with ElasticSearch. During the course there will be time for discussion as well as attendee case studies. At the end of the training you will have an in-depth understanding of how Elasticsearch works, you will be able to reliably analyze, understand, and solve common problems, and be ready to build state-of-the-art search applications

Intended Audience:

• Networking specialists
• Programmers
• Engineers

Key Skills:

• Real Time Search Skills
• Spinning up a Cluster in class
• room. Lucene Internals
• Fully immersive experience in managing and operating Elasticsearch clusters.
• Best practices and examples for document design, testing, and other software development design considerations are also covered
• Real Time Case Study
• Guidelines and tips on processing custom log formats
• Creation of dashboards in Kibana
• Choosing hardware Logstash
• configuration Designing
• system to scale
• Canaging the lifecycle of your logs

Prerequisites:

• Working knowledge of Linux would help as it would be the platform used.

Instructional Method:

• This is an instructor led course which provides lecture topics and the practical application of Hadoop and the underlying technologies. It pictorially presents most concepts and there is a detailed case study that strings together the technologies, patterns and design.

Introduction to ElasticSearch

• Getting familiar with Lucene
• Index
• Introducing Apache Lucene
• Introducing ElasticSearch
• Shard
• Gateway
• Cluster
• Node
• Mapping
• Understanding the Basic
• Document
• Type
• Basic Concepts
• Overall architecture
• Replica

Managing Mapping

• Mapping a GeoShape field
• Mapping different analyzers
• Mapping base types
• Mapping an attachment field
• Mapping an IP field
• Mapping a GeoPoint field
• Adding generic data to mapping
• Mapping arrays
• Managing a child document
• Using dynamic templates in document mapping
• Introduction
• Mapping a document
• Managing nested objects
• Using explicit mapping creation
• Mapping a multifield
• Mapping an object

Scripting

• Sorting using script
• Installing additional script plugins
• Introduction
• Filtering a search via scripting
• Computing return fields with scripting
• Updating with scripting

Power User Query DSL
• MultiSearch
• Bulk Operations
• MultiGet
• Filters and caching
• Creating and deleting documents using the Update API
• Sorting data
• Conditional modifications using scripting
• Using filters to optimize your queries
• Sorting with multivalued fields
• Sorting with multivalued geo fields

Index Distribution Architecture

• Replicas
• Multiple shards versus multiple indices
• Configuration
• A positive example of over allocation
• Data volume and queries specification
• Querying
• Indexing with routing
• Introducing ShardAllocator
• Sharding and over allocation
• Deciders
• Filtering
• Shards and data
• Assumptions
• Introducing the preference parameter
• Routing explained
• Choosing the right amount of shards and replicas

ElasticSearch Java APIs

• Suggestions
• The code
• Bulk
• CRUD operations
• Faceting
• Becoming the ElasticSearch node
• Paging
• Introducing the ElasticSearch Java API
• Counting
• Sorting
• Performing multiple actions
• Highlighting
• Filtering
• Using the geo shape query
• Scrolling
• Building JSON queries and documents
• Fetching documents
• Querying ElasticSearch
• Connecting to your cluster
• Handling errors

Elastic Search Aggregations

• Introduction to Elastic Search Aggregations
• Metric Aggregations
• Values Source
• Caching Heavy Aggregations
• Bucket Aggregations
• Structure of an Aggregation

Key concepts behind ElasticSearch architecture

• Failure detection
• The boostrap process
• Communicating with ElasticSearch
• Working of ElasticSearch

ElasticSearch Administration

• Index-level recovery settings
• Zen discovery
• Introducing the segments API
• Discovery configuration
• Choosing the right directory implementation – the store module
• Recovery configuration
• Minimum master nodes
• The filter cache
• Zen discovery fault detection
• Unicast
• Cluster-level recovery configuration
• The response
• Segments statistics

Troubleshooting with Elastic Search

Controlling I/O throttling
• Java memory
• Configuration example
• Knowing the garbage collector
• Querying with warmer present
• Adjusting garbage collector work in ElasticSearch
• Adding warmers to templates
• Creating memory dumps
• Querying without warmers present
• Adding warmers during index creation
• Turning on logging of garbage collection work
• Maximum throughput per second
• Very hot threads
• Using JStat
• Reason for using warmers
• Configuration
• Speeding up queries using warmers

Kibana

• Visualization and related features
• Security/Access control to elastic search
• Visualization using kibana
• using queries, single and multiquery
• Utilization of filters, logger tools
• Introduction
• Deployment strategies around indexing and searching high data inflow (lacks of records per second)
• Search criteria and filters, JSON Input