Elasticsearch Fundamentals for Engineers

Cluster Topology & Single-Intent Query Routing

Elasticsearch operates on a distributed, document-oriented architecture built atop Apache Lucene. The engineering decision this guide resolves is how to lay out nodes, shards, and segments so that read and write paths scale independently. Engineering teams must separate master-eligible, data, and coordinating nodes. This isolation prevents split-brain scenarios and optimizes query routing. Align node allocation with foundational principles from Search Engine Selection & Architecture to prevent resource contention.

Target 30–50GB per primary shard during index creation. Configure replicas strictly for fault tolerance. Disable dynamic mapping in production templates to enforce schema stability. Route all client traffic through dedicated coordinating nodes.

Index to shard to segment hierarchy across nodes An index splits into primary shards distributed across data nodes, each replicated, with every shard composed of immutable Lucene segments. Index: products 3 primaries, 1 replica Data node A Data node B Data node C P0 (primary) R2 (replica) P1 (primary) R0 (replica) P2 (primary) R1 (replica) Shard P0 = immutable Lucene segments seg_1 seg_2 seg_3 merge -> fewer segments 
# elasticsearch.yml (Master Node)
node.name: "master-01"
node.roles: ["master"]
discovery.seed_hosts: ["10.0.1.10", "10.0.1.11", "10.0.1.12"]
cluster.initial_master_nodes: ["master-01", "master-02", "master-03"]

Implementation Steps

  • Configure dedicated master-eligible nodes (minimum 3) with node.roles: [master]
  • Set discovery.seed_hosts and cluster.initial_master_nodes for deterministic bootstrapping
  • Enforce index.number_of_shards at creation; avoid post-creation shard splitting
  • Route client requests through coordinating nodes to isolate query parsing from data retrieval

Measurable Tradeoffs Increasing replicas improves read throughput and availability but linearly increases indexing latency (15–25% per replica) and storage overhead. Dedicated node roles reduce garbage collection pauses by 40% but require more precise capacity planning.

Schema Enforcement & Ingestion Pipeline Design

Production indexing demands strict schema enforcement to prevent mapping explosions. Configure dynamic: strict in index templates. Define explicit field types (text, keyword, date, geo_point) upfront. While Elasticsearch handles unstructured data gracefully, teams requiring lightweight defaults should evaluate alternatives via Meilisearch vs Typesense Comparison before committing to heavy mapping configurations.

Use ingest pipelines with processors like gsub, date, and script to normalize payloads. Pre-validate payloads against JSON Schema before transmission. Leverage the bulk API with refresh=false during high-throughput windows.

curl -X PUT "localhost:9200/_index_template/production_logs" \
  -H 'Content-Type: application/json' \
  -d '{
    "index_patterns": ["logs-*"],
    "template": {
      "settings": {"index.refresh_interval": "30s"},
      "mappings": {
        "dynamic": "strict",
        "properties": {
          "message": {"type": "text"},
          "level": {"type": "keyword"}
        }
      }
    }
  }'

Implementation Steps

  • Define index templates with dynamic: strict and explicit properties
  • Build ingest pipelines for field normalization, PII redaction, and timestamp parsing
  • Use _bulk API with batch sizes of 5–10MB and refresh_interval: 30s during high-throughput ingestion
  • Implement retry logic with exponential backoff for 429/503 responses

Measurable Tradeoffs Strict mapping ensures predictable query performance and reduces cluster memory pressure by 20–30%, but requires upfront schema governance and breaks backward compatibility on field type changes. Disabling auto-refresh during bulk loads improves indexing throughput by 3–5x but delays document visibility.

Lucene Execution & Latency Tuning

Query performance hinges on understanding Lucene’s inverted index and segment merging mechanics. Optimize by leveraging filter contexts for caching. Avoid wildcard or regex queries on high-cardinality fields. search_type: query_then_fetch (the default) is the correct mode for distributed pagination — dfs_query_then_fetch adds a global term-frequency pre-fetch for more accurate scoring at higher latency cost.

Tune indices.fielddata.cache.size to no more than 20% of heap. Configure index.max_result_window (default 10,000) to prevent heap exhaustion on deep pagination — prefer search_after instead. Use track_total_hits: false when exact counts are unnecessary.

curl -X POST "localhost:9200/products/_search" \
  -H 'Content-Type: application/json' \
  -d '{
    "track_total_hits": false,
    "query": {
      "bool": {
        "filter": [
          {"term": {"status": "active"}},
          {"range": {"price": {"gte": 10}}}
        ],
        "must": [{"match": {"description": "wireless headphones"}}]
      }
    }
  }'

Implementation Steps

  • Wrap exact-match and range queries in bool.filter to bypass scoring and leverage segment cache
  • Set indices.query.bool.max_clause_count to 1024 for complex boolean logic
  • Implement search_after for deep pagination instead of from/size
  • Monitor search.query_time_in_millis and fetch_time_in_millis via _stats API

Measurable Tradeoffs Filter caching reduces CPU utilization by 30–50% for repeated queries but increases heap pressure. Disabling track_total_hits cuts query latency by 15–20% on large datasets but sacrifices accurate pagination metadata. To track search.query_time_in_millis against an SLO and alert before degradation reaches users, wire these metrics into observability and SRE for search.

Index Lifecycle & Tiered Storage Orchestration

Managing data retention and tiered storage is non-negotiable for production clusters. Implement Index Lifecycle Management (ILM) policies to automate hot-warm-cold-frozen transitions. Configure rollover triggers at 50GB or 30 days. Apply shrink operations in the warm phase and enforce delete after compliance windows. Detailed policy orchestration is covered in Elasticsearch index lifecycle management.

Attach policies directly to index templates. Monitor step progression via _ilm/explain. Configure snapshot repositories for disaster recovery.

curl -X PUT "localhost:9200/_ilm/policy/log_retention" \
  -H 'Content-Type: application/json' \
  -d '{
    "policy": {
      "phases": {
        "hot": {"actions": {"rollover": {"max_size": "50gb", "max_age": "30d"}}},
        "warm": {"actions": {"shrink": {"number_of_shards": 1}, "forcemerge": {"max_num_segments": 1}}},
        "delete": {"min_age": "90d", "actions": {"delete": {}}}
      }
    }
  }'

Implementation Steps

  • Define ILM policies with hot (rollover), warm (shrink/forcemerge), cold (allocate to low-cost nodes), and delete phases
  • Attach policies to index templates via index.lifecycle.name
  • Configure snapshot repositories to S3/GCS with snapshot lifecycle policies
  • Monitor index.lifecycle.step via _ilm/explain to detect policy stalls

Measurable Tradeoffs Automated rollover prevents shard bloat and maintains query consistency, but shrink operations require temporary disk space equal to index size. Cold-tier migration cuts storage costs by 60–75% but increases retrieval latency and requires explicit searchable_snapshots configuration.

Embedding Integration & Semantic Retrieval Scaling

Modern search pipelines increasingly combine lexical BM25 scoring with dense vector embeddings. While Elasticsearch supports dense_vector fields and k-NN search, production hybrid retrieval requires careful weight calibration. Integrate embedding generation upstream. Use script_score or rank queries to blend relevance signals. For teams scaling beyond traditional keyword matching, review Vector Search Integration Strategies to align embedding pipelines with cluster capacity.

Define vector dimensions explicitly. Configure num_candidates to balance recall against compute overhead. Profile query latency continuously.

curl -X PUT "localhost:9200/semantic_docs" \
  -H 'Content-Type: application/json' \
  -d '{
    "mappings": {
      "properties": {
        "content": {"type": "text"},
        "embedding": {"type": "dense_vector", "dims": 768, "index": true, "similarity": "cosine"}
      }
    }
  }'

Implementation Steps

  • Define dense_vector fields with dims matching your embedding model (e.g., 768 for BERT, 1536 for OpenAI)
  • Configure index.knn settings with num_candidates (100–500) and similarity (cosine/dot_product)
  • Implement hybrid scoring using rank or rrf (Reciprocal Rank Fusion) in multi_match + knn queries
  • Profile knn query latency and adjust num_candidates to balance recall vs. p95 response time

Measurable Tradeoffs Hybrid retrieval improves zero-query and synonym handling by 25–35%, but vector indexing increases cluster memory footprint by 3–4x. Tuning num_candidates below 200 reduces latency by 40% but may degrade recall for rare, infrequent queries.

Prerequisites

  • A running Elasticsearch 8.x or OpenSearch 2.x instance you can change settings on, reachable at localhost:9200.
  • A realistic estimate of corpus size at 12 and 36 months, because shard decisions are effectively permanent.
  • Node specifications: vCPU, RAM, and whether storage is local SSD or network-attached.
  • An expected query shape — filtered, faceted, sorted — since it determines how much work each shard does.

Sizing shards: the decision you cannot easily undo

Shard count is fixed at index creation, and changing it means a rebuild. That makes it the highest-stakes number in the whole configuration and the one most often chosen by copying a default.

Two forces pull in opposite directions. Too few shards and a single shard grows past the point where merges and queries are efficient — the practical ceiling is 30–50 GB per shard for typical search workloads. Too many and every query pays fan-out cost across shards that hold almost nothing, every refresh runs per shard whether or not it has work, and cluster state grows with the shard count on every node.

The rule that survives contact with production is to size for the corpus you expect in a year, target 20–40 GB per shard, and never exceed roughly 20 shards per GB of JVM heap across the whole node. A 10-million-document catalog of 2 KB records is about 20 GB of primary data after indexing overhead, which is one or two shards — not the five that most examples use.

# Measure before deciding: index a representative sample and extrapolate.
curl -s 'localhost:9200/_cat/indices/sample?v&h=docs.count,pri.store.size'
# docs.count pri.store.size
# 100000     212mb              -> ~2.1 KB per doc on disk
# 10M docs ≈ 21 GB primary → 1 shard today, 2 for headroom
Cost of too few and too many shards Few large shards make merges and queries slow, many small shards multiply fan-out and refresh overhead, and the workable band sits between them. too few shards huge segments, slow merges no parallelism to exploit 20–40 GB per shard merges finish, queries parallelise cluster state stays small too many shards fan-out on every query refresh cost per empty shard the common mistake is the right-hand box, not the left defaults and tutorials suggest five shards for corpora that need one
Over-sharding is the more common error and the more expensive one, because its cost is paid on every query and every refresh forever.

Heap is the resource that decides everything else

The JVM heap is the constraint that most often turns a working cluster into a struggling one, and it behaves unlike disk or CPU: it does not degrade gracefully. Below the limit, everything is fine; above it, garbage collection pauses lengthen, queries time out, and the node may leave the search cluster entirely.

Two rules bound it. Set heap to no more than half of available RAM, leaving the rest for the filesystem cache that Lucene depends on — a node with 64 GB of RAM should run a 30–31 GB heap, not 48 GB. And keep heap under about 31 GB regardless of machine size, because above that threshold the JVM loses compressed object pointers and effectively wastes the additional memory. A 128 GB machine runs two nodes with 31 GB heaps, not one with 64 GB.

# Check both numbers on every node before diagnosing anything else.
curl -s 'localhost:9200/_cat/nodes?v&h=name,heap.percent,ram.percent,heap.max'
# name      heap.percent ram.percent heap.max
# es-data-1           68          94    30.9gb
# es-data-2           71          95    30.9gb

Sustained heap above roughly 75% is a warning; above 85% the node is effectively in trouble whether or not anything has failed yet. The usual causes are, in order of frequency: too many shards (each carries fixed overhead), unbounded aggregation requests, and large size values on queries that then have to hold thousands of documents in memory to merge them.

How node memory should be divided Half of RAM goes to the JVM heap capped near thirty-one gigabytes, and the remainder is left for the filesystem cache that Lucene relies on. 64 GB node JVM heap ≈ 31 GB filesystem cache — Lucene lives here the mistake heap 48 GB — no compressed pointers, starved page cache, slower than 31 GB More heap is not more capacity. Past ~31 GB it is strictly worse, which surprises almost everyone the first time.
The half-of-RAM rule and the 31 GB ceiling interact: on large machines the correct answer is more nodes, not more heap.

Where query time actually goes

Latency in a search query is not one number; it is a sum over stages that respond to different fixes. The coordinating node parses and fans out, each shard rewrites the query and executes it against its segments, results are collected and merged, then documents are fetched. A slow query is slow in one of those stages, and the profile API tells you which.

curl -s 'localhost:9200/products/_search?pretty' -H 'Content-Type: application/json' -d '{
  "profile": true,
  "query": { "bool": { "must": [{"match": {"title": "trail runner"}}],
                       "filter": [{"term": {"brand": "acme"}}] } }
}' | jq '.profile.shards[0].searches[0].query[0] | {type, time_in_nanos}'
# => { "type": "BooleanQuery", "time_in_nanos": 2841300 }

Two findings recur. First, filters that are not cached dominate — a term filter on a high-cardinality field, or a range on a date with now in it, defeats the filter cache and is recomputed per query. Rounding date ranges to the hour is often the single largest query optimisation available. Second, the fetch phase can exceed the query phase when documents are large: retrieving twenty 200 KB documents costs more than matching them. Source filtering to the fields the UI actually renders frequently halves total latency without touching the query at all.

Stages of a search request and what fixes each Parse and fan-out, shard query execution, result merge and document fetch, each with a different optimisation. parse + fan-out fix: fewer shards shard execution fix: cacheable filters merge fix: smaller size fetch fix: source filtering optimising the wrong stage is the usual outcome of tuning without a profile the fetch phase is the one most often overlooked and the easiest to fix
Four stages, four different fixes. The profile API names the stage in one request, which is why it belongs at the start of any latency investigation.

Performance & Scale Notes

Numbers from a three-node cluster with 8 vCPU and 32 GB RAM per node, indexing 2 KB documents, useful mainly for their ratios:

  • Query latency scales with segment count, not document count, over a wide range. Halving segments through a force-merge typically cuts p95 by 15–30% on a read-heavy index, which is why merges matter more than most tuning.
  • Shard fan-out costs roughly 1–3 ms per shard on a trivially matching query. At five shards that is invisible; at fifty it is the dominant term for a fast query.
  • Filter caching is worth 5–20× on repeated filters, and it is defeated entirely by a now-relative date range. Rounding to the hour is usually the single largest query win available.
  • Replica count scales read throughput close to linearly until the coordinating node saturates, which happens around six to eight replicas on typical hardware.
  • Recovery time after a node loss is bounded by network throughput and shard size: a 30 GB shard over a 1 Gbps link takes about four minutes to copy, and that number multiplies by the shards the failed node held.

Configuration Reference

Name Default Type Effect
index.number_of_shards 1 integer Primary shard count, fixed at creation. Target 20–40 GB per shard at projected size; changing it requires a rebuild.
index.number_of_replicas 1 integer Copies per shard. Each adds read capacity and multiplies write cost; zero during bulk loads, at least one in production.
ES_JAVA_OPTS heap quarter of RAM size JVM heap. Half of RAM, capped near 31 GB; larger values lose compressed pointers and starve the page cache.
index.max_result_window 10000 integer Deep-pagination ceiling. Raising it invites heap exhaustion; use search_after for deep paging instead.
indices.query.bool.max_clause_count 1024 integer Clause limit per boolean query. Synonym expansion and long filter lists hit this before anything else does.
index.routing.allocation.total_shards_per_node unbounded integer Caps shards per node for an index, preventing one index from concentrating on a single node after a restart.

A useful habit when any of these settings comes up in review: ask whether the change can be made in place or requires a rebuild. That single question sorts every setting into two piles with very different risk profiles, and it is the question that most reviews skip.

Failure Modes & Debugging

Cluster turns yellow and stays there

Symptom: health is yellow indefinitely; replicas are unassigned while primaries are fine.

Root cause: usually a single-node cluster where replicas cannot be allocated, or a disk watermark preventing allocation.

Remediation: ask the search cluster directly rather than guessing — the allocation explain API names the reason:

curl -s 'localhost:9200/_cluster/allocation/explain?pretty' | jq -r '.allocate_explanation'
Queries slow down as the corpus grows, then recover after a merge

Symptom: p95 latency drifts upward over days and drops sharply without any deploy.

Root cause: segment count growing faster than merges consolidate it, typically because indexing is running near the disk-bandwidth ceiling.

Remediation: track segment count per shard as a first-class metric. If it trends upward across a week, indexing is outrunning merges and needs either more headroom or a lower sustained write rate.

Sudden `circuit_breaking_exception` on aggregations

Symptom: faceted queries that worked yesterday start failing under load.

Root cause: an aggregation on a high-cardinality field builds a large in-memory structure per shard, and the request-level circuit breaker correctly refuses it.

Remediation: bound the aggregation with size and shard_size, or move the field to keyword with eager global ordinals if it is faceted constantly. Raising the breaker limit converts a clean rejection into a node failure and should be the last resort.

Index becomes read-only during a bulk load

Symptom: writes fail with cluster_block_exception mid-load; reads keep working.

Root cause: the flood-stage disk watermark tripped, which puts indices into read-only mode to protect the node.

Remediation: free disk, then explicitly clear the block — it does not lift by itself:

curl -s -X PUT 'localhost:9200/_all/_settings' -H 'Content-Type: application/json' \
  -d '{"index.blocks.read_only_allow_delete": null}'

The recurring theme across all of this is that the expensive decisions — shard count, mapping, heap — are made once and paid for continuously, while the cheap ones are adjustable at any time. Spending an afternoon on the first group before the index exists is worth more than weeks of tuning the second group afterwards, and it is the difference between a search cluster that grows with the product and one that has to be rebuilt to accommodate it.