Elasticsearch FAQ & Answers

Elasticsearch is a distributed search and analytics engine built on Apache Lucene, released as open source under AGPL since version 8.16.0. It stores data as JSON documents in searchable indices with near real-time capabilities. Architecture consists of clusters (node collections), nodes (servers), indices (document collections), and shards (index subdivisions). Working process: (1) Data indexed as JSON documents, (2) Text analyzed and stored in inverted index structures, (3) Documents distributed across shards for horizontal scaling, (4) Queries executed in parallel across shards with SIMD optimization, (5) Results aggregated and scored using BM25 algorithm. Elasticsearch 8.x adds vector search (HNSW graphs), semantic_text fields, data streams, and runtime fields. Primary use cases: full-text search, log analytics, metrics monitoring, ML-powered search, and RAG (Retrieval-Augmented Generation) pipelines. All operations via REST API with JSON over HTTP.

Documents are JSON objects storing data and represent the basic unit of information in Elasticsearch. They contain fields (key-value pairs) and are identified by unique _id (auto-generated or user-specified). Each document belongs to exactly one index and has metadata (_index, _id, _version, _seq_no, _primary_term). Indices are collections of documents with similar characteristics, similar to database tables. In Elasticsearch 8.x, data streams provide a higher-level abstraction over indices for time-series data with automatic rollover. Index operations: PUT /index/_doc/id (create/update), GET /index/_doc/id (read), POST /index/_update/id (partial update), DELETE /index/_doc/id. Documents are immutable internally - updates create new versions with incremented _version field. Use _source field to store original JSON. Multiple indices per cluster enable data isolation, different mappings, and lifecycle policies (hot-warm-cold tiers).

Dynamic mapping automatically detects and creates field mappings when new documents are indexed without predefined schema. Elasticsearch analyzes incoming JSON and infers types: strings become text fields with .keyword subfields (multi-field), numbers become long or double, dates detected from ISO 8601 formats, booleans auto-detected, arrays inherit element type, objects become nested objects, null values ignored. Example: {'price': 29.99} creates double, {'title': 'Product'} creates text+keyword. Control with dynamic parameter: true (default, add new fields), false (ignore new fields), strict (reject documents with unmapped fields). Dynamic templates customize rules: PUT /index {"mappings": {"dynamic_templates": [{"strings_as_keywords": {"match_mapping_type": "string", "mapping": {"type": "keyword"}}}]}}. While convenient for exploration, production should use explicit mappings to avoid mapping explosions, type conflicts, and suboptimal field configurations. Once created, field types are immutable - changes require reindexing.

Explicit mappings provide precise control over field types, analyzers, and indexing behavior. Define at index creation: PUT /products {"mappings": {"properties": {"title": {"type": "text", "analyzer": "english"}, "price": {"type": "float"}, "in_stock": {"type": "boolean"}, "tags": {"type": "keyword"}, "location": {"type": "geo_point"}, "embedding": {"type": "dense_vector", "dims": 384, "index": true, "similarity": "cosine"}}}}. Core types: text (analyzed full-text), keyword (exact values), numeric (long, double, integer, float), date, boolean, binary. Complex types: object, nested (independent object arrays), flattened (dynamic JSON), dense_vector (ML embeddings). Geo types: geo_point, geo_shape. Multi-field mapping enables multiple uses: {"name": {"type": "text", "fields": {"keyword": {"type": "keyword", "ignore_above": 256}}}}. Runtime fields (Elasticsearch 8.x) provide schema-on-read flexibility. Field types are immutable after creation - changes require reindex. Use index templates to apply mappings to multiple indices automatically.

Elasticsearch 8.x offers comprehensive field types for diverse data. Core text types: text (analyzed full-text search), keyword (exact match, aggregations, sorting), match_only_text (optimized for logs). Numeric types: long, integer, short, byte, double, float, half_float, scaled_float, unsigned_long. Date types: date (timestamps), date_nanos (nanosecond precision). Boolean: true/false values. Complex types: object (JSON objects), nested (independent array object queries), flattened (entire object as single keyword), join (parent-child relations). Specialized types: geo_point, geo_shape (geospatial), ip (IPv4/IPv6), completion (autocomplete), search_as_you_type, token_count. ML types: dense_vector (embeddings with HNSW indexing), sparse_vector, rank_feature, rank_features. New in 8.x: semantic_text (automatic embedding generation). Example with vector search: {"embedding": {"type": "dense_vector", "dims": 384, "index": true, "similarity": "cosine"}}. Multi-field: {"name": {"type": "text", "fields": {"keyword": {"type": "keyword"}}}}. Choose based on use case: keyword for filters/aggs, text for search, dense_vector for semantic similarity.

Analyzers transform text into searchable terms through a three-stage pipeline. (1) Character filters: preprocess text (html_strip removes HTML tags, mapping replaces characters, pattern_replace uses regex). (2) Tokenizer: splits text into tokens (standard splits on word boundaries, whitespace splits on spaces, ngram for partial matching, edge_ngram for autocomplete). (3) Token filters: modify tokens (lowercase, stop removes common words, stemmer reduces to root form, synonym adds equivalents, asciifolding converts accents). Built-in analyzers: standard (general purpose), simple, whitespace, keyword (no analysis), language-specific (english, german, etc.). Custom analyzer: PUT /index {"settings": {"analysis": {"analyzer": {"custom_english": {"type": "custom", "tokenizer": "standard", "filter": ["lowercase", "english_stop", "english_stemmer"]}}}}}, Test: GET /_analyze {"analyzer": "standard", "text": "The QUICK Brown Foxes!"}. Analyzer must be identical at index and search time for accurate results. Elasticsearch 8.x adds improved language support and plugin-based analyzers for specialized domains.

Inverted index is the fundamental data structure powering Elasticsearch's sub-millisecond search. Unlike traditional databases that scan documents linearly, inverted indexes map terms to document locations for O(1) lookups. Structure: term → [docID, frequency, positions]. Example: 'elasticsearch' → [doc1:pos5, doc3:pos12, doc7:pos3], 'search' → [doc1:pos6, doc2:pos8, doc3:pos13]. Search process: (1) Query text analyzed into terms, (2) Term dictionary lookup finds posting lists instantly, (3) Boolean logic applies (AND/OR/NOT on posting lists), (4) BM25 scoring using term frequencies, (5) Top-K results returned. Each text field has its own inverted index stored as Lucene segments. With SIMD optimization in Elasticsearch 8.x, term intersections are hardware-accelerated. Tradeoff: write amplification (indexing requires building/merging segments) for massive read performance gain (10,000+ queries/sec). Doc values provide column-oriented storage for aggregations. Understanding inverted indexes explains why Elasticsearch excels at full-text search versus row-oriented databases.

Full-text queries analyze search text and find matching documents with BM25 relevance scoring. Core query types: match (analyzes text, finds similar documents), match_phrase (exact phrase with word order), match_phrase_prefix (phrase with last term prefix matching), multi_match (searches multiple fields), query_string (Lucene query syntax), simple_query_string (user-friendly syntax), match_bool_prefix (prefix on last term), combined_fields (treats multiple fields as single field). Match example: GET /products/_search {"query": {"match": {"description": "wireless bluetooth speaker"}}}. Multi-field with boosting: {"query": {"multi_match": {"query": "elasticsearch", "fields": ["title^3", "content^1", "tags^2"], "type": "best_fields"}}}. Fuzziness for typo tolerance: {"match": {"title": {"query": "elastcsearch", "fuzziness": "AUTO"}}}. Elasticsearch 8.x supports semantic search via dense_vector fields with kNN. Full-text queries use analyzers (same as indexing) and score via BM25 (k1=1.2, b=0.75). Essential for natural language search with relevance ranking.

Term-level queries search exact values without text analysis, optimized for structured data filtering. Query types: term (single exact value), terms (multiple values), terms_set (minimum match count), range (numeric/date ranges), exists (field non-null check), prefix (starts with), wildcard (glob patterns with * and ?), regexp (regular expressions), fuzzy (edit distance matching), ids (document IDs). Term query: GET /products/_search {"query": {"term": {"status.keyword": "active"}}}. Multiple values: {"terms": {"tags": ["electronics", "featured"]}}. Range: {"range": {"price": {"gte": 50, "lt": 200}, "created": {"gte": "2025-01-01||/d"}}}. Date math supported: "now-7d/d". Prefix: {"prefix": {"sku.keyword": "PROD-"}}. Critical: use keyword type fields, not text (text fields are analyzed). Term queries bypass analysis, use filter context for caching, and execute faster than full-text queries. Elasticsearch 8.x optimizes with segment-level caching. Essential for faceted search, filters, and structured data queries.

Bool query combines multiple clauses with boolean logic for complex search. Four clause types: must (AND logic, all match, contributes to score), should (OR logic, at least one matches, contributes to score), must_not (NOT logic, excludes documents, no scoring), filter (AND logic, must match, no scoring, cacheable). Example: GET /products/_search {"query": {"bool": {"must": [{"match": {"description": "laptop"}}], "filter": [{"term": {"in_stock": true}}, {"range": {"price": {"gte": 500, "lte": 2000}}}], "should": [{"term": {"brand.keyword": "Dell"}}, {"term": {"brand.keyword": "HP"}}], "must_not": [{"term": {"status.keyword": "discontinued"}}], "minimum_should_match": 1}}}. Filter and must_not execute in filter context (faster, cached). Use filter for exact matches, must for relevance scoring. Minimum_should_match controls required should clauses (integer count or percentage). Bool queries nest infinitely for complex logic. Elasticsearch 8.x optimizes with query result caching and SIMD acceleration. Combine scoring (must/should) with filters for optimal performance and relevance.

Query context calculates relevance scores answering "how well does this match?" while filter context performs binary yes/no matching answering "does this match?". Query context (must, should): calculates BM25 scores, affects ranking, slower but provides relevance. Filter context (filter, must_not, range in filter): binary match, no scoring, faster execution, automatically cached by Elasticsearch, results reused across queries. Example query context: {"must": [{"match": {"title": "elasticsearch"}}]} returns documents scored 0.5-8.2. Example filter context: {"filter": [{"term": {"status.keyword": "active"}}, {"range": {"price": {"lte": 100}}}]} returns binary matches with _score=0. Performance: filters are 2-10x faster (no scoring overhead), cached at segment level, ideal for structured data. Best practice: combine both - use filters for exact matches/ranges/booleans, queries for full-text search. Combined: {"bool": {"must": [{"match": {"description": "laptop"}}], "filter": [{"term": {"in_stock": true}}]}}. Understanding context critical for optimal Elasticsearch performance.

Metrics aggregations compute numeric values from document fields for analytics. Single-value metrics: avg (average), sum (total), min (minimum), max (maximum), value_count (non-null count), cardinality (unique count using HyperLogLog), median_absolute_deviation, weighted_avg. Multi-value metrics: stats (avg, min, max, sum, count), extended_stats (adds variance, std_deviation, sum_of_squares), percentiles (P50, P95, P99), percentile_ranks, boxplot (for outlier detection). Example: GET /sales/_search {"size": 0, "aggs": {"avg_amount": {"avg": {"field": "amount"}}, "amount_stats": {"stats": {"field": "amount"}}, "unique_customers": {"cardinality": {"field": "customer_id.keyword", "precision_threshold": 40000}}, "latency_percentiles": {"percentiles": {"field": "response_ms", "percents": [50, 95, 99, 99.9]}}}}. Use doc_values (column-store, enabled by default) for fast aggregations. Scripted metrics enable custom calculations. Aggregations execute per shard then merge at coordinator. Elasticsearch 8.x adds t_test, rate metrics. Essential for dashboards and real-time analytics.

Pipeline aggregations process outputs of sibling or parent aggregations rather than raw document fields, enabling advanced analytics. Parent pipelines (operate on parent aggregation): derivative (rate of change), moving_avg (smoothing with models), moving_fn (custom moving window), serial_diff (period-over-period), cumulative_sum (running total). Sibling pipelines (operate on sibling buckets): avg_bucket, sum_bucket, min_bucket, max_bucket, stats_bucket, percentiles_bucket, bucket_script (custom calculations), bucket_selector (filter buckets). Example profit margin calculation: GET /sales/_search {"size": 0, "aggs": {"monthly": {"date_histogram": {"field": "@timestamp", "calendar_interval": "month"}, "aggs": {"revenue": {"sum": {"field": "revenue"}}, "cost": {"sum": {"field": "cost"}}, "margin": {"bucket_script": {"buckets_path": {"rev": "revenue", "cost": "cost"}, "script": "(params.rev - params.cost) / params.rev * 100"}}}}}}. Derivative for growth: {"derivative": {"buckets_path": "monthly_sales"}}. Pipeline aggs enable KPIs, trend analysis, and complex business metrics.

BM25 (Best Matching 25) is Elasticsearch's default relevance algorithm since version 5.0, improving upon TF-IDF with better term frequency saturation and field length normalization. BM25 formula: score = IDF(q_i) * (f(q_i, D) * (k1 + 1)) / (f(q_i, D) + k1 * (1 - b + b * |D| / avgdl)) where f(q_i, D) is term frequency, |D| is field length, avgdl is average field length. Parameters: k1 (default 1.2, controls term frequency saturation - higher k1 means more weight to repeated terms), b (default 0.75, controls field length normalization - 0=no normalization, 1=full normalization). BM25 addresses TF-IDF problems: term frequency doesn't grow linearly (diminishing returns), shorter documents naturally score higher. Example: GET /articles/_search {"query": {"match": {"content": "elasticsearch"}}}. Field boosting: {"multi_match": {"query": "search", "fields": ["title^3", "content"]}}. Tune with similarity settings. Use _explain API for score debugging. Most workloads should use default k1 and b values.

Shards are subdivisions of indices enabling horizontal scaling and parallelism in Elasticsearch. Each index splits into primary shards (hold original data) and replica shards (copies for HA and read throughput). Sharding enables: (1) Distributing data across nodes beyond single-node storage limits, (2) Parallel query execution across shards, (3) Load balancing read requests via replicas, (4) Fault tolerance through replica promotion. Configuration: PUT /products {"settings": {"number_of_shards": 3, "number_of_replicas": 2}}. Primary shard count is immutable after creation (requires reindex to change). Replica count is mutable: PUT /products/_settings {"number_of_replicas": 1}. Optimal shard size: 10-50GB (Elastic recommendation), <200M documents per shard. Oversarding (too many small shards) causes overhead in cluster state, memory, and recovery. Undersarding (too few large shards) slows rebalancing and recovery. Monitor: GET /_cat/shards?v. Elasticsearch distributes shards across nodes with automatic rebalancing. Use _split and _shrink APIs to adjust shard count on existing indices.

CRUD operations via REST API with JSON payloads. Create: PUT /products/_doc/1 {"name": "Laptop", "price": 999} (specified ID) or POST /products/_doc {"name": "Mouse"} (auto-generated ID). Use _create for insert-only: PUT /products/_create/1 (fails if exists). Read: GET /products/_doc/1 returns document with _source and metadata (_index, _id, _version, _seq_no, _primary_term). Multi-get: GET /_mget {"docs": [{"_index": "products", "_id": "1"}, {"_index": "products", "_id": "2"}]}. Update: POST /products/_update/1 {"doc": {"price": 899}} (partial) or {"script": {"source": "ctx._source.price -= params.discount", "params": {"discount": 100}}} (scripted). Delete: DELETE /products/_doc/1. Optimistic concurrency control: PUT /products/_doc/1?if_seq_no=42&if_primary_term=1 prevents lost updates. Bulk API for batch operations: POST /_bulk with NDJSON. Use refresh=wait_for to make changes immediately searchable. Documents are immutable internally - updates create new versions.

Bulk API batches multiple operations (index, create, update, delete) in single HTTP request for 10-100x throughput improvement. Format: newline-delimited JSON (NDJSON) with action metadata line followed by optional source. Example: POST /_bulk
{"index": {"_index": "products", "_id": "1"}}
{"name": "Laptop", "price": 999}
{"create": {"_index": "products", "_id": "2"}}
{"name": "Mouse", "price": 29}
{"update": {"_index": "products", "_id": "3"}}
{"doc": {"price": 49}}
{"delete": {"_index": "products", "_id": "4"}}
. Best practices: (1) Optimal batch size: 1000-5000 docs or 5-15MB payload, (2) Disable refresh during bulk load: PUT /index/_settings {"refresh_interval": "-1"}, re-enable after, (3) Parallel requests with 4-8 threads, (4) Monitor errors: check "errors": true in response, retry failed docs, (5) Use pipeline parameter for ingest preprocessing. Performance: typical bulk indexing reaches 10,000-50,000 docs/sec. Bulk API reduces HTTP overhead and enables transaction-like batching. Essential for data migrations and ETL pipelines.

Refresh and flush serve different purposes in Elasticsearch's write path. Refresh: makes recent changes searchable by creating new in-memory Lucene segments from index buffer. Default: every 1 second. Data is searchable but not fsync'd to disk (not durable). Manual: POST /index/_refresh. During bulk indexing, disable: PUT /index/_settings {"refresh_interval": "-1"}, then re-enable: {"refresh_interval": "1s"}. Flush: persists in-memory segments to disk via Lucene commit, fsyncs data, clears translog. Data is durable but was already searchable after refresh. Automatic: every 30 minutes or when translog size reaches threshold (default 512MB). Manual: POST /index/_flush. Key difference: refresh = searchability (memory → searchable segments), flush = durability (memory → disk fsync). Performance: frequent refresh adds indexing overhead, impacts throughput. Flush adds I/O but ensures crash recovery. Translog provides durability between flushes. Understanding critical for balancing near-real-time search vs indexing performance.

Aliases provide virtual names for one or more indices, enabling zero-downtime operations and application decoupling. Create: POST /_aliases {"actions": [{"add": {"index": "logs-2025-01", "alias": "logs-current"}}]}. Atomic alias swap for zero-downtime reindex: POST /_aliases {"actions": [{"remove": {"index": "products-v1", "alias": "products"}}, {"add": {"index": "products-v2", "alias": "products"}}]}. Write alias with is_write_index: {"add": {"index": "logs-2025-02", "alias": "logs-write", "is_write_index": true}} enables single write target when alias points to multiple indices. Filtered alias for security/multi-tenancy: {"add": {"index": "products", "alias": "premium-products", "filter": {"term": {"tier": "premium"}}, "routing": "premium"}}. Multiple indices per alias for time-series: alias "logs" → [logs-2025-01, logs-2025-02] enables querying across time ranges. List: GET /_cat/aliases?v. Best practice: applications use aliases exclusively, never direct index names. Elasticsearch 8.x optimizes alias resolution. Essential for blue-green deployments, rollover patterns, and index lifecycle management.

Reindex copies documents between indices, required when mappings are immutable (field type changes, analyzer changes, shard count adjustments). Common scenarios: adding new fields with different types, changing analyzers, optimizing shard count, migrating to new Elasticsearch versions. Reindex API: POST /_reindex {"source": {"index": "products-v1"}, "dest": {"index": "products-v2"}}. With query filter: {"source": {"index": "logs", "query": {"range": {"@timestamp": {"gte": "now-30d"}}}}, "dest": {"index": "logs-recent"}}. Transform with Painless script: {"script": {"source": "ctx._source.price_usd = ctx._source.price * 1.2", "lang": "painless"}}. Zero-downtime workflow: (1) Create destination index with new mappings, (2) Reindex data, (3) Atomic alias swap, (4) Delete old index. Remote reindex across clusters: {"source": {"remote": {"host": "https://old-cluster:9200", "username": "user", "password": "pass"}, "index": "source"}}. Monitor async: POST /_reindex?wait_for_completion=false, track with task API. Throttle: requests_per_second parameter. Handle conflicts: "conflicts": "proceed". Elasticsearch 8.x adds reindex from snapshot.

Proper shard sizing critical for performance and stability. Elastic's official guidance: 10-50GB per shard, <200M documents per shard. Calculate: estimated_data_size / target_shard_size = number_of_shards. Example: 500GB dataset / 30GB target = 17 shards, round to 20 for growth. Oversharding problems: excessive cluster state overhead, memory pressure, slow recovery, segment merging overhead. Heap guideline: <20 shards per GB heap (e.g., 32GB heap supports ~640 shards cluster-wide). Undersharding problems: large shards slow recovery, poor load distribution, rebalancing bottlenecks. Configuration: PUT /my-index {"settings": {"number_of_shards": 5, "number_of_replicas": 1}}. For time-series data, use Index Lifecycle Management (ILM) with rollover: automatic new index creation at size/age thresholds. Data tiers: hot tier (smaller shards, frequent writes), warm tier (larger shards, read-optimized), cold/frozen tier (searchable snapshots). Adjust existing: _split API (increase shards), _shrink API (decrease shards). Monitor: GET /_cat/shards?v&s=store:desc. Start conservative (1-3 shards), scale based on actual growth patterns.

Optimize indexing throughput with these techniques. (1) Disable refresh during bulk load: PUT /index/_settings {"refresh_interval": "-1"}, restore after: {"refresh_interval": "1s"}. (2) Use bulk API with 1000-5000 docs or 5-15MB batches. (3) Disable replicas during initial load: {"number_of_replicas": 0}, re-enable after. (4) Use auto-generated document IDs (faster than explicit IDs). (5) Async translog for higher throughput: {"translog.durability": "async", "translog.sync_interval": "30s"} (trade durability for speed). (6) Increase indexing buffer: indices.memory.index_buffer_size: 20% (default 10%). (7) Parallel indexing: 4-8 bulk threads per node. (8) Disable dynamic mapping if schema is known. (9) Use data streams for time-series. (10) Hardware: SSD storage, more CPU cores, sufficient heap (32GB max). Monitor: GET /_nodes/stats/thread_pool for rejections, GET /index/_stats for indexing rate. Typical optimized throughput: 10,000-100,000 docs/sec depending on document complexity. After bulk load, restore production settings (refresh_interval, replicas, translog.durability: request).

Monitor Elasticsearch clusters via APIs and observability tools. Cluster health: GET /_cluster/health returns green (all shards allocated), yellow (primary allocated, some replicas missing), red (some primary shards unassigned - data unavailable). Key metrics: number_of_nodes, active_primary_shards, active_shards, unassigned_shards, relocating_shards, initializing_shards, pending_tasks, task_max_waiting_in_queue_millis. Node stats: GET /_nodes/stats provides CPU usage, JVM heap (used/max), disk I/O, network, thread pools, GC stats. Index stats: GET /_stats shows doc count, store size, indexing rate, search rate, latency. Cat APIs (human-readable): GET /_cat/health?v, /_cat/nodes?v&h=name,heap.percent,ram.percent,cpu,load_1m,disk.used_percent, /_cat/indices?v&s=store.size:desc. Critical thresholds: heap <75%, disk <85-90% (triggers flood stage watermark), CPU <80%, GC pause <1s. Elasticsearch 8.x adds health API: GET /_health_report. Production monitoring: use Stack Monitoring (Metricbeat + Kibana), Prometheus exporter, or Elastic Cloud observability. Set alerts for red/yellow status, disk watermarks, heap pressure, search latency spikes.

Deep pagination requires specialized approaches due to from/size performance costs. Standard pagination: GET /index/_search {"from": 100, "size": 20} limited to index.max_result_window (default 10,000) - coordinator must sort (from + size) * num_shards results. For real-time pagination, use search_after (stateless, efficient): GET /products/_search {"size": 20, "query": {...}, "sort": [{"price": "asc"}, {"_id": "asc"}]}. Next page uses last doc's sort values: {"search_after": [99.99, "prod123"], "sort": [...]}. Sort must include tie-breaker (e.g., _id). For batch exports, use Point in Time (PIT) + search_after: (1) Open PIT: POST /products/_pit?keep_alive=5m, (2) Search with PIT: {"size": 1000, "query": {...}, "pit": {"id": "...", "keep_alive": "5m"}, "sort": [...]}. PIT provides consistent snapshot across pagination. Scroll API (legacy, deprecated in 8.x): POST /index/_search?scroll=2m {"size": 1000} returns scroll_id for subsequent requests. Best practices: search_after for UI pagination, PIT for data exports, avoid from/size beyond 10K. Elasticsearch 8.x optimizes search_after with segment-level shortcuts.

Elasticsearch security (included with Basic license, enabled by default in 8.x) provides authentication, authorization, encryption, and audit logging. Authentication realms: native (built-in user DB), file (users in elasticsearch-users), LDAP/Active Directory, SAML, Kerberos, PKI (x.509 certificates), OIDC, API keys, service tokens. Create user: POST /_security/user/analyst {"password": "SecurePass123!", "roles": ["kibana_admin", "monitoring_user"]}. Role-Based Access Control (RBAC): PUT /_security/role/logs_reader {"cluster": ["monitor"], "indices": [{"names": ["logs-*"], "privileges": ["read", "view_index_metadata"]}]}. Granular controls: field-level security (FLS) restricts field visibility per role, document-level security (DLS) filters documents via query DSL. TLS encryption: transport layer (inter-node, required) and HTTP layer (client-node, recommended) with CA-signed certificates. API keys for service accounts: POST /_security/api_key {"name": "app-key", "expiration": "365d", "role_descriptors": {...}}. Audit logging: xpack.security.audit.enabled: true tracks authentication, authorization, and data access. Elasticsearch 8.x auto-generates passwords and certificates during setup. Best practices: TLS everywhere, least privilege roles, rotate credentials, monitor audit logs.