How to configure OpenSearch ISM hot-warm-cold architecture

This guide walks through the exact node settings, index template, policy JSON, and Python attachment workflow needed to move time-series indices deterministically from NVMe to SSD to HDD and finally to deletion, without unassigned shards or stalled transitions.

A tiered storage model only works when hardware capabilities, shard routing rules, and Index State Management (ISM) policy execution stay in lockstep. Misconfigured node attributes, malformed phase thresholds, or silent policy detachment translate directly into unassigned shards, uncontrolled rollovers, and degraded query latency. This procedure builds on the broader OpenSearch ISM Architecture & Fundamentals model and the capacity ratios established in the parent Hot-Warm-Cold Tier Design guide — read those first if you have not sized your tiers yet.

Prerequisites #

Confirm each item before applying any payload to a production cluster. Every step below assumes these are already true.

• Three data node groups exist (hot, warm, cold) with distinct storage backends (NVMe, SSD, HDD).
• Node Role Allocation is planned — you know which node.roles and node.attr.data value each group will carry.
• The opensearch-security plugin grants your automation account write access to _plugins/_ism/* and _index_template/*, scoped per Security & Access Boundaries.
• Disk watermarks are tuned for multi-tier hardware (defaults throttle HDD cold nodes prematurely).
• opensearch-py is installed in the automation environment (pip install opensearch-py).
• You have a rough hourly ingestion figure so rollover age and size thresholds can be sized realistically.

Size the hot tier from measured ingestion so a full retention window fits before the first transition fires:

Step-by-step configuration #

1. Assign node roles and tier attributes #

OpenSearch routes indices to hardware tiers using node attributes, and the ISM allocation action enforces placement through index.routing.allocation.require.data. Set both the native role and an explicit node.attr.data value in opensearch.yml on each node group. Do not rely on implicit tier roles alone — the explicit attribute guarantees deterministic routing across cluster upgrades, exactly as described in Data Tier Routing Patterns.

# Hot tier — high IOPS NVMe, handles ingest and real-time query
node.name: "hot-01"
node.roles: ["data_hot", "ingest"]
node.attr.data: "hot"
path.data: /mnt/nvme/opensearch

# Warm tier — balanced SSD, recent history and retained search
node.name: "warm-01"
node.roles: ["data_warm"]
node.attr.data: "warm"
path.data: /mnt/ssd/opensearch

# Cold tier — high-capacity HDD, archive and infrequent access
node.name: "cold-01"
node.roles: ["data_cold"]
node.attr.data: "cold"
path.data: /mnt/hdd/opensearch

Restart nodes sequentially after applying changes, then confirm the attribute propagated:

curl -s "localhost:9200/_cat/nodeattrs?v&h=node,attr,value" | grep '\bdata\b'

Expected output — one row per node with the tier value you set:

hot-01   data   hot
warm-01  data   warm
cold-01  data   cold

Gotcha: if data does not appear in this output, the attribute never loaded and every ISM allocation transition will silently fail to relocate shards. Fix opensearch.yml and restart before continuing.

2. Create a versioned index template with routing enforcement #

The template binds new indices to the policy and pins their initial placement on the hot tier at creation time. Use the composable _index_template (v2) API so versioned templates never collide with legacy ones.

PUT _index_template/logs_tiered_v2
{
  "index_patterns": ["logs-app-*"],
  "priority": 500,
  "template": {
    "settings": {
      "index.plugins.index_state_management.policy_id": "hot_warm_cold_policy",
      "index.routing.allocation.require.data": "hot",
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "refresh_interval": "5s"
    },
    "mappings": {
      "properties": {
        "@timestamp": { "type": "date" },
        "message":    { "type": "text" },
        "trace_id":   { "type": "keyword" }
      }
    }
  }
}

Expected response:

{ "acknowledged": true }

Key fields:

• index.plugins.index_state_management.policy_id binds the index to the ISM policy at creation. This is the OpenSearch setting; the Elasticsearch ILM equivalent index.lifecycle.name is ignored by OpenSearch.
• index.routing.allocation.require.data forces the first shards onto hot nodes only.
• priority must exceed any legacy template (> 100) so it wins the merge.

Gotcha: a version suffix (_v2) lets you roll the template forward or back without disrupting active indices. Bumping a template never rewrites indices that already exist — those keep the settings they were created with.

3. Define the hot-warm-cold ISM policy #

The policy is the state machine: it defines rollover triggers, per-phase routing reallocation, and delete conditions. ISM evaluates conditions every 5 minutes by default (plugins.index_state_management.job_interval). Tune thresholds to your storage capacity and query SLA.

PUT _plugins/_ism/policies/hot_warm_cold_policy
{
  "policy": {
    "description": "Tiered lifecycle for application logs",
    "default_state": "hot",
    "states": [
      {
        "name": "hot",
        "actions": [
          { "rollover": { "min_index_age": "1d", "min_primary_shard_size": "50gb" } }
        ],
        "transitions": [
          { "state_name": "warm", "conditions": { "min_index_age": "30d" } }
        ]
      },
      {
        "name": "warm",
        "actions": [
          { "allocation": { "require": { "data": "warm" } } },
          { "replica_count": { "number_of_replicas": 0 } },
          { "force_merge": { "max_num_segments": 1 } }
        ],
        "transitions": [
          { "state_name": "cold", "conditions": { "min_index_age": "60d" } }
        ]
      },
      {
        "name": "cold",
        "actions": [
          { "allocation": { "require": { "data": "cold" } } },
          { "read_only": {} }
        ],
        "transitions": [
          { "state_name": "delete", "conditions": { "min_index_age": "90d" } }
        ]
      },
      {
        "name": "delete",
        "actions": [ { "delete": {} } ]
      }
    ]
  }
}

Threshold guidance:

• Hot → Warm at 30d offloads recent indices before NVMe saturates.
• Warm → Cold at 60d, after force_merge collapses segments to one — this reduces query overhead and disk footprint before the shard lands on slow HDD.
• Cold → Delete at 90d for compliance retention; the read_only action blocks stray writes during archival.
• Match min_primary_shard_size to real shard density — oversized shards delay transitions, undersized shards inflate cluster overhead.

Gotcha: only indices created after the template is in place inherit policy_id automatically. Existing indices need the explicit attach in step 4.

4. Attach the policy across existing indices with Python #

For indices that predate the template, attach the policy through the _plugins/_ism/add API — it is an ISM operation, not an index setting. This idempotent script pages through matching indices, treats an already-attached index as success, and logs failures without blocking ingestion.

import logging
from opensearchpy import OpenSearch, exceptions

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)


def attach_ism_policy(client: OpenSearch, index_pattern: str, policy_id: str) -> int:
    """Attach an ISM policy to every index matching a pattern. Returns count attached."""
    attached = 0
    try:
        indices = client.cat.indices(index=index_pattern, h="index", format="json")
    except exceptions.ConnectionError as exc:
        logger.critical("Cluster connection failed: %s", exc)
        return 0

    for entry in indices:
        idx = entry["index"]
        try:
            client.transport.perform_request(
                "POST",
                f"/_plugins/_ism/add/{idx}",
                body={"policy_id": policy_id},
            )
            attached += 1
        except exceptions.RequestError as exc:
            if "already" in str(exc).lower():
                logger.info("Policy already attached to %s", idx)
            else:
                logger.error("Failed to attach policy to %s: %s", idx, exc)
    logger.info("Attached policy to %d indices.", attached)
    return attached


if __name__ == "__main__":
    client = OpenSearch(
        hosts=["https://opensearch-cluster.internal:9200"],
        http_auth=("admin", "secure_password"),
        verify_certs=True,
        timeout=30,
    )
    attach_ism_policy(client, "logs-app-*", "hot_warm_cold_policy")

Schedule this via cron or a Kubernetes CronJob so policy compliance is re-enforced after every template change or cluster migration. See the OpenSearch Python Client documentation for authentication and connection-pooling options.

Gotcha: _ism/add attaches but does not retroactively rewind an index that is already mid-lifecycle under a different policy. Remove the old policy first with _plugins/_ism/remove if you are re-homing indices.

Verification #

Run these three checks after the first ISM poll cycle (allow ~5 minutes) to confirm the pipeline is live.

Confirm the managed index is executing.

curl -s "localhost:9200/_plugins/_ism/explain/logs-app-2026.07.01"

Expected — the index reports its policy, current state, and a successful last step:

{
  "logs-app-2026.07.01": {
    "index.plugins.index_state_management.policy_id": "hot_warm_cold_policy",
    "state": { "name": "hot" },
    "action": { "name": "rollover" },
    "step": { "name": "attempt_rollover", "status": "condition_not_met" }
  }
}

Confirm shards landed on the correct tier.

curl -s "localhost:9200/_cat/shards/logs-app-2026.07.01?v&h=index,shard,prirep,state,node"

Every shard should show STARTED on a hot-* node. A warm-* or cold-* node here means the template routing setting was missing at creation.

Confirm the write alias rolls over. After a rollover, _cat/indices should show a new incremented index (logs-app-000002) in the hot state while the previous generation continues aging toward warm.

Common failure modes #

When a require filter matches no available node, a graceful secondary target keeps the index from deadlocking — implement one using Fallback Routing Strategies. For the sequencing rules behind rollover, force-merge, and delete, see Index Lifecycle Basics.

The timeline below maps the four states configured above onto the index-age axis, showing the routing attribute ISM stamps, the actions each state runs, and the storage medium the shards live on at every stage — the complete contract this procedure implements.

Frequently asked questions #

Related #

• Hot-Warm-Cold Tier Design — capacity ratios and watermark tuning that set the thresholds this procedure uses.
• Data Tier Routing Patterns — how require, include, and exclude deciders resolve each shard placement.
• Fallback Routing Strategies — keep a transition from deadlocking when a tier runs out of matching nodes.

Up: Hot-Warm-Cold Tier Design