SOASAP
Sign In Sign Up Free

High Availability

Local evaluation keeps applications operational during control-plane, synchronization, and network disruptions.

Introduction

SOASAP separates feature flag evaluation from configuration delivery. Applications evaluate flags from an in-memory snapshot inside the application process. SOASAP Cloud manages and distributes configuration, but it is not part of the request-time evaluation path.

As a result, temporary failures involving SOASAP Cloud, Internet connectivity, DNS resolution, TLS connections, or SSE synchronization do not automatically make the application unavailable.

Availability vs freshness. High availability in SOASAP means that application request handling can continue even when configuration synchronization is temporarily unavailable. It does not mean every SDK always has the newest configuration during an outage.

Availability model

SOASAP splits responsibility between a control plane that distributes configuration and a data plane that evaluates it inside the application.

Control plane

The control plane includes:

  • SOASAP Dashboard
  • configuration storage
  • flag management
  • environment management
  • API key management
  • snapshot distribution
  • SSE synchronization

The control plane determines which configuration should be active.

Data plane

The application-side data plane includes:

  • the SDK
  • the in-memory snapshot
  • local flag evaluation
  • explicit default values
  • persistent cache where supported

The data plane determines how flags are evaluated inside the running application.

The control plane distributes state. The data plane evaluates state. Application requests do not need to contact the control plane.

Request-time evaluation path

No network request is made during flag evaluation. No connection to SOASAP Cloud is required for each read. No disk access occurs in the evaluation hot path.

This isolates application request handling from:

  • cloud-service latency
  • remote API failures
  • network packet loss
  • DNS failures
  • rate limits
  • synchronization interruptions

A synchronization problem affects configuration freshness, not the availability of the local evaluation path. See Local Evaluation.

Online and degraded states

The SDK operates through several states. These states affect configuration freshness, but local evaluation remains available when a snapshot or safe defaults exist.

Synchronized

The SDK is connected and has the latest received snapshot. Evaluation uses the in-memory snapshot.

Reconnecting

The synchronization connection has been interrupted. Evaluation continues using the last known snapshot. The SDK attempts to reconnect in the background.

Offline

The control plane or network remains unavailable. Evaluation continues from the last known snapshot. Values remain stable until synchronization resumes.

Uninitialized

No snapshot has ever been synchronized or restored. Code-defined defaults are used for missing values. See Default Values.

Control-plane failure

When SOASAP Cloud is unavailable, synchronization pauses while local evaluation continues.

  • existing flag values remain available
  • application requests continue to evaluate locally
  • dashboard changes cannot reach disconnected SDKs
  • reconnection occurs after service recovery
  • no application restart should be required

The control plane is not irrelevant. It remains necessary for configuration changes and synchronization. High availability means request handling does not depend on it being reachable at evaluation time. See Outages.

Network failure

The same evaluation model applies when an application cannot reach SOASAP Cloud because of outbound firewall rules, DNS failures, proxy failures, routing problems, regional connectivity issues, or temporary Internet loss.

Application availability is preserved because evaluation does not depend on the failed network path. See Offline Operation and SSE Disconnected.

Application restart during an outage

Restart behavior differs from a running-process outage because the in-memory snapshot is lost unless it can be restored from persistent storage.

Persistent cache available

The last persisted snapshot can be restored before network synchronization succeeds. See Persistent Cache and Cache Strategy.

Persistent cache unavailable

The application can still start without blocking, but only code-defined defaults are available until the first successful synchronization. Persistent cache improves restart resilience. Explicit defaults provide the final safety boundary. See Non-Blocking Startup and Cache Not Restored.

Configuration freshness vs application availability

These properties must be monitored and reasoned about separately.

  • Application availability asks: can the application continue handling requests and evaluating flags?
  • Configuration freshness asks: does the SDK have the newest configuration from SOASAP Cloud?

During an outage, application availability may remain unaffected while configuration freshness stops advancing.

Design trade-off. A stale snapshot is intentionally preferred over introducing a remote dependency into the application request path. Stale configuration is not always safe — teams must decide which flags are acceptable to freeze during extended synchronization loss.

What remains available

During a synchronization or control-plane outage, the following continue to work when a valid local snapshot exists:

  • ✓ Boolean flag evaluation — on/off reads from the frozen snapshot
  • ✓ String flag evaluation — variants and mode strings remain readable
  • ✓ Number flag evaluation — limits, timeouts, and thresholds remain readable
  • ✓ JSON flag evaluation — structured configuration remains available in memory
  • ✓ Local O(1) lookups — evaluation stays in-process with no network I/O
  • ✓ Existing environment values — previously synchronized environment state remains usable
  • ✓ Code-defined defaults — fill gaps when a key has never been synchronized
  • ✓ Application request handling — request paths are not blocked on SOASAP connectivity
  • ✓ Persistent snapshot restoration — where configured, restarts can restore last-known state

What becomes unavailable or delayed

The following depend on control-plane connectivity:

  • receiving newly created flags
  • receiving updated flag values
  • receiving deleted flag state
  • dashboard-driven kill-switch changes
  • environment configuration updates
  • synchronization-status recovery
  • first synchronization on a new installation without cache

These limitations affect changes to configuration, not reads from the existing local snapshot.

Real-time synchronization and recovery

SSE keeps snapshots current while connectivity is available. After temporary failures, the SDK should reconnect in the background. Application code should not need to recreate the SDK client or restart the process. See Real-Time Synchronization.

High availability does not mean zero risk

Local evaluation removes SOASAP Cloud from the request-time dependency chain, but it does not eliminate every production risk.

  • stale flag values may remain active during a long outage
  • a newly created kill switch may not exist in disconnected SDKs
  • an unsafe default may be used on first startup
  • cache storage may be missing or corrupted
  • application-level failures remain possible
  • incorrect flag configuration can still cause incidents

High availability must be combined with safe defaults, testing, monitoring, and operational planning. See Production Safety.

Flag design for high availability

Classify flags by their behavior during extended synchronization loss.

Release flags

Usually acceptable to remain at the last known value for a temporary period.

Kill switches

Should use a safe code-defined default. Critical kill switches should exist and be synchronized before the related feature is deployed — a kill switch that has never reached an offline instance cannot protect that instance.

Operational configuration

Review whether stale values remain safe for limits, timeouts, and capacity controls.

External integration controls

Should fail safely when external dependencies are unavailable, independent of SOASAP sync state.

Document for important flags:

  • purpose
  • owner
  • safe default
  • acceptable stale duration
  • rollback behavior
  • removal date where applicable

Multi-instance and distributed applications

Each application instance maintains its own local snapshot and synchronization connection. During partial network failure, different instances may temporarily hold snapshots from different synchronization points.

This is eventual convergence, not request-time consensus. After connectivity is restored, instances synchronize toward the latest available state. SOASAP should not be used as a distributed locking or strong-consistency system. See Scaling.

Regional and partial failures

Partial-failure scenarios include:

  • one application instance loses connectivity
  • one region loses outbound Internet access
  • one cluster cannot resolve the SOASAP endpoint
  • the control plane is temporarily unavailable
  • SSE is interrupted while other application traffic remains healthy

Because each SDK evaluates independently, unaffected instances continue using their own snapshots. Affected instances continue from their last known state where possible. Test partial failure — not only full outages.

Monitoring

Application availability and synchronization health should be monitored separately.

Recommended signals:

  • application request success rate
  • application readiness and liveness
  • SDK synchronization status
  • time since last successful synchronization
  • reconnect attempts
  • snapshot availability
  • cache restoration failures
  • cache write failures
  • evaluations that fall back to defaults
  • configuration version or snapshot age, where exposed
Do not couple liveness to SSE. An SSE disconnection should usually be treated as degraded configuration freshness, not immediate application failure.

Health checks

Separate health signals by operational question.

Liveness

Answers whether the application process is functioning. SOASAP connectivity should not normally determine liveness.

Readiness

Answers whether the application can safely serve traffic. A temporary synchronization interruption should not automatically make the application unready when a valid snapshot or safe defaults are available.

Dependency or diagnostic health

Expose synchronization status separately for operators. Possible statuses:

  • synchronized
  • reconnecting
  • offline with snapshot
  • offline without snapshot
  • cache restore failed

Exact integrations depend on the SDK and application platform.

Failure-scenario table

Scenario Evaluation Behavior Configuration Updates Application Restart Required Operator Action
SOASAP Cloud outage Continues from last in-memory snapshot Paused until service recovery No Monitor sync age; prepare dashboard changes for reconnect
Temporary SSE disconnect Continues from last snapshot Delayed until reconnect No Treat as freshness degradation; verify auto-reconnect
Application network loss Continues locally while process remains up Unavailable on the affected path No Restore outbound connectivity; do not fail app health on sync alone
DNS failure Continues from last snapshot Paused while resolution fails No Fix DNS/resolvers; confirm endpoint resolution after recovery
Process restart with cache Restores persisted snapshot, then evaluates locally Background reconnect after start N/A (restart already occurred) Verify cache path and restore metrics
Process restart without cache Uses code-defined defaults until first sync Blocked until connectivity succeeds N/A Confirm safe defaults; restore network or enable cache
Corrupted cache Falls back to defaults until successful sync Requires successful network sync No (unless process already restarting) Investigate storage integrity; see Cache Not Restored
Cache write failure In-memory evaluation unaffected while process lives Sync may continue; restart resilience reduced No Alert on write failures; fix disk/permissions before next restart
Invalid API key Uses existing snapshot if present; otherwise defaults Authentication prevents refresh No Rotate/replace key; see Invalid API Key
Newly deployed application with no previous snapshot Defaults only until first successful sync Depends on first connect No Ensure safe defaults and non-blocking startup; verify first sync

Testing high availability

Validate failure behavior in staging before production incidents expose gaps.

Test 1: Disconnect SSE

Verify:

  • evaluations continue
  • existing values remain stable
  • reconnect occurs automatically

Test 2: Block outbound traffic

Verify:

  • request handling continues
  • the SDK reports degraded synchronization
  • no request-time network errors are introduced

Test 3: Restart with network blocked

Verify:

  • cached snapshot is restored
  • flags remain available
  • the application becomes ready without waiting for SOASAP Cloud

Test 4: Restart without cache

Verify:

  • explicit defaults are used
  • startup does not block indefinitely
  • synchronization succeeds after connectivity returns

Test 5: Partial instance failure

Verify:

  • one disconnected instance remains operational
  • connected instances continue receiving updates
  • snapshots converge after recovery

Common misconceptions

"High availability means every SDK always has the latest value."

False. Availability and freshness are separate properties.

"SOASAP Cloud is called during every evaluation."

False. Evaluation uses the local in-memory snapshot.

"An SSE disconnection should fail application health checks."

Usually false. Treat it as a synchronization-health signal unless the application has a specific requirement for fresh configuration.

"Persistent cache guarantees current configuration."

False. It restores the most recently persisted snapshot.

"Defaults replace all flags during an outage."

False. Existing snapshot values are used first. Defaults apply when no valid synchronized value exists.

"Local evaluation provides strong consistency across all instances."

False. Instances converge through synchronization and may temporarily hold different snapshot versions during partial failures.

Production recommendations

  • ✓ Use explicit defaults for every evaluation — defaults provide deterministic behavior when no snapshot value exists.
  • ✓ Enable persistent cache where supported — cache restoration improves resilience across process and host restarts.
  • ✓ Do not place synchronization in the request path — request handling should remain independent of control-plane connectivity.
  • ✓ Monitor synchronization separately from application uptime — distinguish degraded freshness from application failure.
  • ✓ Test network and control-plane outages in staging — validate failure behavior before production incidents.
  • ✓ Document acceptable stale behavior for critical flags — not every flag is safe to freeze indefinitely.
  • ✓ Create kill switches before deploying dependent functionality — a kill switch that has never synchronized cannot protect an offline instance.
  • ✓ Use separate API keys and environments — environment isolation reduces blast radius.
  • ✓ Avoid using feature flags for strong-consistency coordination — feature flags are not locks, leader-election signals, or transaction primitives.

Relationship to other SOASAP concepts

Concept Relationship to high availability
Local Evaluation Keeps SOASAP Cloud outside the request-time evaluation path
Real-Time Synchronization Keeps local snapshots current while connectivity is available
Persistent Cache Allows snapshots to survive process restarts
Offline Operation Defines runtime behavior while synchronization is unavailable
Non-Blocking Startup Allows applications to become ready without waiting for the first network round-trip
Default Values Provide deterministic behavior when no valid snapshot value exists
Production Safety Defines safe defaults, kill switches, and environment-isolation practices

Related documentation