Offline Operation
Feature flag evaluation continues using the last known snapshot when SOASAP Cloud or network connectivity becomes unavailable.
Introduction
SOASAP treats offline operation as a first-class production scenario. Applications should continue evaluating feature flags even when SOASAP Cloud is unavailable, internet connectivity is interrupted, SSE connections disconnect, or control-plane maintenance is in progress.
Flag evaluation does not depend on active connectivity to SOASAP Cloud. The SDK holds a synchronized snapshot in memory; evaluation continues against that snapshot using the same local O(1) read path as when online. Only synchronization pauses — not flag reads.
Why offline operation exists
Remote-evaluation systems require a network request on every flag read. The traditional architecture looks like this:
Coupling flag reads to live connectivity introduces predictable consequences:
- Network dependency — every request path failure in the flag service becomes an application failure
- Additional failure modes — timeouts, rate limits, DNS errors, and TLS failures surface in application code
- Reduced resilience — control-plane outages degrade or halt production traffic
- Request-path latency — flag evaluation inherits network round-trip time and tail latency variance
SOASAP avoids this architecture. Local Evaluation decouples reads from connectivity. Offline operation is the production consequence of that design: when sync stops, reads continue from the snapshot already in memory.
Offline evaluation flow
When connectivity is lost, the SDK does not discard its snapshot or switch to a different evaluation mode. The in-memory snapshot from the last successful synchronization remains available. Getters perform the same lookup against that snapshot as they did before the outage.
No runtime behavior changes on the read path. No network calls are introduced. No exceptions are thrown from getters because SSE disconnected. Application code that evaluated flags while online continues to work unchanged while offline.
What continues to work
During an outage, the following continue functioning normally:
- Boolean flags — on/off evaluation from the frozen snapshot
- String flags — variant and mode strings remain readable
- Number flags — limits, timeouts, and thresholds remain readable
- JSON flags — structured configuration objects remain deserializable from memory
- Local evaluation — O(1) in-memory lookups with no network I/O
- Persistent cache — already-persisted snapshots remain on disk for subsequent restarts during the outage
- Explicit defaults — applied for keys absent from the snapshot, same as online
- Existing snapshot data — all flag values synchronized before the outage remain active
These capabilities continue because evaluation never depended on live connectivity in the first place. Offline operation is not a special degraded mode — it is the normal local evaluation path with synchronization temporarily paused.
What does not work during an outage
New dashboard changes cannot reach disconnected SDKs. The control plane cannot push updates over SSE while the connection is down or SOASAP Cloud is unreachable.
Operations that pause until connectivity returns include:
- Enabling a feature — a flag set to
truein the dashboard does not reach offline SDKs - Disabling a feature — kill switch activations do not propagate until reconnect
- Changing configuration — string, number, and JSON value edits remain pending
- Creating new flags — new keys are absent from the frozen snapshot; getters return code defaults
- Deleting flags — removed keys may still appear in the snapshot until sync refreshes state
Updates become available after connectivity is restored and Real-Time Synchronization resumes. Propagation delay after reconnect depends on network conditions; evaluation itself is unaffected during the catch-up period.
Failure scenarios
SOASAP Cloud outage
Evaluation continues from the in-memory snapshot. SSE cannot connect or receive events; synchronization pauses. Applications serve traffic with frozen flag values. The background worker retries with exponential backoff. No application restart is required.
Internet connectivity loss
Evaluation continues. Outbound HTTPS to SOASAP Cloud fails; SSE disconnects. The snapshot in memory remains readable. Synchronization pauses until connectivity returns. Processes restarted during the outage may hydrate from Persistent Cache if a prior snapshot was persisted.
Temporary SSE disconnect
Evaluation continues without interruption. Proxy timeouts, load balancer rotation, or brief network blips drop the SSE stream. The SDK begins automatic reconnect attempts with backoff. Dashboard changes during the disconnect are not reflected until reconnect.
Extended outage
The snapshot remains frozen at the last synchronized state — potentially hours or days stale relative to the dashboard. Evaluation still continues against that snapshot. Operators cannot assume kill switches or rollbacks reached offline instances until sync resumes and confirms delivery.
Plan which flags are safe to freeze and which require operational runbooks during extended disconnects. See Production Safety.
Reconnect behavior
SDKs automatically reconnect after disconnect. No application restart is required. No manual intervention is required in application code — the background SSE worker handles connect, disconnect, and retry independently of flag evaluation.
When connectivity returns, the SDK re-establishes the SSE stream, receives a current environment snapshot or delta update, and atomically refreshes the in-memory snapshot. Subsequent getter calls return updated values. Catch-up logic ensures updates published during the disconnect are applied once the connection is restored.
Configure OnError / onError handlers
to log synchronization events without affecting reads. See
SSE Disconnected.
Relationship to other architecture components
Offline operation is the runtime resilience layer. It depends on other SOASAP architecture components but does not replace them:
| Component | Role during offline operation |
|---|---|
| Local Evaluation | Provides fast O(1) reads from memory — unchanged during outage |
| Persistent Cache | Restores last snapshot after restart during an ongoing outage |
| Real-Time Synchronization | Pauses during outage; resumes and refreshes snapshot on reconnect |
| Default Values | Fill gaps for keys never synced or missing from snapshot — not a replacement for existing values |
Persistent cache ensures restarts during an outage still have a snapshot. Local evaluation serves reads from that snapshot. Synchronization recovery applies pending dashboard changes when connectivity returns.
Default values during outages
Defaults are not automatically used for all evaluations during outages. This is a common source of confusion.
If a flag already exists in the snapshot, the snapshot value is returned — even if that value is hours stale relative to the dashboard. Defaults do not override synchronized snapshot data during offline operation.
Defaults are used only when a valid snapshot value cannot be returned:
- The flag was never synchronized to this SDK instance
- The flag key is missing from the snapshot
- A type mismatch occurs between the stored value and the getter
- JSON deserialization fails and no valid object can be produced
- No snapshot exists yet — first start with empty cache during an outage
During an extended outage, most production flags continue returning their last synchronized values. Defaults matter for new keys, first-start scenarios, and keys removed from the dashboard after the last sync. See Default Values.
Production planning
Organizations should identify which flags are safe to freeze at their last synchronized value during disconnects, and which require alternative operational controls when the dashboard cannot reach running instances.
Feature releases
Usually safe to freeze. A feature enabled before the outage remains enabled; a feature disabled before the outage remains disabled. Delayed rollouts — enabling a feature during the outage — do not reach offline instances until reconnect.
Kill switches
Require careful defaults and planning. A kill switch activated in the dashboard during an
outage does not reach disconnected SDKs. If the risky path was enabled in the snapshot
before disconnect, it remains enabled until sync resumes. Default kill switches to
false in code so never-synced instances fail safe.
Operational controls
Maintenance mode, rate limits, routing modes, and JSON policy objects should be reviewed for freeze tolerance. An outdated limit or routing string may be acceptable briefly; an outdated maintenance flag may not. Document expected behavior and fallback procedures for extended outages.
Outage planning matters because SOASAP prioritizes application availability over configuration freshness during failures. Operators must understand which dashboard actions are ineffective until connectivity returns.
Testing offline behavior
Validate offline behavior in staging before production incidents expose gaps. A practical approach: block outbound traffic to SOASAP Cloud endpoints in a staging environment while the application continues running.
Verify the following:
- Flag evaluation continues — request handlers return expected flag values without errors
- Existing values remain available — flags synchronized before the block still resolve correctly
- Reconnection occurs automatically — after unblocking traffic, SSE reconnects and dashboard changes propagate without restart
- No application errors appear — synchronization failures surface through error handlers, not through getter exceptions or HTTP 500s from flag reads
- Default values behave as expected — keys never synced return explicit code defaults, not null or thrown exceptions
Outage testing confirms that application health checks pass while sync is degraded, that kill switch defaults are safe, and that operators understand frozen-configuration limits. Repeat the test after deploy pipeline or network policy changes.
Common misconceptions
"If SOASAP Cloud is unavailable, flag evaluation stops"
False. Evaluation continues using the local in-memory snapshot. Only synchronization and dashboard-to-SDK updates pause.
"Defaults replace all values during outages"
False. Existing snapshot values remain active. Defaults apply only when no valid synchronized value exists for the requested key.
"Applications must restart after connectivity returns"
False. SDKs reconnect automatically in the background. The in-memory snapshot updates atomically when new SSE events arrive — no redeploy or process restart required.
"Offline mode changes evaluation latency"
False. Evaluation continues using the same local O(1) memory path. Offline operation does not introduce slower reads, retries, or fallback network calls.
"HTTP health checks indicate configuration freshness"
False. An application can pass health checks while serving a stale snapshot. Monitor synchronization health separately from request availability.
Production recommendations
- Always provide explicit defaults — cover first-start, never-synced keys, and type mismatch cases; defaults are the safety net when snapshot data is absent, not when it is stale.
- Test outage scenarios — block outbound SOASAP traffic in staging; validate evaluation, reconnect, and kill switch behavior before production incidents.
- Monitor synchronization health — alert on prolonged SSE disconnect, repeated
OnErrorcallbacks, and snapshot age; do not infer sync status from application HTTP metrics alone. - Identify flags that are safe to freeze — document which flags can remain at last-sync values during extended outages and which require non-SOASAP operational fallbacks.
- Keep evaluation and synchronization concerns separate — do not fail requests when sync is down; do not block handlers waiting for reconnect.
- Enable persistent cache — ensure restarts during an ongoing outage restore the last snapshot from disk rather than starting with empty state and defaults only.
Related documentation
- Local Evaluation — O(1) flag reads decoupled from network connectivity
- Real-Time Synchronization — SSE delivery that pauses during outage and resumes on reconnect
- Persistent Cache — snapshot restoration across restarts during ongoing outages
- Default Values — when defaults apply versus snapshot values during offline operation
- Production Safety — kill switches, defaults, and environment isolation
- SDK Documentation — platform-specific offline behavior and error handlers