Scaling

Introduction

SOASAP SDKs evaluate flags locally from an in-memory snapshot. Evaluation cost remains independent of network latency and does not increase with the number of application replicas.

As an application grows, scaling considerations move primarily to:

• the number of flags
• the size of flag values
• the number of projects and environments
• snapshot memory per SDK instance
• the number of application replicas
• synchronization connections
• cache storage per instance
• flag ownership and lifecycle management

Scaling model

Separate runtime scale from configuration scale. High request volume and high flag count are related but distinct.

Runtime scale

Concerns:

• application request volume
• number of SDK instances
• evaluations per second
• snapshot memory
• synchronization connections
• startup and cache behavior

Configuration scale

Concerns:

• number of flags
• number of projects
• number of environments
• payload size
• flag ownership
• stale flags
• naming and organization

A service may perform millions of evaluations against a small snapshot. Another service may evaluate infrequently but hold a large configuration snapshot.

Local evaluation complexity

Each flag read is resolved from the SDK’s in-memory snapshot.

Flag lookup remains O(1) under the SDK’s key-based evaluation model. That means:

• no scan through all flags for each read
• no request-time call to SOASAP Cloud
• no dependency on dashboard latency
• no per-read disk access
• predictable lookup behavior as request volume increases

O(1) describes lookup complexity. It does not mean evaluation has zero cost or that all value types have identical processing and allocation characteristics.

Request volume

Because evaluation is local, application request volume does not produce an equivalent volume of remote SOASAP requests.

The same in-memory snapshot can be read repeatedly by the application process. Runtime cost depends on:

• number of evaluations per request
• value type
• application-language allocations
• JSON deserialization behavior
• logging or telemetry added by the application
• concurrency characteristics of the SDK and runtime

Avoid repeatedly evaluating the same flag inside tight loops when one evaluation can be reused for the logical operation.

Horizontal application scaling

Each SDK instance maintains its own in-memory snapshot, synchronization lifecycle, reconnect state, and persistent cache where configured.

Adding replicas increases aggregate snapshot memory, synchronization connections, cache files or volumes, startup synchronization activity, and observability cardinality. It does not introduce shared request-time contention between SDK instances. Each replica evaluates independently.

Memory per instance

Each SDK process holds a local representation of the environment snapshot. Memory usage generally grows with:

• number of flags
• length of flag keys
• size of string values
• size and complexity of JSON values
• snapshot metadata
• runtime object overhead
• temporary allocations during snapshot replacement
• application-level copies or deserialization

Conceptual fleet memory:

# Approximate Fleet Memory
Memory Per Snapshot × Number of SDK Instances

There is no universal byte-per-flag estimate. Exact usage varies by SDK language, runtime, object representation, flag type, JSON structure, SDK version, garbage collector, and process architecture. Measure representative snapshots in the actual production runtime.

Snapshot replacement and temporary memory

When a new snapshot arrives, an SDK may temporarily hold the current snapshot, the incoming snapshot, parsing or validation structures, serialization buffers, and runtime allocations. Peak memory during synchronization may therefore be higher than steady-state snapshot memory.

Recommend:

• leave headroom above observed steady-state usage
• test large configuration updates
• observe memory during startup and synchronization
• avoid setting container memory limits exactly at idle usage
• investigate repeated allocation spikes or garbage collection pressure

Flag count

A larger flag inventory increases snapshot size and operational complexity. Technical effects may include:

• additional memory per SDK instance
• longer snapshot parsing or replacement
• larger persistent cache files
• more dashboard inventory to manage
• increased chance of duplicate or ambiguous flag purposes
• greater difficulty identifying obsolete flags

Flag inventory governance

As flag count grows, define a lifecycle for every flag. Recommended metadata or internal documentation:

• owner
• purpose
• project
• environment
• creation date
• expected removal date
• safe default
• rollout status
• incident relevance
• related service or feature

Temporary release flags should not automatically become permanent configuration. Stale flags increase cognitive load, snapshot size, test combinations, incident ambiguity, code complexity, and risk of contradictory behavior.

Removing obsolete flags

Use a safe removal workflow:

1. Identify the flag owner.
2. Confirm the rollout is complete.
3. Search all supported codebases for the flag key.
4. Remove conditional branches or make the final behavior permanent.
5. Deploy the code change.
6. Confirm older application versions no longer depend on the flag.
7. Delete the flag from SOASAP.
8. Verify no missing-key fallback alerts appear.

Deleting the dashboard flag before removing application references may cause old deployments to fall back to their code-defined defaults. Account for rolling deployments, mobile application versions, desktop clients, long-lived workers, delayed jobs, and rollback versions.

JSON flag size

JSON flags allow structured configuration but can dominate snapshot size when used for large documents.

Appropriate JSON flag examples:

• small UI configuration
• feature parameters
• bounded lists
• compact structured options
• rollout-specific settings

Poor JSON flag examples:

• multi-megabyte documents
• large catalogs
• localization databases
• complete authorization policies
• binary data encoded as text
• frequently changing bulk datasets
• content-management payloads

Large JSON values are risky because they create:

• higher memory per replica
• larger cache files
• more parsing work
• greater synchronization payloads
• longer garbage-collection cycles
• accidental copying by application code
• slower startup restoration
• higher blast radius when the value is malformed

JSON on hot paths

A JSON flag evaluated on a hot request path may introduce application-side costs beyond the key lookup. Possible costs include deserialization, object allocation, validation, copying, and traversal of large structures.

Recommend:

• keep JSON payloads small
• deserialize once when supported by the application design
• avoid repeated conversions inside loops
• validate expected schema
• provide a safe default object
• measure allocations in the target runtime
• consider Boolean, String, or Number flags when the value is scalar

The flag lookup may remain O(1), while application processing of the returned JSON still depends on payload size.

Flag type selection

Select the simplest type that represents the required behavior.

Boolean

Best for feature enablement, kill switches, and binary behavior selection. Usually the smallest and simplest value type. See Boolean Flags.

String

Best for compact modes, strategy names, identifiers, and small textual configuration. Avoid large documents. See String Flags.

Number

Best for thresholds, limits, percentages, and bounded numeric tuning. Validate ranges in application code. See Number Flags.

JSON

Best for small structured configuration and grouped related values. Use carefully because size and processing cost can vary significantly.

Project boundaries

Projects provide an organizational and configuration boundary.

A single project may be appropriate when:

• flags belong to one product or service family
• the same teams govern the inventory
• environments follow the same lifecycle
• snapshot size remains operationally reasonable

Consider separate projects when:

• independent products have unrelated flags
• different teams own configuration
• deployment lifecycles differ
• security boundaries differ
• flag inventory becomes difficult to navigate
• services do not need the same configuration surface
• one project’s snapshot grows because it includes unrelated domains

Do not split projects solely to optimize a small number of lookups. Use project boundaries primarily for ownership, isolation, maintainability, and relevant snapshot scope.

Project splitting trade-offs

Benefits:

• smaller and more relevant flag inventories
• clearer ownership
• reduced accidental coupling
• independent environments
• easier access management
• smaller per-project configuration scope

Costs:

• more API keys
• more SDK configuration
• additional operational ownership
• cross-project coordination
• more dashboard navigation
• potentially multiple clients in one application

An application requiring flags from multiple projects may need multiple SDK clients, depending on the SDK and supported configuration model. Confirm multi-project setup in the platform documentation.

Environment scale

Each environment has its own values and API key boundary. Common environments include Development, Testing, Staging, and Production.

As environment count grows, consider:

• naming consistency
• ownership
• API key management
• cache isolation
• deployment configuration
• cleanup of temporary environments
• validation before production changes

Avoid creating many long-lived environments without lifecycle ownership. Temporary preview environments should have a defined creation and cleanup process.

Synchronization connections

Each active SDK instance may maintain its own synchronization connection. Fleet connection count generally grows with the number of replicas, services, SDK clients per process, environments, and projects used by each application.

Recommend:

• reuse one SDK client per project and environment within a process where supported
• avoid creating a new client per request
• avoid repeatedly initializing and disposing SDK clients
• monitor connection churn
• use the SDK’s documented dependency-injection or singleton lifecycle
• distinguish expected autoscaling churn from reconnect storms

SDK lifecycle design is important when scaling replicas. See Real-Time Synchronization.

Client lifetime

A SOASAP client should normally be long-lived.

Anti-pattern:

Per-request clients create repeated connections, repeated initialization, unnecessary memory allocation, loss of snapshot continuity, ineffective persistent cache use, synchronization churn, and slower evaluations caused by application design. Follow SDK-specific lifecycle guidance.

Autoscaling events

Considerations during rapid horizontal scaling:

• many instances may start simultaneously
• each instance needs snapshot memory
• cache may or may not be present
• synchronization connections increase
• instances may briefly use different snapshot versions
• explicit defaults protect first-start scenarios

Test scale-out while SOASAP Cloud is reachable, synchronization is delayed, network access is blocked, cache is available, and cache is unavailable. See Non-Blocking Startup.

Deployment scale

Large rolling deployments can create temporary overlap between old and new application versions. During this period:

• both versions may evaluate the same flags
• code-defined defaults may differ
• old versions may depend on flags removed by new versions
• snapshot versions may update at different times
• aggregate SDK instance count temporarily increases

Recommend:

• maintain backward-compatible flag semantics during deployment
• do not delete flags until old instances are gone
• avoid changing a flag’s meaning in place
• plan defaults across both application versions
• monitor synchronization on new replicas

Mobile, web, and client-side fleets

Client-side fleets scale differently from server-side replicas. Characteristics may include many independent SDK instances, intermittent connectivity, long-lived old application versions, delayed upgrades, limited device memory, background suspension, and variable cache durability.

Recommend:

• keep client snapshots compact
• avoid large JSON values
• maintain backward-compatible flag keys
• preserve safe defaults in older versions
• avoid deleting flags immediately after a rollout
• consider the maximum supported application version lifetime
• test low-memory and offline devices

Serverless and short-lived instances

Serverless platforms may create many short-lived instances. Scaling considerations include frequent initialization, limited cache durability, repeated synchronization startup, high instance churn, memory limits, and uncertain reuse of local storage.

Recommend:

• initialize the SDK outside individual request handlers where the platform allows reuse
• enable preload where supported
• use explicit defaults
• keep snapshots compact
• avoid assuming local cache survives instance replacement
• test cold starts under concurrency
• follow SDK-specific serverless guidance

Multi-tenant applications

Do not automatically create one SOASAP SDK client or project per application tenant. Potential consequences include excessive client count, excessive synchronization connections, higher memory usage, complex key and cache management, and difficult operational ownership.

Evaluate whether tenant-specific behavior belongs in application data, authorization rules, a targeting system, separate product configuration, or genuinely isolated SOASAP projects. SOASAP feature flags should not be used as a general-purpose tenant database.

Capacity planning

Step 1: Measure a representative snapshot

Use production-like flag count, key lengths, value types, JSON sizes, and environment configuration.

Step 2: Measure one SDK instance

Observe:

• idle memory before SDK initialization
• memory after snapshot load
• peak memory during synchronization
• memory after garbage collection
• cache file size
• startup duration

Step 3: Multiply across the fleet

Consider:

• steady-state replica count
• maximum autoscaled replica count
• rolling deployment overlap
• multiple SDK clients per process
• multiple projects or environments

Step 4: Add operational headroom

Allow for application workload, runtime overhead, synchronization replacement, traffic spikes, diagnostics, garbage collection, and future flag growth.

Step 5: Repeat after significant growth

Re-measure after major flag-count increases, large JSON additions, SDK upgrades, runtime upgrades, project restructuring, and deployment-platform changes.

Conceptual capacity formulas

These formulas are planning approximations. Actual resource use must be measured in the target runtime and deployment model.

Fleet Snapshot Memory
≈ Snapshot Memory Per Client
  × Clients Per Process
  × Process Count

Aggregate Cache Storage
≈ Cache Size Per Client
  × Persisted Instance Count

Synchronization Connections
≈ Long-Lived SDK Clients
  × Active Instances

Scaling limits and warning signals

Signals that the current organization or snapshot design needs review:

• flag inventory is difficult to navigate
• no clear flag ownership exists
• obsolete flags accumulate
• JSON values grow continuously
• SDK memory becomes material relative to the process limit
• synchronization causes visible allocation spikes
• cache restoration becomes slower
• container out-of-memory events appear during snapshot updates
• teams use flags as a general configuration database
• one project contains unrelated products
• many SDK clients are created in one process
• connection churn grows with request traffic
• mobile clients receive configuration they never use

No single signal automatically requires project splitting. Use measurements and ownership boundaries together.

Performance testing

Validate scaling assumptions in staging before production growth.

Test 1: Large flag inventory

Create a production-like number of flags. Verify snapshot memory, initialization behavior, synchronization behavior, cache size, and evaluation latency.

Test 2: Large JSON value

Introduce a representative large structured value. Verify memory growth, parsing cost, application deserialization cost, garbage collection, and cache restoration.

Test 3: High evaluation rate

Exercise realistic request traffic. Verify CPU use, allocation rate, application latency, and absence of network calls per evaluation.

Test 4: Rapid scale-out

Start many replicas. Verify startup readiness, connection count, snapshot convergence, per-replica memory, and behavior with and without cache.

Test 5: Rolling deployment

Run old and new versions simultaneously. Verify flag compatibility, defaults, snapshot memory during replica overlap, and safe removal timing.

Test 6: Synchronization update under load

Change several flags while the application is busy. Verify that request handling remains stable, memory peak remains acceptable, snapshot replacement succeeds, and all replicas converge.

Observability

Recommend monitoring:

• SDK instance count
• active synchronization connections
• reconnect rate
• time since last synchronization
• snapshot version where exposed
• flag count per project and environment
• cache file size
• process memory
• container working set
• memory-limit utilization
• garbage-collection pressure
• process restarts
• out-of-memory terminations
• startup duration
• cache restoration duration where measurable
• frequency of default fallbacks
• large JSON value growth
• obsolete flag inventory

Not every metric needs to come directly from the SDK. Combine application, runtime, container, filesystem, and SOASAP operational signals.

Alerting considerations

Memory pressure

Alert when SDK and application growth approach safe process or container limits.

Connection churn

Alert when repeated client creation or reconnect storms produce abnormal synchronization behavior.

Snapshot staleness

Alert when instances remain disconnected beyond the application’s acceptable stale duration.

Flag inventory growth

Use governance reporting rather than an urgent runtime page in most cases.

Oversized configuration

Review unusually large JSON values or sudden cache-size growth.

Default fallback increase

Investigate missing keys, new deployments, cache failures, or synchronization problems.

Exact thresholds depend on workload and deployment architecture.

Failure-scenario table

Common anti-patterns

Client per request

Creates unnecessary initialization and synchronization churn.

One large JSON document

Turns a feature flag into a bulk data-delivery mechanism.

Permanent temporary flags

Increases inventory and code complexity.

Unrelated products in one project

Creates unclear ownership and oversized configuration scope.

Flag key reuse

Changing the semantic meaning of an existing key creates version-compatibility risks.

Immediate deletion after rollout

Can break old instances, mobile versions, or rollback deployments.

Shared mutable application state derived repeatedly

Repeatedly parsing or copying large values adds avoidable hot-path cost.

Assuming O(1) means zero memory growth

Lookup complexity and snapshot capacity are different concerns.

Common misconceptions

"More application traffic creates more SOASAP Cloud evaluation requests."

False. Evaluation is local and does not call SOASAP Cloud per read.

"O(1) evaluation means flag count has no resource impact."

False. Flag count affects snapshot memory, cache size, synchronization, and governance.

"All flag types have the same memory and processing cost."

False. Value size and runtime representation matter, especially for JSON.

"Every replica shares one central SDK snapshot."

False. Each SDK instance holds its own local snapshot.

"More replicas reduce memory used by SOASAP."

False. Aggregate snapshot memory generally increases with the number of SDK instances.

"A feature flag is a suitable place for any configuration document."

False. Large datasets should use systems designed for bulk data and configuration delivery.

"Project splitting is required whenever flag count increases."

False. Use ownership, isolation, relevance, and measured resource impact to determine project boundaries.

"Deleting an obsolete flag from the dashboard is the first cleanup step."

False. Remove code dependencies and account for older deployments first.

Production recommendations

Scaling checklist

Application runtime

• one long-lived SDK client
• no per-request initialization
• representative performance tests
• memory headroom
• safe container limits
• startup tested under scale-out

Flag inventory

• clear owner
• documented purpose
• expected removal date
• obsolete flags removed
• stable key semantics
• safe defaults

Values

• simplest suitable type
• bounded string size
• validated number range
• compact JSON
• no bulk datasets
• no binary content

Deployment fleet

• memory calculated per replica
• maximum replica count considered
• rolling overlap considered
• client-side version lifetime considered
• synchronization connections monitored
• partial convergence understood

Organization

• projects aligned with product boundaries
• environments have lifecycle ownership
• API keys scoped correctly
• temporary projects and environments cleaned up

Relationship to other SOASAP concepts

Related documentation

• Cache Strategy — per-instance cache storage and volume planning
• Security — API keys, environments, and access boundaries
• High Availability — evaluation independence from the control plane
• Outages — partial connectivity and temporary snapshot differences
• Local Evaluation — O(1) in-memory evaluation model
• Real-Time Synchronization — SSE delivery to each active SDK
• Persistent Cache — snapshot restoration across restarts
• Non-Blocking Startup — readiness without waiting on first sync
• Default Values — code-defined fallbacks
• Boolean Flags — enablement and kill-switch patterns
• String Flags — compact textual configuration
• Number Flags — thresholds and limits
• JSON Flags — structured values and size guidance
• SDK Documentation — platform-specific lifecycle and preload
• Flag Not Updating — diagnosing propagation and convergence
• SSE Disconnected — synchronization interruptions