Multi-Instance Sync Engine
Run multiple named sync engine instances to scale horizontally, distribute workloads across regions, or isolate connectors for independent scheduling and fault domains.
Overview
By default, Identity Mesh runs a single sync engine instance named IdentityMeshEngine that processes all connectors. Multi-instance support lets you run additional named instances, each responsible for its assigned connectors.
Instances self-register automatically via a heartbeat mechanism — no manual instance creation is needed. Simply start a new engine process with a unique name and it will appear in the admin UI.
Key Concepts
Instance Name
A unique string identifier for each engine process. Also used as the Windows service name when installed.
Connector Assignment
Each connector can be assigned to a specific instance. Unassigned connectors (NULL) are processed by every instance.
Heartbeat
Each instance writes a heartbeat every sync cycle (~1 minute). Instances with no heartbeat for 3+ minutes are shown as Offline.
Per-Instance Pause
Each instance can be paused independently from the admin UI, or all instances can be paused globally.
How It Works
Self-Registration
When an engine instance starts, it registers itself in the IM_EngineInstances table with its name, machine name, version, and a timestamp. No manual setup is required.
The instance also marks any orphaned runs from a previous crash as Interrupted, scoped to its own instance name so it never interferes with runs from other instances.
Connector Filtering
Each sync cycle, the engine queries for due schedules filtered to connectors assigned to this instance or connectors with no assignment (NULL). This means:
- Connectors explicitly assigned to this instance are always processed
- Unassigned connectors are processed by any running instance
- Connectors assigned to a different instance are never processed
Heartbeat & Status
On each sync loop, the instance updates its heartbeat timestamp. The admin API derives the effective status:
| Status | Meaning |
|---|---|
| Online | Heartbeat within last 3 minutes, not paused |
| Paused | Instance or global pause is active; engine is running but skipping sync cycles |
| Offline | No heartbeat for 3+ minutes, or the instance shut down gracefully |
Graceful Shutdown
When an instance stops (service stopped, Ctrl+C in console mode), it marks itself as Offline in the database. Any in-progress sync runs are marked as Interrupted and will be cleaned up when the instance restarts.
Running Multiple Instances
Console Mode (Development / Testing)
Run additional instances from the command line with the --instance-name flag:
# Default instance (name: IdentityMeshEngine)
IdentityMesh.Service.exe --console
# Named instance
IdentityMesh.Service.exe --console --instance-name EngineEurope
# Another named instance
IdentityMesh.Service.exe --console --instance-name EngineAPACWindows Service (Production)
Install each instance as a separate Windows service. The service name matches the instance name:
# Install the default instance
IdentityMesh.Service.exe --install
# Install a named instance
IdentityMesh.Service.exe --install --instance-name EngineEurope
# Uninstall a named instance
IdentityMesh.Service.exe --uninstall --instance-name EngineEuropeBackward Compatible
If you don't use --instance-name, the engine defaults to IdentityMeshEngine and processes all connectors — exactly the same behavior as before multi-instance support was added.
Assigning Connectors to Instances
Connectors can be assigned from the Settings > Engine Instances tab in the admin UI, or via the REST API.
Via Admin UI
- 1 Navigate to Settings > Engine Instances
- 2 Scroll to the Unassigned Connectors table at the bottom
- 3 Use the Assign To dropdown to select a target instance
- 4 The connector will immediately appear under that instance's card
Via REST API
PUT /api/connectors/{connectorId}/instance
Content-Type: application/json
{
"instanceName": "EngineEurope"
}{
"connectorId": "a1b2c3d4-...",
"instanceName": "EngineEurope"
}
Set instanceName to null to unassign a connector, making it available to all instances.
A connector can only be assigned to one instance at a time. To move a connector to a different instance, you must unassign it first (set instanceName to null), then assign it to the new instance. Attempting to reassign directly will return 409 Conflict.
Pausing & Resuming Instances
Instances support two levels of pause control:
Global Pause
Pauses all instances. Set via Settings > System > Pause All Services or POST /api/system/pause.
Per-Instance Pause
Pauses a single instance. Set via the Pause button on the instance card or POST /api/instances/{name}/pause.
An instance is paused if either the global pause or its per-instance pause is active. Pausing takes effect mid-sync — if a sync cycle is in progress, it will be interrupted and the run marked as Paused.
API Examples
POST /api/instances/EngineEurope/pause
Response:
{
"instanceName": "EngineEurope",
"paused": true
}{
"paused": false,
"instancePauseStates": {
"EngineEurope": true,
"IdentityMeshEngine": false
}
}Renaming Instances
Instance names can be changed from the admin UI or via the REST API. Renaming cascades automatically across all related data — connector assignments, run history, and pause settings are all updated in a single transaction.
Via Admin UI
Click the Rename button on an instance card in Settings > Engine Instances, enter the new name, and confirm.
Via REST API
PUT /api/instances/{instanceId}/rename
Content-Type: application/json
{
"newName": "EngineEurope-v2"
}{
"instanceId": "b7e4f2a1-...",
"instanceName": "EngineEurope-v2"
}Restart Required
Renaming only updates the database. If the engine is still running with the old --instance-name, it will re-register under the old name on its next heartbeat. Restart the service with the new --instance-name for a permanent rename.
Instance API Reference
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/instances | List all registered instances with effective status |
| POST | /api/instances/{name}/pause | Pause a specific instance |
| POST | /api/instances/{name}/resume | Resume a paused instance |
| PUT | /api/instances/{id}/rename | Rename an instance (cascades to connectors, run history, and pause settings) |
| DELETE | /api/instances/{name} | Remove an offline instance and unassign its connectors |
| PUT | /api/connectors/{id}/instance | Assign or unassign a connector to an instance |
Common Deployment Patterns
Geographic Distribution
Run one instance per region to keep sync traffic local.
-
EngineUSon US servers → US-based AD connectors -
EngineEUon EU servers → EU-based LDAP connectors -
EngineAPACon APAC servers → APAC connectors
Workload Isolation
Separate heavy connectors from lightweight ones.
-
EngineBulk→ large AD forests (100k+ objects) -
EngineFast→ cloud connectors (Okta, Entra ID)
Fault Isolation
A failing connector on one instance won't block connectors on another.
- Pause or restart one instance without affecting others
- Isolate a problematic connector for debugging
Maintenance Windows
Pause a single instance during scheduled maintenance.
- Pause EU instance during EU maintenance window
- US and APAC instances continue syncing uninterrupted
Database Schema
Multi-instance support requires three SQL migration scripts to be run against the Identity Mesh database:
Create_IM_EngineInstances.sql
New table storing instance registrations, heartbeat timestamps, status, and version information.
Alter_IM_Connectors_AddInstanceName.sql
Adds a nullable InstanceName column to IM_Connectors for connector-to-instance assignment.
Alter_IM_RunHistory_AddInstanceName.sql
Adds a nullable InstanceName column to IM_RunHistory to track which instance executed each run.
All scripts are idempotent (use IF NOT EXISTS) and safe to run multiple times. The new columns are nullable, so existing data is unaffected.