Microsoft Sentinel direct ingestion

Operator runbook for shipping IdentityMesh log events directly to a Log Analytics workspace — the data plane Microsoft Sentinel queries via KQL. This sidesteps the Azure Monitor Agent (AMA) + Data Collection Rule (DCR) hop described in siem-integration.md for deployments that want fewer moving parts on the host.

When to choose direct ingestion

Pick direct ingestion when:

• The IdentityMesh hosts are not already managed by AMA / Defender for Cloud / Azure Arc and you don’t want a new agent installed.
• You want a single shared workspace across many hosts and the per-host AMA enrolment is overhead you’d rather avoid.
• You’re consolidating to Sentinel as the SIEM of record and want IdentityMesh logs alongside the Azure platform logs in the same workspace.
• The on-prem hosts have outbound HTTPS to *.ods.opinsights.azure.com / *.ingest.monitor.azure.com (the Logs Ingestion endpoint).

Pick AMA + DCR (the siem-integration.md recipe) instead when:

• Defender for Cloud / Azure Arc is already onboarding the hosts — DCRs centralise the collection config.
• You need performance counters or ETW alongside the application log — AMA picks those up natively.
• Compliance pins you to AMA’s chain-of-custody.

Pick the syslog forwarder (siem-syslog.md) into a Linux Sentinel forwarder when neither outbound HTTPS nor AMA is permitted off the IdentityMesh host.

Two API paths — modern vs legacy

Microsoft has two ingestion APIs for Log Analytics workspaces:

1. Logs Ingestion API (modern, current). Authenticated via Microsoft Entra (managed identity / service principal), schema-bound to a DCR + custom table, supports stream-based transformations. The Microsoft-recommended path.
2. HTTP Data Collector API (legacy). Authenticated via the workspace shared key, posts to <workspaceId>.ods.opinsights.azure.com, creates a *_CL custom table on first write. Deprecated by Microsoft in 2024 with a transition window through 14 September 2026, after which new posts will be rejected.

The Serilog sink ecosystem at the time of writing ships Serilog.Sinks.AzureAnalytics (mihaichivu), which targets the legacy HTTP Data Collector API. The example config below uses that path because it works today and lets a customer get running with three config values. The runbook flags the deprecation timeline so operators can plan the migration.

A future revision of this sample will switch to the Logs Ingestion API once a stable Serilog sink is published. In the meantime, customers who must use the modern API today should:

• Configure a DCR + custom table (Custom-IdentityMesh_CL) manually.
• Use AMA’s file-tailing input pointed at <install dir>\logs\identitymesh-*.log (the default rolling file sink stays on by default — no IdentityMesh-side change required).
• Or stand up an Azure Function / Logic App in front of the webhook (siem-webhook.md) that re-emits via the Logs Ingestion API.

The remainder of this document assumes the legacy HTTP Data Collector path unless explicitly noted.

Workspace setup (legacy HTTP Data Collector)

You need three values to configure the sink. All three come from the workspace’s “Agents” page in the Azure portal:

The first POST creates the IdentityMeshLogs_CL table in the workspace and provisions the columns from the JSON shape — no upfront schema management required. Indexing kicks in within a few minutes.

Sample config

The Sync Engine ships with a sample at:

<install dir>\appsettings.Sentinel.example.json

Copy the Serilog.Using array entry and the WriteTo block into the live appsettings.json (or, preferred for a host-specific override, appsettings.Production.json). Restart the IdentityMesh service.

{
  "Serilog": {
    "Using": [
      "Serilog.Sinks.Console",
      "Serilog.Sinks.File",
      "Serilog.Sinks.EventLog",
      "Serilog.Sinks.AzureAnalytics"
    ],
    "WriteTo": [
      {
        "Name": "AzureAnalytics",
        "Args": {
          "workspaceId": "<log-analytics-workspace-guid>",
          "authenticationId": "<base64-shared-key>",
          "logName": "IdentityMeshLogs",
          "restrictedToMinimumLevel": "Information",
          "batchSize": 100
        }
      }
    ]
  }
}

Key knobs:

Authentication options

Workspace shared key (legacy)

Simplest. The key is embedded in appsettings.json. Treat as a secret — back it with Azure Key Vault per secrets-keyvault.md rather than shipping plaintext, and rotate via the workspace’s “Regenerate key” button.

The shared key authenticates the workspace, not the identity. Anyone with the key can write any custom log to the workspace. Production deployments should plan migration to managed identity (modern API) before the September 2026 cutoff.

Managed identity (modern API path)

Requires the Logs Ingestion API. The IdentityMesh host runs under an Entra-attached identity (system-assigned managed identity if Azure-hosted, service principal + cert if hybrid / on-prem) granted the Monitoring Metrics Publisher role on the DCR. The sink pulls the bearer token via DefaultAzureCredential.

This path requires a Serilog sink targeting the modern API — not yet shipped in this release. See “Modern API migration” below.

Service principal (hybrid)

A registered Entra application with a client cert, granted Monitoring Metrics Publisher on the DCR. Use this when the host is on-prem (no managed identity available) but you want to avoid shared keys.

Modern API migration (planned)

When IdentityMesh ships a sink targeting the Logs Ingestion API, the migration shape will be:

1. Create a Data Collection Endpoint (DCE) in the same region as the workspace.
2. Define a custom table schema for IdentityMeshLogs_CL — the JSON column shape Serilog emits.
3. Author a Data Collection Rule (DCR) tied to the DCE, referencing the custom table.
4. Grant the IdentityMesh identity (managed identity or service principal) Monitoring Metrics Publisher on the DCR.
5. Swap the appsettings.Sentinel.example.json block for the modern variant (DCE endpoint URI + DCR immutable ID + table stream name).
6. The legacy Serilog.Sinks.AzureAnalytics block can stay in place during the cutover window — both will write to the same *_CL table.

The September 2026 deprecation date is firm; plan the migration window in advance.

Sample KQL queries

Once events arrive in IdentityMeshLogs_CL, the JSON shape Serilog emits surfaces as <key>_s (string), <key>_d (double), <key>_b (bool) columns. Examples:

// Recent admin-audit rows (last 24 h)
IdentityMeshLogs_CL
| where TimeGenerated > ago(1d)
| where SourceContext_s endswith "SqlAdminAuditStore"
| project TimeGenerated, ActorUpn_s, Action_s, TargetKind_s, TargetId_s, StatusCode_d
| order by TimeGenerated desc

// Privilege escalation — role mutations grouped by actor
IdentityMeshLogs_CL
| where TimeGenerated > ago(7d)
| where Action_s startswith "POST /api/roles" or Action_s startswith "PUT /api/roles"
| summarize Edits = count(), First = min(TimeGenerated), Last = max(TimeGenerated)
        by ActorUpn_s
| order by Edits desc

// Connector deletes — high-blast-radius operations
IdentityMeshLogs_CL
| where Action_s == "DELETE /api/connectors/{id}"
| project TimeGenerated, ActorUpn_s, TargetId_s, StatusCode_d, MachineName_s
| order by TimeGenerated desc

// Off-hours admin activity
IdentityMeshLogs_CL
| where Action_s startswith "POST /api/" or Action_s startswith "PUT /api/" or Action_s startswith "DELETE /api/"
| where dayofweek(TimeGenerated) in (0d, 6d)
        or hourofday(TimeGenerated) !between (7 .. 19)
| project TimeGenerated, ActorUpn_s, Action_s, TargetKind_s, TargetId_s
| order by TimeGenerated desc

// Sync engine run failures over time
IdentityMeshLogs_CL
| where SourceContext_s contains "IdentityMesh.Engine"
| where Level_s == "Error"
| summarize Failures = count() by bin(TimeGenerated, 1h), MachineName_s
| render timechart

Failure modes

• Workspace unreachable at boot. The sink queues events in-memory while it retries the connection asynchronously. The default queue is bounded; sustained outages drop oldest events first.
• Workspace unreachable mid-stream. Same pattern. The rolling file sink remains the durable record — treat the workspace as the live tap, not the system of record.
• Shared key rotated without config update. The Logs Ingestion endpoint returns 403; the sink logs via Serilog SelfLog and drops the batch. Watch for ingest-lag alerting on the workspace side.
• Workspace ingestion quota exceeded. The endpoint returns 429. The sink retries with backoff; eventually drops on sustained throttle. Right-size the workspace pricing tier per the cost note below.

To surface the sink’s own diagnostics, enable Serilog SelfLog once at startup or temporarily point SELF_LOG at a writable file.

Cost considerations

Log Analytics ingestion is billed per GB. Watch for:

• Pay-as-you-go vs Commitment Tier. At sustained > 100 GB / day the commitment tiers (per-100 GB) cut the bill by half.
• Basic Logs vs Analytics Logs. Tables defined as Basic Logs ingest at ~25% the price of Analytics Logs but lose cross-workspace query, full KQL, alerting, and 30-day retention; useful for high-volume INFO categories where you only need search-by-known-value.
• Retention. Default 30 days; keep IM_AdminAudit rows durably in SQL (per audit-retention.md) and use the workspace as the SIEM-side index, not the system of record.
• High-volume INFO. The sink ships every event above the configured restrictedToMinimumLevel. Engine cycles emit on the order of 50-200k object-audit log lines per day per 100k-object connector. At ~1 KB / event that’s roughly 50-200 MB / day. Set the level to Warning if cost is primary and use the SQL audit tables as the detail feed.

Operational notes

• Time zones. TimeGenerated is set workspace-side at ingest time; the IdentityMesh @t field is the original emission UTC. They will differ by the queue + transport delay. Pivot on @t for forensic timing.
• PII. ActorUpn and IM_ObjectAudit.ChangeJson may carry identifying data. Use the workspace’s customer-managed-key encryption tier and / or strip sensitive properties via a Serilog enricher before they hit the sink.
• Combine with file sink. The default rolling file sink remains active alongside Sentinel. The file is the durable record; the workspace is the SIEM index.

Configurable fields cheat-sheet

Related

• siem-integration.md — broader SIEM field map; AMA + DCR alternative for Sentinel.
• siem-syslog.md — syslog forwarder path for Sentinel via a Linux forwarder VM.
• siem-webhook.md — generic HTTPS webhook for non-Sentinel SOAR / collector targets.
• secrets-keyvault.md — back the workspace shared key with Key Vault rather than plaintext.
• audit-retention.md — durable audit evidence channel that the Sentinel feed complements.
• Microsoft Learn -> “Logs Ingestion API in Azure Monitor” - modern-API authoritative reference.
• Microsoft Learn -> “HTTP Data Collector API” - legacy-API reference + deprecation timeline.