VirtualMetric DataStream + Google SecOps Integration: Pre-Ingest UDM Normalization at Scale

Google SecOps (formerly Chronicle) is widely used for large-scale security analytics, long-term telemetry retention, and detection across diverse environments. Its Unified Data Model (UDM) enables correlation across sources and supports analytics that operate over long time horizons.

To take full advantage of these capabilities, security data must arrive in a consistent and well-structured UDM format.

In practice, this is rarely the case.

Security telemetry arrives in dozens of vendor-specific formats, including syslog, CEF, LEEF, Windows events, firewall logs, and cloud-native signals. And Google SecOps must identify those log types, parse events, map fields, and normalize data into UDM after ingestion.

VirtualMetric DataStream now integrates directly with Google SecOps, adding pre-ingest UDM normalization, schema validation, and ingestion automation. This allows teams to deliver consistent, validated UDM events into Google SecOps while retaining control over structure, routing, and data quality.

The challenges teams face with Google SecOps today

Google SecOps provides powerful analytics capabilities, but its default ingestion model places most normalization responsibility inside the platform. Over time, this creates operational challenges that many SOC teams recognize.

Parser-based normalization relies on heuristic detection and pattern matching. Parser logic evolves as vendors change formats and as Google updates coverage. When this happens, fields can change, meaning, appear or disappear. These changes are difficult to detect early and often surface only after detections or analytics behave differently.

Ingestion reliability is also tied to parsing success. When raw logs are ingested, Google SecOps must classify the log type, apply the appropriate parser, validate the mapping, and then normalize the event into UDM. In high-volume environments or with custom and noisy logs, this increases latency and raises the likelihood of partial ingestion or parser failures.

Visibility into schema issues is limited. Google SecOps does not expose detailed feedback on schema mismatches, parser drift, or field-level validation failures. Teams often discover problems indirectly, through missed detections or inconsistent query results.

Many organizations also operate more than one analytics destination. They may use BigQuery for investigation, internal analytics platforms, or additional SIEMs. In these environments, relying on post-ingest normalization means repeating parsing and mapping logic across systems.

Finally, cost optimization is constrained when normalization happens after ingestion. All events are ingested before any reduction, deduplication, or filtering can take place.

Introducing the DataStream + Google SecOps integration

The VirtualMetric DataStream integration with Google SecOps moves normalization and validation upstream.

DataStream now supports native UDM normalization, UDM schema validation, Google SecOps automations and content packs, and automated routing to both Google SecOps and Google BigQuery as a data lake destination.

Instead of treating Google SecOps as both the ingestion parser and analytics engine, DataStream acts as the security data pipeline layer. It prepares events before ingestion and delivers already structured, validated, and ready for analysis data.

What teams gain from pre-ingest UDM normalization

Deterministic structure and stable semantics

Pre-ingest UDM normalization gives teams a consistent and predictable data structure. Field mappings are defined explicitly and remain stable unless teams choose to change them. Vendor format changes are isolated at the pipeline level rather than affecting detections downstream.

This is particularly important for regulated environments, long-running detections, and investigations that depend on historical consistency.

More reliable ingestion at scale

When events arrive in UDM format, Google SecOps can skip parsing and focus on validation and ingestion. This reduces processing overhead per event and leads to more consistent ingestion behavior, especially for high-volume or custom log sources. Ingestion latency is more predictable, and parser-related failures are reduced.

A cleaner multi-destination architecture

Many organizations send the same security events to multiple destinations. When normalization happens once, upstream, the same UDM-normalized event can be routed to Google SecOps, stored in BigQuery, or forwarded to other analytics platforms without re-parsing.

This simplifies pipeline design and avoids maintaining multiple normalization paths for the same data.

Schema governance and drift detection

DataStream adds capabilities that are not exposed by Google SecOps itself. These include UDM schema validation, drift detection, and configurable validation modes.

By validating schema correctness before ingestion, teams can detect vendor breakage or mapping issues early and address them before they affect detections.

Direct control over ingestion volume and cost

Pre-ingest normalization allows teams to reduce data volume before it reaches Google SecOps. Deduplication, removal of non-security events, and field-level payload optimization can be applied upstream.

When normalization happens only after ingestion, all data is ingested and billed first. Pre-ingest pipelines allow cost control to be applied directly at the source.

How the integration works

The Google SecOps integration is implemented through Google SecOps–specific automation and content packs in VirtualMetric DataStream.

These packs normalize incoming security telemetry into UDM and validate the resulting schema before delivery. The processing standardizes heterogeneous inputs while preserving important vendor-specific context.

The integration supports ingestion of syslog data, Windows Security Events, Windows Firewall and DNS logs, Linux Event Logs, and other common security telemetry sources.

Event flow

Logs enter DataStream through existing collection mechanisms.

Intelligent source detection based on VirtualMetric event definition IDs identifies log formats and vendors.

Events are routed through optimized, vendor-aware pipelines and normalized directly into UDM.

Windows security telemetry is processed through dedicated pipelines to ensure accurate representation of identity, process, network, and DNS activity.

Linux logs are filtered, with normalization planned for future updates.

Events are processed using conditional routing, pipeline isolation, parallel processing, recursive cleanup of empty fields, and metadata lineage tracking.

This architecture delivers consistent, efficient, and Google SecOps-ready security data across complex, multi-vendor environments.

Getting started

Getting started does not require changes to existing log sources.

Configure your Google SecOps target in VirtualMetric DataStream.

2. Enable Google Security Operations Automation and Normalization Pack in Content Hub for automatic normalization.

enable google secops automation and normalization pack

3. Open Quick Routes, add a new target (the one you configured in step 1 above), and select which logs should be sent to Google SecOps, such as Windows and Linux logs, firewall and network device events, and cloud service telemetry.

4. For each route, attach the installed Google Security Operations Automation and Normalization Pack.

5. Start streaming data in real time: with no agent updates or custom parsing scripts.

Telemetry from on-prem, cloud, and hybrid environments can then be ingested through a single, consistent pipeline.

A better way to power Google SecOps

Google SecOps provides strong analytics and detection capabilities, but its effectiveness depends on the quality and consistency of the data it receives.

With VirtualMetric DataStream, teams can normalize security telemetry into UDM before ingestion, validate schema correctness, control drift, and route the same data to multiple destinations. This improves reliability, supports long-term detections, and gives SOC teams greater confidence in their data.

Want to explore how DataStream can be integrated into your environment? Start with our documentation, deploy a free trial in your own infrastructure, or schedule a technical session with our engineers to review pipeline design, UDM alignment, and intelligent routing.