The challenge
You own every update, every patch, every fix
Logstash is open-source; enterprise support is available via Elastic subscriptions, but not as a standalone managed service. Updates must be managed and deployed by your team. There are no automatic patches, no centrally managed rollouts, and compliance depends entirely on how your deployment is configured and maintained.
The solution
DataStream – intelligent security data pipeline platform
A platform that maintains itself
DataStream handles collection, normalization, and reduction automatically, with expert validated logic maintained by VirtualMetric. DataStream is a commercially supported platform (ISO 27001 certified and SOC 2 Type II compliant) with security patches deployed automatically and pre-built integrations for every major security source are maintained and updated by VirtualMetric.
Features
Logstash is a pipeline engine. DataStream is a full pipeline platform.
Logstash is typically deployed as part of a broader stack (Beats, Elasticsearch, Kibana), requiring teams to assemble and maintain multiple components. DataStream replaces that complexity with a single, integrated platform.
|
VirtualMetric DataStream
|
Logstash
|
|
|---|---|---|
| Primary role | End-to-end security data pipeline platform | Data processing engine (ETL component) |
| Data collection | Built-in — agentless + agent-based | Limited (input plugins); typically relies on Beats |
| Pipeline configuration | No-code / declarative | Code-based (Ruby DSL, Grok) |
| Pre-built integrations | Vendor packs — security-focused | Community plugins |
| Data transformation | Built-in processors (170+) | Filters via plugins (manual config) |
| Schema normalization | Automatic — ASIM, ECS, OCSF, CIM, UDM | Limited (ECS mainly; others manual) |
| Enrichment | Native — TI, context, metadata | Plugin-based (manual setup) |
| Routing | Intelligent, multi-destination | Conditional outputs |
| Real-time processing | ||
| Performance efficiency | High — optimized, compressed pipeline | Resource-intensive (heap tuning required) |
| Data reliability | WAL-based — no data loss | Persistent queue (at-least-once) |
| Setup time (new source) | Minutes — pre-built packs | Hours–days (parsing + testing) |
| Maintenance overhead | Vendor-maintained | User-maintained configs |
| Pipeline visibility | Built-in — end-to-end | Via APIs / Kibana (external) |
Why security teams choose DataStream
Agentless or agent-based – your choice
DataStream supports both agentless pull collection and push-based ingestion (Syslog, TCP/UDP, HTTP) – with nothing installed on target systems. Where deeper endpoint visibility is needed, lightweight agents are available. Logstash provides input plugins for ingestion, but in practice relies on external shippers like Beats. Before a single log arrives, you’re already managing a separate fleet on every source system.
170+ no-code processors – no Ruby required
DataStream ships with 170+ processors in a declarative, no-code syntax. Filtering, enrichment, normalization, masking, and routing are all configured without scripting. Logstash pipelines are built in Ruby DSL with Grok patterns – every rule is custom code your team owns, debugs, and maintains indefinitely. Every new source, every schema change, and every edge case adds to that burden.
10x speed – and a fraction of the memory
A vectorized engine delivers 10x faster processing with up to 99% VMF compression and a built-in WAL for 100% delivery guarantee. For a 2 TB/day workload: 2 cores and 256 MB RAM. Logstash runs on the JVM, typically requiring JVM tuning and significantly more memory compared to lightweight pipeline engines, and is often scaled horizontally to handle high-volume workloads.
Automatic multi-schema normalization
DataStream automatically maps data to ASIM (Sentinel), OCSF (Amazon Security Lake), ECS (Elastic), CIM (Splunk), and UDM (Google SecOps) – per destination, with no manual configuration. Logstash supports ECS natively, but only for Elastic destinations. Normalizing for Sentinel or any other SIEM requires building and maintaining custom field mapping logic from scratch.
Production-ready in under 30 minutes
Pre-built vendor packs deploy out of the box, so data reaches your target in the correct schema immediately, without scripting or a dedicated pipeline engineer. A typical Logstash deployment for a new source takes from hours to days, depending on log format complexity: writing Grok patterns, testing against sample logs, handling edge cases, and deploying Beat configuration across the estate.
Security-specific vendor packs – maintained for you
DataStream’s Content Hub ships pre-built packs for Fortinet, Palo Alto, Check Point, Cisco, and more – all validated against real detection content. Logstash has no equivalent. Community plugins exist, but there is no SLA, no validation against detection content, and no guarantee they are updated when a vendor ships a firmware update. Your team owns every fix.
Full visibility into your pipeline
DataStream provides end-to-end pipeline visibility: source health, volume analytics, schema drift alerting, and anomaly detection. If something breaks upstream, you know before your detection coverage is affected. Logstash has no built-in, unified observability layer. Visibility typically requires Elastic Stack components. If a Grok pattern silently breaks, someone finds out during an incident.
Full audit trail – every change, every access
Every pipeline configuration change, every access event, and every modification in DataStream is logged, attributable, and available for compliance review. Logstash has no native audit trail for pipeline changes without external tooling. For teams operating under GDPR, NIS2, HIPAA, or SOX, this is a meaningful compliance gap.
Frequently asked questions
Can DataStream replace Logstash without disrupting existing pipelines?
Yes. DataStream can run in parallel with your existing Logstash setup, collecting from the same sources simultaneously. This lets you validate DataStream’s output against what Logstash currently produces before cutting over, with no risk to production visibility. Most teams complete the transition source by source within a few days.
We’ve invested months in our Logstash config. What carries over?
The routing logic and destination endpoints carry over directly. The parsing and normalization logic does not – DataStream replaces Grok patterns and Ruby filters with pre-built vendor packs and its 170+ processor library. For most teams, this is the point: the config you’ve built is what DataStream is designed to make unnecessary going forward. Source-specific tuning that took weeks in Logstash typically takes minutes in DataStream.
Our team knows Ruby and Grok. Is DataStream flexible enough for custom logic?
Yes. DataStream’s 170+ processors cover the vast majority of what custom Logstash filters handle: field removal, event filtering, enrichment, masking, schema transformation, and conditional routing – without scripting. For logic that goes beyond the built-in processors, custom pipeline conditions based on your specific alert queries are fully supported. Teams that have spent months tuning Logstash typically find the equivalent logic is simpler to express in DataStream and no longer breaks when upstream sources change.
Does DataStream work with other SIEMs besides Microsoft Sentinel?
Yes. DataStream is vendor-agnostic and can route to Splunk, Elastic, Google SecOps, Amazon Security Lake, CrowdStrike, and any destination that accepts standard formats. Sentinel is a common starting point because DataStream’s ASIM normalization is particularly deep there, but the platform works equally well across any combination of destinations simultaneously.
Who maintains the vendor packs when a source changes its log format?
VirtualMetric does. When a vendor updates their log format (a firewall firmware update that reshuffles field names, a cloud service that changes its event schema), the corresponding vendor pack is updated. Your pipeline continues working without any action from your team. With Logstash, a vendor update that breaks a Grok pattern lands in your queue: your engineers diagnose the failure, update the config, test it, and redeploy. That cycle repeats for every vendor, every update.
Talk to our experts
Schedule a technical session with our engineering team to see how DataStream compares to what you’re running today.
Try DataStream
Route data to your SIEM in the correct schema, with automatic normalization and up to 90% data volume reduction.
Try now