Malformed data - Tero Documentation

Malformed data includes binary payloads, corrupted output, truncated structured data, and strings that cannot be parsed in the expected format. Typical causes include:

Binary protocols or files routed into a text log pipeline
Application crashes or buffer limits that truncate structured output
Encoding mismatches that produce invalid characters
Partial serialization of JSON or other structured formats

Signals

Signal	Description
Binary prefix	A log body starts with a file signature or non-text bytes, such as a PNG header.
Parser failure	A field expected to contain JSON, XML, or another structured format fails parsing.
Truncation	A structured value ends before required delimiters, quotes, or braces.
Invalid encoding	The payload contains characters that cannot be decoded in the expected encoding.

Example

Binary data
Truncated JSON

Before
After

{
  "@timestamp": "2024-01-15T10:30:00Z",
  "service.name": "image-processor",
  "body": "\u0089PNG\r\n\u001a\n\u0000\u0000\u0000\rIHDR..."
}

id: drop-binary-data-image-processor
name: Drop binary data from image-processor
description: PNG image data routed to log pipeline. Not parseable, not queryable.
log:
  match:
    - resource_attribute: service.name
      exact: image-processor
    - log_field: body
      regex: "^\\x89PNG\\r\\n"
  keep: none

Before
After

{
  "@timestamp": "2024-01-15T10:30:00Z",
  "service.name": "api-service",
  "body": "{\"user_id\": \"usr_123\", \"event\": \"login\", \"metadata\": {\"ip\":"
}

id: drop-truncated-json-api-service
name: Drop truncated JSON from api-service
description: Incomplete JSON from buffer overflow or crash. Unparseable.
log:
  match:
    - resource_attribute: service.name
      exact: api-service
    - log_field: body
      malformed: json
  keep: none

Recommended enforcement

Enforce at edge

Drop malformed log events before they reach the destination provider.

Tero removes malformed events whole because their fields fail parsing.

Detection notes

Tero can match explicit binary signatures with regular expressions.
Tero can match fields that fail the expected parser, such as malformed JSON.
Scope malformed-data policies to the emitting service or field when the pattern is specific.
Dropping malformed events preserves valid error, warning, and diagnostic logs that are parseable.

​Signals

​Example

​Recommended enforcement

Enforce at edge

​Detection notes

Signals

Example

Recommended enforcement

Detection notes