Excessive payloads

An excessive payload is a large value embedded in a log event, such as a full HTTP response body, serialized object, data blob, or oversized stack trace. These fields increase event size. In many cases, the log keeps its operational meaning after a policy removes the large field.

Signals

Signal	Description
Large field size	A single field contributes most of the event size.
Serialized object	The field contains a full object that can be retrieved by ID elsewhere.
Response or request body	The log includes full HTTP payload content.
Repeated oversized stack trace	Many events contain the same long stack trace or traceback.

Example

Response body
Stack trace

Before
After

{
  "@timestamp": "2024-01-15T10:30:00Z",
  "service.name": "order-service",
  "event": "order.created",
  "order_id": "ORD-12345",
  "http.response.body": "{\"id\":\"ORD-12345\",\"items\":[{\"sku\":\"SKU-001\",\"name\":\"Widget Pro\",\"quantity\":2,\"price\":29.99},{\"sku\":\"SKU-002\",\"name\":\"Gadget Plus\",\"quantity\":1,\"price\":49.99}],\"shipping\":{\"method\":\"express\",\"address\":{\"street\":\"123 Main St\",\"city\":\"Seattle\",\"state\":\"WA\",\"zip\":\"98101\",\"country\":\"US\"}},\"billing\":{\"method\":\"card\",\"last4\":\"4242\"},\"totals\":{\"subtotal\":109.97,\"shipping\":12.99,\"tax\":10.45,\"total\":133.41}}"
}

{
  "@timestamp": "2024-01-15T10:30:00Z",
  "service.name": "order-service",
  "event": "order.created",
  "order_id": "ORD-12345"
}

id: remove-response-body-order-service
name: Remove response body from order-service
description: Drop full HTTP response body. The order_id is sufficient for lookup.
log:
  match:
    - resource_attribute: service.name
      exact: order-service
    - log_attribute: event
      exact: order.created
  transform:
    remove:
      - log_attribute: http.response.body

Before
After

{
  "@timestamp": "2024-01-15T10:30:00Z",
  "service.name": "api-service",
  "severity_text": "ERROR",
  "error.message": "Connection refused",
  "error.stack_trace": "Error: Connection refused\n    at Socket.connect (net.js:1141:16)\n    at DBClient.connect (db.js:89:12)\n    ... 200 more lines ..."
}

{
  "@timestamp": "2024-01-15T10:30:00Z",
  "service.name": "api-service",
  "severity_text": "ERROR",
  "error.message": "Connection refused"
}

If the full stack trace is available in a tracing backend, the log event can keep the error message without keeping a duplicate stack trace.

Recommended enforcement

Open PRs

Change the source instrumentation when the large field comes from application logging.

Use open PR enforcement when application code added the payload and a developer needs to decide which fields stay.

Detection notes

Tero flags fields that are large relative to routine log events for the same service or pattern.
Common candidates include full HTTP bodies, entire serialized objects, and long stack traces.
Policies can remove a single large field while preserving the rest of the event.
A source change is preferable when the log statement should stop emitting the payload.

Introduction

Issues

Master Catalog

Policies

Integrations

Reference

Signals

Example

Recommended enforcement

Open PRs

Detection notes

​Signals

​Example

​Recommended enforcement

Open PRs

​Detection notes

Signals

Example

Recommended enforcement

Detection notes