> ## Documentation Index > Fetch the complete documentation index at: https://docs.usetero.com/llms.txt > Use this file to discover all available pages before exploring further. # OpenTelemetry Collector > Apply policies to logs in the OpenTelemetry Collector Apply Tero policies to logs passing through your OpenTelemetry Collector. Choose the option that fits your environment. | Option | Best for | | -------------------- | ---------------------------------------------------- | | **Tero Distro** | New deployments or replacing your existing collector | | **Edge Proxy** | Adding to an existing collector without rebuilding | | **Policy Processor** | Custom collector distributions you build yourself | Run the Tero Collector, a pre-built OpenTelemetry Collector distribution with the Policy Processor included. ## How it works The Tero Collector is a standard OTel Collector with our Policy Processor baked in. Deploy it like any collector, configure policies, and it filters logs before they reach your backend. ```mermaid theme={null} flowchart LR Apps[Applications] --> Collector[Tero Collector] Collector --> Backend[Backend] style Apps fill:#262626,stroke:#262626,color:#fafafa style Collector fill:#10b981,stroke:#10b981,color:#fff style Backend fill:#262626,stroke:#262626,color:#fafafa ``` The Policy Processor supports three actions: * **Drop**: Remove logs matching patterns * **Keep**: Retain only logs matching patterns * **Sample**: Sample at configurable rates Policies hot-reload without restarting the collector. ## Included components The Tero Collector includes standard OTel components: | Type | Components | | -------------- | ------------------------------------------------------------- | | **Receivers** | OTLP (gRPC, HTTP) | | **Processors** | Policy, Batch, Memory Limiter, Attributes, Filter, Resource | | **Exporters** | Debug, OTLP (gRPC, HTTP) | | **Connectors** | Forward | | **Extensions** | Health Check v2, zPages, PProf, Basic Auth, Bearer Token Auth | ## Prerequisites * Docker or Kubernetes cluster * Tero account ## Deploy Create a collector config: ```yaml config.yaml theme={null} receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 exporters: otlphttp: endpoint: https://your-backend-endpoint.com service: pipelines: logs: receivers: [otlp] processors: [policy] exporters: [otlphttp] ``` Run the collector: ```bash theme={null} docker run --rm -p 4317:4317 -p 4318:4318 \ -e TERO_API_KEY=YOUR_API_KEY \ -v $(pwd)/config.yaml:/etc/tero-collector/config.yaml:ro \ ghcr.io/usetero/tero-collector-distro:latest ``` Create the namespace and secret: ```bash theme={null} kubectl create namespace observability kubectl create secret generic tero-collector \ --namespace observability \ --from-literal=api-key=YOUR_API_KEY ``` Deploy the collector: ```yaml tero-collector.yaml theme={null} apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: tero-collector namespace: observability spec: mode: deployment image: ghcr.io/usetero/tero-collector-distro:latest env: - name: TERO_API_KEY valueFrom: secretKeyRef: name: tero-collector key: api-key config: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 exporters: otlphttp: endpoint: https://your-backend-endpoint.com service: pipelines: logs: receivers: [otlp] processors: [policy] exporters: [otlphttp] ``` ```bash theme={null} kubectl apply -f tero-collector.yaml ``` Create the namespace and secret: ```bash theme={null} kubectl create namespace observability kubectl create secret generic tero-collector \ --namespace observability \ --from-literal=api-key=YOUR_API_KEY ``` Add to your `values.yaml`: ```yaml values.yaml theme={null} mode: deployment image: repository: ghcr.io/usetero/tero-collector-distro tag: latest extraEnvs: - name: TERO_API_KEY valueFrom: secretKeyRef: name: tero-collector key: api-key config: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 exporters: otlphttp: endpoint: https://your-backend-endpoint.com service: pipelines: logs: receivers: [otlp] processors: [policy] exporters: [otlphttp] ``` Install the chart: ```bash theme={null} helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts helm install tero-collector open-telemetry/opentelemetry-collector \ --namespace observability \ -f values.yaml ``` Check the container is running: ```bash theme={null} docker ps | grep tero-collector ``` Check logs for policy sync: ```bash theme={null} docker logs 2>&1 | grep -i policy ``` ```bash theme={null} kubectl get pods -n observability -l app.kubernetes.io/name=tero-collector-collector kubectl logs -n observability -l app.kubernetes.io/name=tero-collector-collector | grep -i policy ``` ```bash theme={null} kubectl get pods -n observability -l app.kubernetes.io/instance=tero-collector kubectl logs -n observability -l app.kubernetes.io/instance=tero-collector | grep -i policy ``` ## Policy providers Configure where the collector loads policies from. ### File provider Load from a local file. Use this for static policies or GitOps workflows. ```yaml theme={null} processors: policy: providers: - type: file id: local path: /etc/tero-collector/policies.json poll_interval_secs: 30 ``` ### HTTP provider Fetch from a remote endpoint. Choose this for dynamic policies managed via Tero. ```yaml theme={null} processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 ``` Run Edge as a sidecar to your existing OpenTelemetry Collector. Edge proxies telemetry, applies policies, and forwards to your backend. ## How it works Edge runs alongside your OTel Collector. The collector exports telemetry through Edge, which applies policies and forwards to your backend. ```mermaid theme={null} flowchart LR subgraph Deployment OTel[OTel Collector] Edge[Tero Edge] OTel --> Edge end Apps --> OTel Edge --> Backend[Backend] style Apps fill:#262626,stroke:#262626,color:#fafafa style OTel fill:#f59e0b,stroke:#f59e0b,color:#000 style Edge fill:#10b981,stroke:#10b981,color:#fff style Backend fill:#262626,stroke:#262626,color:#fafafa ``` ## Prerequisites * Existing OpenTelemetry Collector * Tero account ## Deploy Create an Edge config: ```json edge-config.json theme={null} { "listen_address": "0.0.0.0", "listen_port": 8080, "upstream_url": "https://your-backend-endpoint.com", "log_level": "info", "service": { "name": "edge", "namespace": "production", "resource_attributes": [ { "key": "deployment.environment", "value": "production" } ], "labels": [{ "key": "team", "value": "platform" }] }, "max_body_size": 1048576, "policy_providers": [ { "id": "tero", "type": "http", "url": "https://sync.usetero.com/v1/policy/sync", "headers": [ { "name": "Authorization", "value": "Bearer ${TERO_API_KEY}" } ], "poll_interval_secs": 60 } ] } ``` Run Edge: ```bash theme={null} docker run --rm -p 8080:8080 \ -e TERO_API_KEY=YOUR_API_KEY \ -v $(pwd)/edge-config.json:/etc/tero/config.json:ro \ ghcr.io/usetero/edge:latest /etc/tero/config.json ``` Configure your collector to export to Edge: ```yaml theme={null} exporters: otlphttp: endpoint: http://localhost:8080 ``` Create the secret and ConfigMap: ```bash theme={null} kubectl create secret generic tero-edge \ --from-literal=api-key=YOUR_API_KEY ``` ```yaml tero-edge-config.yaml theme={null} apiVersion: v1 kind: ConfigMap metadata: name: tero-edge-config data: config.json: | { "listen_address": "127.0.0.1", "listen_port": 8080, "upstream_url": "https://your-backend-endpoint.com", "log_level": "info", "service": { "name": "edge", "namespace": "production", "resource_attributes": [ { "key": "deployment.environment", "value": "production" } ], "labels": [{ "key": "team", "value": "platform" }] }, "max_body_size": 1048576, "policy_providers": [ { "id": "tero", "type": "http", "url": "https://sync.usetero.com/v1/policy/sync", "headers": [ { "name": "Authorization", "value": "Bearer ${TERO_API_KEY}" } ], "poll_interval_secs": 60 } ] } ``` ```bash theme={null} kubectl apply -f tero-edge-config.yaml ``` Add Edge as a sidecar to your `OpenTelemetryCollector` CR: ```yaml theme={null} apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: otel-collector spec: mode: deployment config: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 exporters: otlphttp: endpoint: http://localhost:8080 service: pipelines: logs: receivers: [otlp] exporters: [otlphttp] metrics: receivers: [otlp] exporters: [otlphttp] traces: receivers: [otlp] exporters: [otlphttp] volumes: - name: tero-edge-config configMap: name: tero-edge-config volumeMounts: - name: tero-edge-config mountPath: /etc/tero readOnly: true additionalContainers: - name: tero-edge image: ghcr.io/usetero/edge:latest args: - /etc/tero/config.json env: - name: TERO_API_KEY valueFrom: secretKeyRef: name: tero-edge key: api-key resources: requests: cpu: 50m memory: 32Mi limits: cpu: 200m memory: 64Mi volumeMounts: - name: tero-edge-config mountPath: /etc/tero readOnly: true livenessProbe: httpGet: path: /_health port: 8080 initialDelaySeconds: 5 periodSeconds: 10 readinessProbe: httpGet: path: /_health port: 8080 initialDelaySeconds: 2 periodSeconds: 5 ``` Create the secret and ConfigMap: ```bash theme={null} kubectl create secret generic tero-edge \ --from-literal=api-key=YOUR_API_KEY ``` ```yaml tero-edge-config.yaml theme={null} apiVersion: v1 kind: ConfigMap metadata: name: tero-edge-config data: config.json: | { "listen_address": "127.0.0.1", "listen_port": 8080, "upstream_url": "https://your-backend-endpoint.com", "log_level": "info", "service": { "name": "edge", "namespace": "production", "resource_attributes": [ { "key": "deployment.environment", "value": "production" } ], "labels": [{ "key": "team", "value": "platform" }] }, "max_body_size": 1048576, "policy_providers": [ { "id": "tero", "type": "http", "url": "https://sync.usetero.com/v1/policy/sync", "headers": [ { "name": "Authorization", "value": "Bearer ${TERO_API_KEY}" } ], "poll_interval_secs": 60 } ] } ``` ```bash theme={null} kubectl apply -f tero-edge-config.yaml ``` Add to your collector `values.yaml`: ```yaml theme={null} extraVolumes: - name: tero-edge-config configMap: name: tero-edge-config extraVolumeMounts: - name: tero-edge-config mountPath: /etc/tero readOnly: true extraContainers: - name: tero-edge image: ghcr.io/usetero/edge:latest args: - /etc/tero/config.json env: - name: TERO_API_KEY valueFrom: secretKeyRef: name: tero-edge key: api-key resources: requests: cpu: 50m memory: 32Mi limits: cpu: 200m memory: 64Mi volumeMounts: - name: tero-edge-config mountPath: /etc/tero readOnly: true livenessProbe: httpGet: path: /_health port: 8080 initialDelaySeconds: 5 periodSeconds: 10 readinessProbe: httpGet: path: /_health port: 8080 initialDelaySeconds: 2 periodSeconds: 5 config: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 exporters: otlphttp: endpoint: http://localhost:8080 service: pipelines: logs: receivers: [otlp] exporters: [otlphttp] metrics: receivers: [otlp] exporters: [otlphttp] traces: receivers: [otlp] exporters: [otlphttp] ``` Check Edge is running: ```bash theme={null} docker ps | grep edge curl http://localhost:8080/_health ``` Check that the collector pod has both containers running: ```bash theme={null} kubectl get pods -l app.kubernetes.io/name=otel-collector kubectl logs -c tero-edge --tail=50 ``` Check that the collector pod has both containers running: ```bash theme={null} kubectl get pods -l app.kubernetes.io/instance=otel-collector kubectl logs -c tero-edge --tail=50 ``` ## Policy providers Edge supports multiple policy sources. Configure them in the `policy_providers` array. ### File provider Load policies from a local file. This suits static policies. ```json theme={null} { "id": "file", "type": "file", "path": "/etc/tero/policies.json" } ``` ### HTTP provider Fetch policies from a remote endpoint. This works well for dynamic policies managed via Tero. ```json theme={null} { "id": "tero", "type": "http", "url": "https://sync.usetero.com/v1/policy/sync", "headers": [{ "name": "Authorization", "value": "Bearer ${TERO_API_KEY}" }], "poll_interval_secs": 60 } ``` Add the Policy Processor to your own custom OpenTelemetry Collector distribution using the OpenTelemetry Collector Builder (OCB). ## How it works Build a custom collector with the Policy Processor included. This gives you full control over which components to include while adding Tero's policy filtering. ```mermaid theme={null} flowchart LR Apps[Applications] --> Collector[Your Custom Collector] Collector --> Backend[Backend] style Apps fill:#262626,stroke:#262626,color:#fafafa style Collector fill:#10b981,stroke:#10b981,color:#fff style Backend fill:#262626,stroke:#262626,color:#fafafa ``` The Policy Processor uses [Hyperscan](https://www.hyperscan.io/) for high-performance regex matching. This requires CGO and platform-specific libraries. ## Prerequisites * Go 1.24+ * [OpenTelemetry Collector Builder](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/builder) (ocb) * CGO-compatible build environment * Hyperscan/Vectorscan libraries * Tero account ### Install Hyperscan ```bash theme={null} brew install vectorscan ``` ```bash theme={null} apt-get install libhyperscan-dev ``` ```bash theme={null} apk add vectorscan-dev ``` ## Build ```yaml manifest.yaml theme={null} dist: name: my-collector description: Custom OTel Collector with Policy Processor output_path: ./build otelcol_version: 0.115.0 # Required for Hyperscan cgo_enabled: true receivers: - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.115.0 processors: - gomod: github.com/usetero/tero-collector-distro/processor/policyprocessor v0.2.0 exporters: - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.115.0 - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.115.0 extensions: - gomod: go.opentelemetry.io/collector/extension/healthcheckextension v0.115.0 ``` `cgo_enabled: true` is required. Without it, the build will fail due to Hyperscan dependencies. ```bash theme={null} ocb --config manifest.yaml ``` The built binary will be in `./build/my-collector`. ```yaml config.yaml theme={null} receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 exporters: otlphttp: endpoint: https://your-backend-endpoint.com service: pipelines: logs: receivers: [otlp] processors: [policy] exporters: [otlphttp] ``` ```bash theme={null} export TERO_API_KEY=YOUR_API_KEY ./build/my-collector --config config.yaml ``` Create a Dockerfile: ```dockerfile Dockerfile theme={null} FROM golang:1.24-bookworm AS builder # Install Hyperscan RUN apt-get update && apt-get install -y libhyperscan-dev # Install OCB RUN go install go.opentelemetry.io/collector/cmd/builder@latest WORKDIR /build COPY manifest.yaml . # Build with CGO enabled ENV CGO_ENABLED=1 RUN builder --config manifest.yaml FROM debian:bookworm-slim # Install runtime dependencies RUN apt-get update && apt-get install -y libhyperscan5 ca-certificates && rm -rf /var/lib/apt/lists/* COPY --from=builder /build/build/my-collector /collector COPY config.yaml /etc/collector/config.yaml ENTRYPOINT ["/collector"] CMD ["--config", "/etc/collector/config.yaml"] ``` Build and run: ```bash theme={null} docker build -t my-collector . docker run --rm -p 4317:4317 -p 4318:4318 \ -e TERO_API_KEY=YOUR_API_KEY \ my-collector ``` After building and pushing your image, deploy with the OpenTelemetry Operator: ```bash theme={null} kubectl create namespace observability kubectl create secret generic tero-collector \ --namespace observability \ --from-literal=api-key=YOUR_API_KEY ``` ```yaml my-collector.yaml theme={null} apiVersion: opentelemetry.io/v1beta1 kind: OpenTelemetryCollector metadata: name: my-collector namespace: observability spec: mode: deployment image: your-registry/my-collector:latest env: - name: TERO_API_KEY valueFrom: secretKeyRef: name: tero-collector key: api-key config: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 exporters: otlphttp: endpoint: https://your-backend-endpoint.com service: pipelines: logs: receivers: [otlp] processors: [policy] exporters: [otlphttp] ``` ```bash theme={null} kubectl apply -f my-collector.yaml ``` After building and pushing your image, deploy with Helm: ```bash theme={null} kubectl create namespace observability kubectl create secret generic tero-collector \ --namespace observability \ --from-literal=api-key=YOUR_API_KEY ``` ```yaml values.yaml theme={null} mode: deployment image: repository: your-registry/my-collector tag: latest extraEnvs: - name: TERO_API_KEY valueFrom: secretKeyRef: name: tero-collector key: api-key config: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: policy: providers: - type: http id: tero url: https://sync.usetero.com/v1/policy/sync headers: - name: Authorization value: Bearer ${TERO_API_KEY} poll_interval_secs: 60 exporters: otlphttp: endpoint: https://your-backend-endpoint.com service: pipelines: logs: receivers: [otlp] processors: [policy] exporters: [otlphttp] ``` ```bash theme={null} helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts helm install my-collector open-telemetry/opentelemetry-collector \ --namespace observability \ -f values.yaml ``` ## Policy configuration The processor accepts a `providers` array for policy sources. ### Provider types | Type | Description | | ------ | ------------------------ | | `file` | Load from local file | | `http` | Fetch from HTTP endpoint | | `grpc` | Fetch from gRPC endpoint | ### Provider fields | Field | Type | Description | | -------------------- | ------ | ---------------------------------------- | | `type` | string | Provider type: `file`, `http`, or `grpc` | | `id` | string | Unique identifier for this provider | | `path` | string | File path (file provider only) | | `url` | string | Remote endpoint (http/grpc only) | | `poll_interval_secs` | int | How often to check for updates | | `headers` | array | HTTP headers (http provider only) | ## Telemetry support | Signal | Status | | ------- | ----------------- | | Logs | Alpha | | Metrics | Not yet supported | | Traces | Not yet supported | ## Metrics The processor emits `processor_policy_records` (Counter) with attributes: * `telemetry_type`: `logs`, `metrics`, or `traces` * `result`: `dropped`, `kept`, `sampled`, or `no_match` ## Example policies Policies use JSON format. Here are common patterns: ```json theme={null} { "policies": [ { "id": "drop-debug-logs", "name": "drop-debug-logs", "enabled": true, "log": { "match": [{ "log_field": "severity_text", "regex": "DEBUG" }], "keep": "none" } }, { "id": "sample-noisy-service", "name": "sample-noisy-service", "enabled": true, "log": { "match": [ { "resource_attribute": "service.name", "regex": "noisy-service" } ], "sample_rate": 0.1 } }, { "id": "keep-errors", "name": "keep-errors", "enabled": true, "log": { "match": [ { "log_field": "severity_text", "regex": "ERROR" }, { "log_field": "severity_text", "regex": "CRITICAL" } ], "keep": "all" } } ] } ``` See [Policy Reference](/edge/policy-reference/log-filter) for all filtering options. ## Troubleshooting **Collector won't start** Check the config syntax: ```bash theme={null} ./collector --config config.yaml --dry-run ``` **Policies not loading** Verify the policy file path is correct and the file is valid JSON. Check collector logs for policy-related errors. **CGO build errors (Policy Processor)** Ensure Hyperscan/Vectorscan is installed and `CGO_ENABLED=1` is set. On macOS, you may need to set `PKG_CONFIG_PATH`: ```bash theme={null} export PKG_CONFIG_PATH="/opt/homebrew/lib/pkgconfig:$PKG_CONFIG_PATH" ```