The Netflow Collector is a robust, UDP-based service responsible for ingesting, validating, and decoding NetFlow and IPFIX network traffic data. It features dynamic configuration reconciliation, per-source template caching, and multi-version support to handle diverse network telemetry streams.

Architecture Overview

The collector operates as a standalone UDP listener that dynamically adjusts to configuration changes without requiring a full agent restart. It routes incoming packets to specific decoders based on the protocol version detected in the packet header.

Collector Lifecycle

The NetflowCollector manages its own lifecycle, including port binding, configuration watching, and background maintenance tasks.

Dynamic Reconciliation

Instead of requiring a restart when ports or listening states change, the collector uses a configwatcher to monitor the configuration file. When a change is detected, the reconcile() method evaluates the current state against the desired state:

Packet Validation

Before any decoding occurs, incoming UDP payloads are passed through validateNetflowPacket(). This prevents malformed packets from crashing the decoders.

The validator reads the first 4 bytes to determine the version and record count/length, enforcing minimum size requirements based on the protocol specification:

Decoding Logic and State Management

NetFlow and IPFIX are stateful protocols. Versions 9 and 10 rely on templates sent periodically by the exporter, while older versions require session tracking to handle sequence numbers properly.

Template System (v9 & IPFIX)

NetFlow v9 and IPFIX separate the data structure definition (Templates) from the actual data records. The collector implements a thread-safe templateSystem to cache these templates.

// templateSystem implements netflow.NetFlowTemplateSystem for goflow2
type templateSystem struct {
	// Map structure: version -> obsDomainId -> templateId -> template
	templates map[uint16]map[uint32]map[uint16]interface{}
	lastUsed  time.Time
	mu        sync.RWMutex
}

Templates are stored hierarchically and are tracked per source IP address. When a data record arrives, the decoder looks up the corresponding template using the Version, Observation Domain ID, and Template ID.

Legacy Decoders (v1, v6, v7)

For older protocols, the collector uses the tehmaze/netflow library. Because these protocols track sequence numbers to detect dropped packets, a dedicated session decoder is maintained for each source IP address via the legacyDecoderEntry struct.

Cache Eviction

To prevent memory leaks from ephemeral exporters or IP address changes, the collector runs a background garbage collection routine (cleanupStaleEntries).