Beyond Basic watch: Using Informers for Efficient Resource Caching and Event Handling

In Chapter 3, we explored the basic watch interface provided by client-go. This allowed us to receive real-time notifications when an Endpoints object changed, giving us immediate insight into the backing Pods for a Service. While functional, relying directly on the low-level watch interface has several significant drawbacks, especially for long-running applications like controllers or monitoring tools:

• API Server Load: Each independent watch maintains its own connection to the API server. If you have many watchers, this can increase load. Furthermore, to get context about related objects (e.g., fetching the Pod details when an Endpoint address appears), you often need to make additional Get requests back to the API server.

• Reliability Boilerplate: Watch connections can (and do) drop due to network issues, API server restarts, or timeouts (like etcd's 5-minute limit on consistent reads). Handling reconnection, managing resource versions (ResourceVersion) to resume the watch without missing events, and dealing with potential errors requires substantial, non-trivial boilerplate code.

• Lack of Caching: The basic watch provides events, but it doesn't maintain a local cache of the objects being watched. If your event handler needs the current state of multiple objects, it has to fetch them repeatedly from the API server.

• Event Handling Complexity: You receive raw events (ADDED, MODIFIED, DELETED) and need to manage state transitions and potentially deduplicate events yourself.

To address these challenges, client-go provides a much more robust and efficient higher-level abstraction: Informers. For almost any real-world controller or application that needs to continuously monitor Kubernetes resources, Informers are the standard and recommended approach.

Beyond Basic watch: Using Informers for Efficient Resource Caching and Event Handling

The Informer framework in client-go is designed specifically to provide a reliable, cached, and event-driven view of Kubernetes resources with minimal load on the API server.

Core Components of the Informer Framework:

1. Reflector: This component internally uses the low-level watch mechanism we saw earlier. It watches a specific resource type on the Kubernetes API server. It performs an initial List to get the current state and then watches for subsequent ADDED, MODIFIED, and DELETED events.

2. Delta FIFO Queue: The Reflector puts the detected changes (known as "deltas" – essentially the event type and the object involved) into a thread-safe FIFO (First-In, First-Out) queue. This queue acts as a buffer and decouples the process of watching the API from the process of updating the local cache and triggering event handlers.

3. Indexer (Local Cache): This is a crucial component. It's a thread-safe, in-memory cache that stores the objects being watched. It processes the deltas from the Delta FIFO queue, updating its local representation of the resources. More than just a cache, it also allows defining index functions (hence the name "Indexer") to efficiently retrieve objects based on fields other than just name/namespace (e.g., by label, by node name). This local cache significantly reduces the need to query the API server directly.

4. Controller/Processor: This part is conceptually linked to the Indexer. As the Indexer processes updates from the Delta FIFO, it triggers registered event handler functions that you provide. This is where your application logic resides – reacting to Add, Update, and Delete events based on the state changes observed in the cache.

Shared Informers and Factories:

While you can create individual informers, client-go heavily promotes the use of Shared Informers via a SharedInformerFactory.

• SharedInformerFactory: A factory object that creates and manages multiple informers.

• Shared Informer: When you request an informer for a specific resource type (Group/Version/Kind) from the factory, it either creates a new one or returns an existing one if another part of your application already requested it.

• Benefits: Crucially, all informers created by the same factory share a single underlying watch connection and a single cache for each resource type. This drastically reduces the number of connections to the API server and minimizes memory usage compared to running many independent informers/watches.

Why Use Informers? (Advantages over Basic Watch)

• Reliability: Informers automatically handle watch reconnection, manage resource versions for consistent event streams, and perform cache resynchronization if needed. You don't need to write complex retry and error-handling logic for the watch itself.

• Caching: Provides a local, synchronized, thread-safe cache (the Indexer). Your event handlers can reliably and efficiently query this cache (lister.Get(), lister.List()) for the state of objects without hitting the API server for every lookup.

• Efficiency: Reduces load on the Kubernetes API server due to shared connections (via the factory) and local caching.

• Structured Event Handling: Offers a clean way to register event handlers (AddFunc, UpdateFunc, DeleteFunc) that are invoked when objects are added, updated, or deleted in the cache. Informers handle event deduplication and ensure handlers are called appropriately.

In essence: Informers abstract away the complexities of reliable watching and caching, letting you focus on the logic of reacting to state changes in the cluster. They are the foundation upon which most Kubernetes controllers and operators are built.

In the next section, we'll see how to set up a SharedInformerFactory and register event handlers to start monitoring network-related resources like Pods, Services, and NetworkPolicies efficiently.