Watcher Documentation

OpenStack Watcher provides a flexible and scalable resource optimization service for multi-tenant OpenStack-based clouds. Watcher provides a complete optimization loopincluding everything from a metrics receiver, complex event processor and profiler, optimization processor and an action plan applier. This provides a robust framework to realize a wide range of cloud optimization goals, including the reduction of data center operating costs, increased system performance via intelligent virtual machine migration, increased energy efficiency and more!

Watcher project consists of several source code repositories:

watcher – is the main repository. It contains code for Watcher API server, Watcher Decision Engine and Watcher Applier.
python-watcherclient – Client library and CLI client for Watcher.
watcher-dashboard – Watcher Horizon plugin.

The documentation provided here is continually kept up-to-date based on the latest code, and may not represent the state of the project at any specific prior release.

Chapter One – System Architecture

This page presents the current technical Architecture of the Watcher system.

1.1 Overview

Below you will find a diagram, showing the main components of Watcher:

(See Table on p.3)

1.2 Components

1.2.1 AMQP Bus

The AMQP message bus handles internal asynchronous communications between the different Watcher components.

1.2.2 Datasource

This component stores the metrics related to the cluster. It can potentially rely on any appropriate storage system (InfluxDB, OpenTSDB, MongoDB,) but will probably be more performant when using Time Series Databases which are optimized for handling time series data, which are arrays of numbers indexed by time (a datetime or a datetime range).

1.2.3 Watcher API

This component implements the REST API provided by the Watcher system to the external world. It enables the Administrator of a Cluster to control and monitor the Watcher system via any interaction mechanism connected to this API:

CLI
Horizon plugin
Python SDK

You can also read the detailed description of Watcher API.

1.2.4 Watcher Applier

This component is in charge of executing the Action Plan built by the Watcher Decision Engine. Taskflow is the default workflow engine for Watcher.

It connects to the message bus and launches the Action Plan whenever a triggering message is received on a dedicated AMQP queue.

The triggering message contains the Action Plan UUID.

It then gets the detailed information about the Action Plan from the Watcher Database which contains the list of Actions to launch.

It then loops on each Action, gets the associated class and calls the execute() method of this class. Most of the time, this method will first request a token to the Keystone API and if it is allowed, sends a request to the REST API of the OpenStack service which handles this kind of atomic Action.

Note that as soon as Watcher Applier starts handling a given Action from the list, a notification message is sent on the message bus indicating that the state of the action has changed to ONGOING.

If the Action is successful, the Watcher Applier sends a notification message on the bus informing the other components of this.

If the Action fails, the Watcher Applier tries to rollback to the previous state of the Managed resource (i.e. before the command was sent to the underlying OpenStack service).

In Stein, added a new config option action_execution_rule which is a dict type. Its key field is strategy name and the value is ALWAYS or ANY. ALWAYS means the callback function returns True as usual. ANY means the return depends on the result of previous action execution. The callback returns True if previous action gets failed, and the engine continues to run the next action. If previous action executes success, the callback returns False then the next action will be ignored. For strategies that arent in ac- tion_execution_rule, the callback always returns True. Please add the next section in the watcher.conf file if your strategy needs this feature.

[watcher_workflow_engines.taskflow]
action_execution_rule = {‘your strategy name’: ‘ANY’}

1.2.5 Watcher CLI

The watcher command-line interface (CLI) can be used to interact with the Watcher system in order to control it or to know its current status. Please, read the detailed documentation about Watcher CLI.

1.2.6 Watcher Dashboard

The Watcher Dashboard can be used to interact with the Watcher system through Horizon in order to control it or to know its current status. Please, read the detailed documentation about Watcher Dashboard.

1.2.7 Watcher Database

This database stores all the Watcher domain objects which can be requested by the Watcher API or the Watcher CLI: