Configuration

Overview

OpenTSDB is now configured via one or more sources that are flattened into a single application wide configuration consisting of keys and values. Multiple sources can be provided with the same keys but different values and later values will override the earlier values. As an example, a common config may be available over HTTP with many settings but a local config may override some of those settings for a specific environment.

Available built-in providers include:

• Environment - Environment variables can be set with keys and values, useful in a containerized environment.
• System Properties - JVM properties can be parsed such as those given over the command line in the -Dmy.key=my.value format.
• Command Line - Traditional command line arguments like --my.key=my.value
• YAML/JSON File - Either a local file or available over HTTP(S). Suffixes supported include .yaml, .yml, .json, .jsn.
• Java Properties File - Either a local file or available over HTTP(S). Suffixes supported include .conf, .config, .txt, .properties.
• Secrets Support - A plugin to load encrypted values from a secure location like Amazon's AWS Secrets, a local key vault or other secure locations. Useful for credential storage.

The config.providers key is used to set the providers and order that they're loaded in. Providers are separated by commas and the last configuration in the list is loaded first, with earlier configs over-ridding the later configs (NOTE: If this is a pain we can reverse it before release, let us know what ya'll think). And example config may look like this:

--config.providers=secrets://net.opentsdb.configuration.providers.AWSSecretsProvider:AWS,https://tsdbconfig.mysite.com/stage/elasticsearch.yaml,https://tsdbconfig.mysite.com/stage/tsdr.yaml,https://tsdbconfig.mysite.com/stage/hbase.yaml,https://tsdbconfig.mysite.com/stage/common.yaml

With secrets, files and remote files, the configuration format uses protocols like https://, file:// or secrets://. These protocols determine which plugin or built-in module parses the config. For remote and files, the suffix will also determine the parser.

Because multiple configurations are flattened into a single structure, key values in earlier providers will be overwritten by those in later providers. For example, lets assume the following providers config: --config.providers=https://tsdbconfig.mysite.com/stage/tsdr.yaml,https://tsdbconfig.mysite.com/stage/common.yaml.

Lets say https://tsdbconfig.mysite.com/stage/common.yaml contains:

tsd.stats.environment: development

and https://tsdbconfig.mysite.com/stage/tsdr.yaml defines

tsd.stats.environment: stage

since we're in stage. The final value read by TSDB for tsd.stats.environment will be stage.