Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Environment Variables

There are a number of environment variables that you can set to have even more granular control over some aspects of Venator, detailed below:

GOMAXPROCS

GOMAXPROCS (integer, optional): Overrides the maximum number of threads available to the program.

This is a very low-level tuning parameter you likely do not need to touch.

From the Golang documentation:

The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit.

If not provided, the value is determined via an appropriate default value from a combination of (source)

  • the number of logical CPUs on the machine,
  • the process’s CPU affinity mask,
  • and, on Linux, the process’s average CPU throughput limit based on cgroup CPU quota, if any.

Venator uses this value a lot to determine the maximum number of concurrent threads that a routine can spawn for operations that may spawn an unbounded number of threads (such as global profile updates). In cases where this limit is explicitly checked for, as many threads as required will be created, but only up to N will be executed - the rest will be waiting on a semaphore.

Keep in mind not all operations that split into concurrent routines respect this limit, only those that expect to spawn a lot.

If you are on a single or even dual-core machine, you may wish to raise this. You can see the calculated value at runtime by running venatorctl debug, or by checking POST /_venator/v0/admin/metrics for max_threads.

VENATOR_CONFIG

VENATOR_CONFIG (non-empty string): The path where the configuration file for Venator can be found.

Overrides the path provided to venatorctl --config, if any.

VENATOR_TEST_DB_URI

VENATOR_TEST_DB_URI (non-empty string): The fully qualified postgresql:// connection URI for a test database.

Used for integration testing - set this if you are running the test suite with go test. Otherwise, database tests will be skipped. The database will be destroyed and recreated repeatedly!

VENATOR_ALLOW_MULTIPLE_SIGNING_KEYS

VENATOR_ALLOW_MULTIPLE_SIGNING_KEYS (literal 1): When value equals 1, permits having multiple active signing keys.

By default, when starting up, Venator will immediately invalidate all but one signing key if multiple active and valid ones are found. If, for whatever reason, you do not wish for this to happen, setting this env var will skip this process. This voids your warranty.

VENATOR_CLIENT_IP_HEADERS

VENATOR_CLIENT_IP_HEADERS (non-empty string): A comma separated list of header names to trust as sources of true incoming client IPs.

By default, only X-Forwarded-For and X-Real-Ip are checked for client IPs. If your reverse proxy uses another one, you will have to specify it here. Note that specifying headers here overrides the built-in ones.

Example:

VENATOR_CLIENT_IP_HEADERS=X-My-Custom-Header,X-Forwarded-For,X-Real-Ip

VENATOR_TRUSTED_PROXIES

VENATOR_TRUSTED_PROXIES (non-empty string): A comma separated list of network ranges (CIDR) that will be trusted to provide proxy information (see VENATOR_CLIENT_IP_HEADERS).

When not provided, defaults to only loopback addresses (127.0.0.0/8 and ::1/128).

Example:

VENATOR_TRUSTED_PROXIES=172.16.0.0/12,127.0.0.0/8,::1/128

VENATOR_TRUST_CLOUDFLARE

VENATOR_TRUST_CLOUDFLARE (literal 1): When enabled, enable Cloudflare direct proxy support.

It is not recommended that you do this, but if you are using Cloudflare as a proxy (i.e. have the orange cloud), and don’t have a proxy like Caddy in front of the service that handles the incoming request properly, you can set VENATOR_TRUST_CLOUDFLARE=1. This will add Cloudflare’s IP ranges to your VENATOR_TRUSTED_PROXIES, and also appends Cf-Connecting-Ip to VENATOR_CLIENT_IP_HEADERS.

It is recommended you leave this disabled unless you know you need it.

VENATOR_NO_CACHE_PREALLOC

VENATOR_NO_CACHE_PREALLOC (boolean): When enabled key-value in-memory caches will not be pre-allocated.

To reduce the number of memory allocations Venator has to do to cache new entries in its various key-value caches, it will pre-allocate all the space it expects to need for each cache at initialisation time. This has the benefit that cache operations will be faster as they will not need to rely on the Go runtime dynamically resizing the underlying storage. It has the downside, however, that Venator will appear to waste memory until the caches fill up.

If the increased performance cost at the benefit of decreased initial memory usage is something you need, the pre-allocation can be disabled by setting this environment variable. This is typically not necessary, however. Ideal cache sizes are constantly re-evaluated based on real-world data from the demo instance, and some caches can be manually adjusted.