Varnish Metrics

Out of the box, section.io brings up Varnish Cache metrics. The following is a description of these metrics.

Varnish Glossary

Key Varnish concepts are described in the Varnish Glossary.

Varnish Counters

Further detail with respect to key Varnish Counters (Source; Mankier)

Counter Levels

Info- Informational counters
Counters giving runtime information

Diag- Diagnostic counters
Counters giving diagnostic information

Debug- Debug counters
Counters giving Varnish internals debug information

Main Counters

sess_conn
Sessions accepted (INFO). Count of sessions successfully accepted

sess_drop
Sessions dropped (INFO).Count of sessions silently dropped due to lack of worker thread.

sess_fail
Session accept failures (INFO). Count of failures to accept TCP connection. Either the client changed its mind, or the kernel ran out of some resource like file descriptors.

sess_pipe_overflow
Session pipe overflow (INFO). Count of sessions dropped due to the session pipe overflowing.

client_req_400
Client requests received, subject to 400 errors (INFO). 400 means we couldn’’t make sense of the request, it was malformed in some drastic way.

client_req_411
Client requests received, subject to 411 errors (INFO). 411 means the client did not send a Content-Length for the req.body.

client_req_413
Client requests received, subject to 413 errors (INFO). 413 means that HTTP headers exceeded length or count limits.

client_req_417
Client requests received, subject to 417 errors (INFO) 417 means that something went wrong with an Expect: header.

client_req
Good client requests received (INFO) The count of parseable client requests seen.

cache_hit
Cache hits (INFO) Count of cache hits. A cache hit indicates that an object has been delivered to a client without fetching it from a backend server.

cache_hitpass
Cache hits for pass (INFO) Count of hits for pass A cache hit for pass indicates that Varnish is going to pass the request to the backend and this decision has been cached in it self. This counts how many times the cached decision is being used.

cache_miss
Cache misses (INFO) Count of misses A cache miss indicates the object was fetched from the backend before delivering it to the backend.

backend_conn
Backend conn. success (INFO) How many backend connections have successfully been established.

backend_unhealthy
Backend conn. not attempted (INFO)

backend_busy
Backend conn. too many (INFO)

backend_fail
Backend conn. failures (INFO)

backend_reuse
Backend conn. reuses (INFO) Count of backend connection reuses This counter is increased whenever we reuse a recycled connection.

backend_toolate
Backend conn. was closed (INFO)

backend_recycle
Backend conn. recycles (INFO) Count of backend connection recycles This counter is increased whenever we have a keep-alive connection that is put back into the pool of connections. It has not yet been used, but it might be, unless the backend closes it.

backend_retry
Backend conn. retry (INFO)

fetch_head
Fetch no body (HEAD) (INFO) beresp with no body because the request is HEAD.

fetch_length
Fetch with Length (INFO) beresp with Content-Length.

fetch_chunked
Fetch chunked (INFO) beresp with Chunked.

fetch_eof
Fetch EOF (INFO) beresp with EOF from lack of other info.

fetch_bad
Fetch bad T-E (INFO) beresp failed due to unknown Transfer-Encoding.

fetch_close
Fetch wanted close (INFO) beresp with EOF due to Connection: Close.

fetch_oldhttp
Fetch pre HTTP/1.1 closed (INFO) beresp with EOF due to HTTP < 1.1

fetch_zero
Fetch zero len body (INFO) beresp with EOF due to keep-live but neither Chunked or Len.

fetch_1xx
Fetch no body (1xx) (INFO) beresp with no body because of 1XX response.

fetch_204
Fetch no body (204) (INFO) beresp with no body because of 204 response.

fetch_304
Fetch no body (304) (INFO) beresp with no body because of 304 response.

fetch_failed
Fetch failed (all causes) (INFO) beresp fetch failed.

fetch_no_thread
Fetch failed (no thread) (INFO) beresp fetch failed, no thread available.

pools
Number of thread pools (INFO) Number of thread pools. See also parameter thread_pools. NB: Presently pools cannot be removed once created.

threads
Total number of threads (INFO) Number of threads in all pools. See also parameters thread_pools, thread_pool_min and thread_pool_max.

threads_limited
Threads hit max (INFO) Number of times more threads were needed, but limit was reached in a thread pool. See also parameter thread_pool_max.

threads_created
Threads created (INFO) Total number of threads created in all pools.

threads_destroyed
Threads destroyed (INFO) Total number of threads destroyed in all pools.

threads_failed
Thread creation failed (INFO) Number of times creating a thread failed. See VSL::Debug for diagnostics. See also parameter thread_fail_delay.

thread_queue_len
Length of session queue (INFO) Length of session queue waiting for threads. NB: Only updates once per second. See also parameter queue_max.

busy_sleep
Number of requests sent to sleep on busy objhdr (INFO) Number of requests sent to sleep without a worker thread because they found a busy object.

busy_wakeup
Number of requests woken after sleep on busy objhdr (INFO) Number of requests taken of the busy object sleep list and and rescheduled.

sess_queued
Sessions queued for thread (INFO) Number of times session was queued waiting for a thread. See also parameter queue_max.

sess_dropped
sessions dropped for thread (INFO) Number of times session was dropped because the queue were too long already. See also parameter queue_max.

n_object
object structs made (INFO) Number of object structs made

n_vampireobject
unresurrected objects (DIAG) Number of unresurrected objects

n_objectcore
objectcore structs made (INFO) Number of objectcore structs made

n_objecthead
objecthead structs made (INFO) Number of objecthead structs made

n_waitinglist
waitinglist structs made (DEBUG) Number of waitinglist structs made

n_backend
Number of backends (INFO) Number of backends known to us.

n_expired
Number of expired objects (INFO) Number of objects that expired from cache because of old age.

n_lru_nuked
Number of LRU nuked objects (INFO) How many objects have been forcefully evicted from storage to make room for a new object.

n_lru_moved
Number of LRU moved objects (DIAG) Number of move operations done on the LRU list.

losthdr
HTTP header overflows (INFO)

s_sess
Total sessions seen (INFO)

s_req
Total requests seen (INFO)

s_pipe
Total pipe sessions seen (INFO)

s_pass
Total pass-ed requests seen (INFO)

s_fetch
Total backend fetches initiated (INFO)

s_synth
Total synthethic responses made (INFO)

s_req_hdrbytes
Request header bytes (INFO) Total request header bytes received

s_req_bodybytes
Request body bytes (INFO) Total request body bytes received

s_resp_hdrbytes
Response header bytes (INFO) Total response header bytes transmitted

s_resp_bodybytes
Response body bytes (INFO) Total response body bytes transmitted

s_pipe_hdrbytes
Pipe request header bytes (INFO) Total request bytes received for piped sessions

s_pipe_in
Piped bytes from client (INFO) Total number of bytes forwarded from clients in pipe sessions

s_pipe_out
Piped bytes to client (INFO) Total number of bytes forwarded to clients in pipe sessions

sess_closed
Session Closed (INFO)

sess_pipeline
Session Pipeline (INFO)

sess_readahead
Session Read Ahead (INFO)

sess_herd
Session herd (DIAG)

shm_records
SHM records (DIAG)

shm_writes
SHM writes (DIAG)

shm_flushes
SHM flushes due to overflow (DIAG)

shm_cont
SHM MTX contention (DIAG)

shm_cycles
SHM cycles through buffer (DIAG)

sms_nreq
SMS allocator requests (DIAG)

sms_nobj
SMS outstanding allocations (DIAG)

sms_nbytes
SMS outstanding bytes (DIAG)

sms_balloc
SMS bytes allocated (DIAG)

sms_bfree
SMS bytes freed (DIAG)

backend_req
Backend requests made (INFO)

n_vcl
Number of loaded VCLs in total (INFO)

n_vcl_avail
Number of VCLs available (DIAG)

n_vcl_discard
Number of discarded VCLs (DIAG)

bans
Count of bans (INFO) Number of all bans in system, including bans superseded by newer bans and bans already checked by the ban-lurker.

bans_completed
Number of bans marked ‘’completed’’ (DIAG) Number of bans which are no longer active, either because they got checked by the ban-lurker or superseded by newer identical bans.

bans_obj
Number of bans using obj.* (DIAG) Number of bans which use obj.* variables. These bans can possibly be washed by the ban-lurker.

bans_req
Number of bans using req.* (DIAG) Number of bans which use req.* variables. These bans can not be washed by the ban-lurker.

bans_added
Bans added (DIAG) Counter of bans added to ban list.

bans_deleted
Bans deleted (DIAG) Counter of bans deleted from ban list.

bans_tested
Bans tested against objects (lookup) (DIAG) Count of how many bans and objects have been tested against each other during hash lookup.

bans_obj_killed
Objects killed by bans (lookup) (DIAG) Number of objects killed by bans during object lookup.

bans_lurker_tested
Bans tested against objects (lurker) (DIAG) Count of how many bans and objects have been tested against each other by the ban-lurker.

bans_tests_tested
Ban tests tested against objects (lookup) (DIAG) Count of how many tests and objects have been tested against each other during lookup. ‘’ban req.url == foo && req.http.host == bar’‘counts as one in’‘bans_tested’‘and as two in’‘bans_tests_tested’’

bans_lurker_tests_tested
Ban tests tested against objects (lurker) (DIAG) Count of how many tests and objects have been tested against each other by the ban-lurker. ‘’ban req.url == foo && req.http.host == bar’‘counts as one in’‘bans_tested’‘and as two in’‘bans_tests_tested’’

bans_lurker_obj_killed
Objects killed by bans (lurker) (DIAG) Number of objects killed by ban-lurker.

bans_dups
Bans superseded by other bans (DIAG) Count of bans replaced by later identical bans.

bans_lurker_contention
Lurker gave way for lookup (DIAG) Number of times the ban-lurker had to wait for lookups.

bans_persisted_bytes
Bytes used by the persisted ban lists (DIAG) Number of bytes used by the persisted ban lists.

bans_persisted_fragmentation
Extra bytes in persisted ban lists due to fragmentation (DIAG) Number of extra bytes accumulated through dropped and completed bans in the persistent ban lists.

n_purges
Number of purge operations executed (INFO)

n_obj_purged
Number of purged objects (INFO)

exp_mailed
Number of objects mailed to expiry thread (DIAG) Number of objects mailed to expiry thread for handling.

exp_received
Number of objects received by expiry thread (DIAG) Number of objects received by expiry thread for handling.

hcb_nolock
HCB Lookups without lock (DEBUG)

hcb_lock
HCB Lookups with lock (DEBUG)

hcb_insert
HCB Inserts (DEBUG)

esi_errors
ESI parse errors (unlock) (DIAG)

esi_warnings
ESI parse warnings (unlock) (DIAG)

vmods
Loaded VMODs (INFO)

n_gzip
Gzip operations (INFO)

n_gunzip
Gunzip operations (INFO)

vsm_free
Free VSM space (DIAG) Number of bytes free in the shared memory used to communicate with tools like varnishstat, varnishlog etc.

vsm_used
Used VSM space (DIAG) Number of bytes used in the shared memory used to communicate with tools like varnishstat, varnishlog etc.

vsm_cooling
Cooling VSM space (DEBUG) Number of bytes which will soon (max 1 minute) be freed in the shared memory used to communicate with tools like varnishstat, varnishlog etc.

vsm_cooling
Cooling VSM space (DEBUG) Number of bytes which will soon (max 1 minute) be freed in the shared memory used to communicate with tools like varnishstat, varnishlog etc.

vsm_overflow
Overflow VSM space (DIAG) Number of bytes which does not fit in the shared memory used to communicate with tools like varnishstat, varnishlog etc.

vsm_overflowed
Overflowed VSM space (DIAG) Total number of bytes which did not fit in the shared memory used to communicate with tools like varnishstat, varnishlog etc.

Lock Counters

creat
Created locks (DEBUG)

destroy
Destroyed locks (DEBUG)

locks
Lock Operations (DEBUG)

Per Malloc Storage Counters

c_req
Allocator requests (INFO)

c_fail
Allocator failures (INFO)

c_bytes
bytes allocated (INFO)

c_freed
Bytes freed (INFO)

g_alloc
Allocations outstanding (INFO)

g_bytes
Bytes outstanding (INFO)

g_space
Bytes available (INFO)

Per File Storage Counters

c_req
Allocator requests (INFO)

c_fail
Allocator failures (INFO)

c_bytes
Bytes allocated (INFO)

c_freed
Bytes freed (INFO)

g_alloc
Allocations outstanding (INFO)

g_bytes
Bytes outstanding (INFO)

g_space
Bytes available (INFO)

g_smf
N struct smf (INFO)

g_smf_frag
N small free smf (INFO)

g_smf_large
N large free smf (INFO)

Per Backend Counters

vcls
VCL references (DEBUG)

happy
Happy health probes (INFO)

bereq_hdrbytes
Request header bytes (INFO) Total backend request header bytes sent

bereq_bodybytes
Request body bytes (INFO) Total backend request body bytes sent

beresp_hdrbytes
Response header bytes (INFO) Total backend response header bytes received

beresp_bodybytes
Response body bytes (INFO) Total backend response body bytes received

pipe_hdrbytes
Pipe request header bytes (INFO) Total request bytes sent for piped sessions

pipe_out
Piped bytes to backend (INFO) Total number of bytes forwarded to backend in pipe sessions

pipe_in
Piped bytes from backend (INFO) Total number of bytes forwarded from backend in pipe sessions