This guide is for engineers and application developers preparing for the Prometheus Certified Associate (PCA). Work from observability foundations through metric collection and querying to alerting and dashboards. Each concept explains a practical distinction, resolves an original example, and identifies a specific mistake to avoid.
Observability Concepts
1. Monitoring questions and observability evidence
Monitoring checks selected conditions, while observability uses emitted evidence to investigate system behavior. A useful investigation connects a user-visible symptom to measurements and then to supporting logs or traces. Prometheus supplies metric evidence; it does not reconstruct every request or internal state.
Worked example: Checkout latency rises while request volume stays steady. Metrics establish the change; traces then show that payment-service spans account for the added delay.
Mistake to avoid: Treating a healthy CPU graph as proof that every user operation is healthy.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
2. Metrics as numerical summaries
Metrics summarize quantities over time, such as completed requests, active connections, or observed durations. They support trends and comparisons with relatively compact data. Their usefulness depends on what was measured and which dimensions were retained; aggregation necessarily discards some individual-event detail.
Worked example: A service reports 240 completed requests during a minute. That establishes an average throughput of four requests per second, but does not identify any particular request.
Mistake to avoid: Expecting an aggregate request counter to explain an individual customer's failed transaction.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
3. Logs and events preserve discrete details
Logs record discrete occurrences with contextual fields or messages. Events can describe meaningful state changes, such as a deployment or leader election. These records help explain metric changes, but counting log lines is meaningful only when logging behavior reliably represents the activity being measured.
Worked example: An error-rate graph changes immediately after a deployment event. Application logs identify a newly rejected input format, connecting the numerical symptom to a concrete cause.
Mistake to avoid: Equating fewer error logs with fewer failures after logging was reduced or sampled.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
4. Traces and spans describe request paths
A trace connects spans belonging to one operation across components. Each span describes a timed unit of work and may reference a parent. Overlapping spans represent concurrent work, so adding all span durations generally does not equal the request's elapsed duration.
Worked example: Two child spans each last 200 milliseconds and run simultaneously. Their parent can finish in roughly 200 milliseconds plus overhead, rather than requiring 400 milliseconds.
Mistake to avoid: Summing concurrent span durations and reporting the result as end-to-end latency.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
5. Pull collection and push collection
In pull collection, a collector initiates requests to metric endpoints. In push collection, a producer sends observations to a receiver. Prometheus normally scrapes targets, giving it direct evidence about scrape success. Push-based arrangements require separate reasoning about producer freshness and lifecycle.
Worked example: A running web service exposes metrics for Prometheus to scrape. A short-lived batch task may need an intermediary because it can finish before a scrape occurs.
Mistake to avoid: Assuming a successful scrape of an intermediary proves that its producer recently ran.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
6. Service discovery identifies collection targets
Service discovery supplies candidate target addresses and metadata as infrastructure changes. It solves the problem of finding endpoints, rather than proving they are reachable or exporting useful metrics. Selection and relabeling determine which discovered targets enter the scrape configuration.
Worked example: Discovery finds three application replicas. Two expose the expected metrics port; the third has an incorrect address and fails its scrape despite being successfully discovered.
Mistake to avoid: Treating discovery success and scrape success as the same condition.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
7. SLIs need explicit numerators and denominators
A service level indicator measures a defined aspect of service behavior. For a success ratio, specify eligible requests, successful outcomes, and the measurement window. Exclusions must be deliberate and consistent. An empty denominator does not establish that the service delivered successful requests.
Worked example: Of 10,000 eligible requests, 9,900 succeed. The success SLI is 9,900 divided by 10,000, or 99%. Excluded health checks belong in neither count.
Mistake to avoid: Including health checks only in the denominator and thereby distorting the success ratio.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
8. SLOs and error budgets
A service level objective sets a target for an SLI over a stated window. For an event-based success objective, the error budget is the permitted fraction of unsuccessful eligible events. Calculate against the same population and window used by the objective.
Worked example: A 99.9% success objective permits 100 failures among 100,000 requests. If 40 have failed, 60 failures remain within that event-based budget.
Mistake to avoid: Applying a monthly objective to a five-minute measurement without accounting for the different window.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
9. SLAs and internal objectives serve different purposes
A service level agreement describes service commitments and their agreed consequences. An SLO is a measurable operating objective, and an SLI supplies its measurement. The terms may use different windows or exclusions, so one operational graph cannot establish compliance without matching the agreement's definitions.
Worked example: A team targets 99.95% successful requests internally while its agreement specifies 99.9% under a particular measurement method. The stricter internal objective provides operating margin.
Mistake to avoid: Assuming an internal SLO automatically has the same terms as a customer agreement.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
10. Request rate, errors, and duration
For request-serving systems, rate, errors, and duration describe demand, unsuccessful outcomes, and user waiting time. Interpret them together: an error count alone lacks traffic context, and average latency can hide a slow tail. Define errors according to the operation's intended outcome.
Worked example: Failures rise from 10 to 20 per minute while traffic rises from 1,000 to 4,000 requests. The failure ratio improves from 1% to 0.5%.
Mistake to avoid: Calling the service less reliable solely because its absolute failure count increased.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
11. Utilization, saturation, and resource errors
Resource analysis distinguishes utilization, saturation, and errors. Utilization measures how busy a resource is; saturation indicates work waiting beyond available capacity; errors indicate failed resource operations. These signals explain bottlenecks more effectively together than a utilization percentage interpreted in isolation.
Worked example: CPU utilization is moderate, but disk requests queue and storage latency rises. The evidence points toward storage saturation rather than insufficient CPU capacity.
Mistake to avoid: Adding CPU capacity because it is the only resource shown on the dashboard.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
Prometheus Fundamentals
12. The collection, query, and notification architecture
Prometheus scrapes metric endpoints, stores samples, evaluates queries, and evaluates rules. Exporters expose measurements from other systems. Alertmanager receives alerts and manages notifications. A dashboard queries data for presentation. Knowing these boundaries helps locate failures without blaming every symptom on collection.
Worked example: A graph updates and an alert is firing, but no message arrives. Investigate the alert delivery path and Alertmanager configuration rather than assuming scraping stopped.
Mistake to avoid: Expecting an exporter to store history or deliver alert notifications.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
13. Time series identity
A Prometheus time series is identified by its metric name and complete label set. Samples supply values at timestamps for that identity. Changing a label value creates another series; it does not update a descriptive field on the existing series.
Worked example: queue_depth{queue="orders"} and queue_depth{queue="returns"} are separate series. Changing the queue label from orders to purchases starts another identity.
Mistake to avoid: Treating label changes as harmless metadata edits when reasoning about historical continuity.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
14. Cardinality grows through label combinations
Cardinality is the number of distinct series, not simply the number of metric names. Independent label dimensions can multiply the possible series count. Actual combinations determine realized cardinality, while new label values over time add further storage and processing work.
Worked example: One metric with four methods, five status classes, and three regions can produce 4 × 5 × 3 = 60 combinations if all occur.
Mistake to avoid: Estimating collection cost from metric names while ignoring their label combinations.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
15. Sample timestamps and stale series
Sample timestamps describe when recorded observations belong in time. Instant queries select eligible recent samples under Prometheus's freshness and staleness rules. A series that becomes stale can disappear from an instant result even though historical samples remain available to range-based analysis.
Worked example: A removed replica no longer contributes to a current queue-depth query, while a historical graph still shows its measurements before removal.
Mistake to avoid: Interpreting a disappeared series as a newly observed value of zero.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
16. Scrape jobs and target instances
A scrape job groups targets collected under a common configuration. The job and instance labels normally identify the configured collection group and individual target. Separate jobs are useful when endpoints need different collection settings, even if the applications expose similar metric names.
Worked example: A web job scrapes two replicas with different instance addresses. Aggregating by job combines their measurements; retaining instance separates the replicas.
Mistake to avoid: Assuming one job label identifies a single running process.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
17. Scrape intervals and timeouts
The scrape interval controls how often collection is attempted; the timeout bounds how long an attempt may take. A timeout cannot exceed its scrape interval. Shorter intervals produce more samples, but cannot recover events an endpoint never measured or fix an unreachable target.
Worked example: With a 30-second interval and a 10-second timeout, an endpoint taking 12 seconds fails collection. Increasing collection frequency alone does not solve its response-time problem.
Mistake to avoid: Confusing the collection interval with the maximum duration allowed for one scrape.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
18. Reading the text exposition format
Text exposition associates metric names and labels with numerical samples. HELP and TYPE lines describe a metric family; sample lines contain its observations. Label values require correct quoting and escaping. An endpoint response must be parseable, not merely look like readable diagnostic text.
Worked example: A TYPE declaration identifies active_sessions as a gauge, and active_sessions{region="west"} 7 reports seven sessions for that label set.
Mistake to avoid: Returning arbitrary JSON from a metrics endpoint and expecting the text-format scraper to interpret it.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
19. What the up metric establishes
Prometheus generates up for scrape targets: a successful scrape produces 1, and a failed scrape produces 0. This measures collection success. It does not establish business availability, dependency health, or correctness of every exposed measurement. Removed targets require separate absence reasoning.
Worked example: up is 1 for a service whose checkout requests all fail. The metrics endpoint works, but the business operation is unhealthy.
Mistake to avoid: Using up alone as the success indicator for a user-facing transaction.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
20. Target relabeling and metric relabeling
Target relabeling acts on discovered targets before scraping and can select targets or construct labels. Metric relabeling acts on scraped samples before ingestion. These stages have different costs: dropping samples after a scrape does not avoid fetching and processing the endpoint response.
Worked example: Dropping a discovered debug target prevents its scrape. Dropping a debug metric through metric relabeling still requires scraping the target that exposes it.
Mistake to avoid: Expecting metric relabeling to prevent network requests to an unwanted target.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
21. Local storage and retention
Prometheus's local time-series database retains collected samples for querying, subject to configured retention and available storage. More series and more frequent samples increase demand. Retention determines historical availability; it is not a guarantee that all potentially useful data was collected.
Worked example: Halving the scrape interval approximately doubles sample volume for the same series population. Historical queries still cannot show periods before those series were collected.
Mistake to avoid: Assuming longer retention compensates for missing instrumentation or failed scrapes.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
22. Federation collects selected upstream series
Federation lets a Prometheus server scrape selected series exposed by another Prometheus server. It can support hierarchical aggregation and selected cross-system views. It is collection of chosen observations, rather than automatic replication of the upstream server's entire historical database.
Worked example: A regional server exposes aggregate request rates for a central server. The central view retains those aggregates without collecting every regional per-instance series.
Mistake to avoid: Expecting federation to transfer all past samples or preserve every original detail.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
23. Monitoring data has measurement limits
Scraped metrics provide sampled observations and application-maintained aggregates. Collection failures, process restarts, missing dimensions, and short-lived states can limit interpretation. Prometheus is useful for operational monitoring, but metric history alone does not guarantee an exact audit of every event.
Worked example: A queue briefly reaches 80 items between scrapes that both observe zero. A queue-depth gauge misses the peak unless the application also records suitable peak or event measurements.
Mistake to avoid: Concluding that an unobserved short-lived state never occurred.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
Instrumentation and Exporters
24. Choose metric types by measurement behavior
Choose a counter for accumulated events, a gauge for a current quantity, and a histogram or summary for distributions of observations. The choice determines meaningful queries. A changing value is not automatically a gauge: an event total changes while remaining a counter.
Worked example: Completed downloads use a counter, downloads currently running use a gauge, and download durations use a distribution metric. Each answers a different operational question.
Mistake to avoid: Using one current-value gauge to represent both completed events and active work.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
25. Counters accumulate and can reset
Counters accumulate nonnegative increments and normally increase until a reset, commonly after a process restart. Raw counter differences can become negative across resets. Counter-aware functions account for observed decreases, although sampling can still miss activity between observations or multiple unseen resets.
Worked example: Observed values 40, 55, 0, and 8 show 15 increments before the reset and eight afterward: 23 observed increments in total.
Mistake to avoid: Subtracting 40 from 8 and interpreting the result as negative event production.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
26. Gauges describe current state
Gauges can rise or fall and may represent quantities such as temperature, queue depth, or available capacity. Set them from authoritative state when possible. Incrementing and decrementing a gauge requires balanced updates, including failure paths, to avoid gradually drifting away from reality.
Worked example: An in-flight request gauge rises from three to four when work starts and returns to three when that work ends, including when it fails.
Mistake to avoid: Decrementing only successful requests and leaving failed requests permanently counted as active.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
27. Classic histograms use cumulative buckets
A classic histogram records observations in cumulative buckets identified by upper bounds, together with count and sum series. An observation contributes to every bucket whose upper bound includes it. Bucket selection determines which thresholds and distribution estimates can be resolved usefully.
Worked example: Durations of 0.08 and 0.30 seconds produce counts of 1, 2, and 2 for bounds 0.1, 0.5, and +Inf, with count 2 and sum 0.38.
Mistake to avoid: Treating cumulative bucket counts as counts for mutually exclusive intervals.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
28. Summary quantiles are calculated at the producer
A summary can expose quantile estimates calculated within the instrumented process, plus observation count and sum. Quantile configuration belongs to that producer's implementation. Per-instance quantiles generally cannot be combined into an accurate fleet-wide quantile; count and sum can support an aggregate mean.
Worked example: Two replicas report 95th-percentile durations of 0.2 and 0.8 seconds. Their average, 0.5 seconds, does not establish the combined 95th percentile.
Mistake to avoid: Averaging summary quantiles across replicas and labeling the result a service percentile.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
29. Client libraries manage metric collection
Client libraries provide metric objects, registries, and exposition support. Create and register collectors deliberately, then update or observe them at the measurement point. Repeatedly registering the same metric during request handling can cause duplicate-registration failures or unnecessary work.
Worked example: A service creates its request counter during initialization and increments the existing labeled child for each completed request. It does not construct a new collector for every request.
Mistake to avoid: Registering an identical collector inside a frequently executed handler.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
30. Names communicate quantity and units
Metric names should communicate a consistent quantity, with clear base units and conventional suffixes where appropriate. Keep one meaning and unit per metric family. Descriptive names reduce query mistakes, especially when different services expose similarly named measurements.
Worked example: request_duration_seconds exposes seconds, while completed_jobs_total expresses an accumulating event count. A duration of 250 milliseconds is recorded as 0.25 seconds in the former.
Mistake to avoid: Recording milliseconds in a metric whose name promises seconds.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
31. Bound labels at the instrumentation point
Labels should describe useful, bounded dimensions such as method or normalized route. Values drawn from user IDs, arbitrary paths, or request identifiers can create unbounded series growth. Preserve individual identifiers in logs or traces rather than making each identifier a metric dimension.
Worked example: Requests to /orders/17 and /orders/42 share route="/orders/{id}". The normalized route supports per-endpoint monitoring without creating a series for every order.
Mistake to avoid: Labeling request metrics with the complete URL when it contains changing identifiers.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
32. Exporters translate external measurements
An exporter collects measurements from a system and exposes them in a format Prometheus can scrape. Exporter reachability and successful collection from the underlying system are separate conditions. Interpret exporter-specific collection health alongside the target's scrape status.
Worked example: An exporter endpoint responds successfully, but its database connection fails. up can remain 1 while a collection-error metric indicates missing database measurements.
Mistake to avoid: Assuming a reachable exporter proves that its underlying system was measured successfully.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
33. Define exactly when an event is counted
Instrumentation must define the event boundary and observation population. Count attempts, completions, or successful outcomes according to the question being answered. Duration measurements need consistent start and end points. Retries and failures should follow the same documented definitions across producers.
Worked example: One user operation makes three backend attempts before succeeding. An attempts counter increases by three, while a completed user-operation counter increases by one.
Mistake to avoid: Dividing successful user operations by backend attempts and presenting the result as user-operation success.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
PromQL
34. Instant vectors and range vectors
An instant-vector selector supplies one eligible sample per matching series at an evaluation time. A range-vector selector supplies samples within a preceding window for each series. Functions have specific input types; a graph's repeated evaluations do not make every expression a range-vector expression.
Worked example: active_jobs returns an instant vector. completed_jobs_total[5m] returns a range vector, which rate can transform into an instant vector of per-second estimates.
Mistake to avoid: Passing an instant vector to a function that requires a range vector.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
35. Label matchers select the intended population
Label matchers support equality, inequality, and regular expressions. Prometheus regex label matching is fully anchored. Matchers that accept an empty value can also match series without that label, so include a positive population selector when exclusions alone would be too broad.
Worked example: http_requests_total{job="api",status=~"2.."} selects three-character 2xx statuses. Using status!="500" also permits series without a status label.
Mistake to avoid: Assuming a negative matcher guarantees that the matched label exists.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
36. Rate estimates counter increments per second
rate estimates a counter's average per-second increase over a range, accounting for observed resets and extrapolating to the window boundaries. Apply rate to individual counter series before aggregating, so each series retains the evidence needed to recognize its own resets.
Worked example: A steadily increasing request counter gains 300 requests over 60 seconds, giving about five requests per second. sum(rate(http_requests_total[5m])) combines individual rates.
Mistake to avoid: Aggregating raw counters first and allowing one replica's reset to be obscured by another's increase.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
37. Increase expresses a counter change over a window
increase expresses an estimated counter increase across the selected window. It uses counter-reset handling and boundary extrapolation, so results can be fractional even when individual increments are integers. It is useful for window totals; rate expresses the same underlying change per second.
Worked example: A steady rate of two completed jobs per second corresponds to an estimated increase of 600 jobs over a five-minute window.
Mistake to avoid: Demanding an integer result or treating an extrapolated increase as an exact event ledger.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
38. Irate emphasizes the most recent counter change
irate estimates a per-second counter rate from the last two samples in a range. It is sensitive to recent bursts, unlike rate's wider-window average. The selected range provides candidate samples; it does not turn irate into an average across the entire range.
Worked example: The last two counter samples are 10 seconds apart and rise from 120 to 150. irate estimates three events per second from that pair.
Mistake to avoid: Interpreting irate(metric[10m]) as the average rate across ten minutes.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
39. Gauge changes and gauge derivatives
delta estimates a gauge's net change over a window, while deriv estimates its per-second slope using linear regression. These functions allow decreases without interpreting them as counter resets. Choose net change for an amount gained or lost and slope for a trend's speed.
Worked example: A gauge rises steadily from 20 to 35 over 300 seconds. Its net change is approximately 15, and its slope is 0.05 units per second.
Mistake to avoid: Applying a counter-rate function to a gauge whose ordinary decreases are meaningful.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
40. Aggregate across dimensions deliberately
Aggregation combines series at one evaluation time. by retains listed grouping labels; without removes listed labels while retaining the others. Choose dimensions based on the question, and verify that unexpected labels will not accidentally preserve distinctions you intended to combine.
Worked example: Rates of two and three requests per second from replicas in the same service become five with sum by (service). Keeping instance instead produces separate replica totals.
Mistake to avoid: Dropping the service label when distinct services must remain separately interpretable.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
41. Aggregate samples across time
Functions such as avg_over_time and max_over_time summarize samples within each series' range. They do not aggregate different label sets. Average-over-time gives equal weight to samples rather than weighting them by elapsed time, which matters when sample spacing is irregular.
Worked example: Queue-depth samples of 10, 20, and 30 produce avg_over_time of 20 and max_over_time of 30 for that series.
Mistake to avoid: Using sum_over_time on a counter and interpreting the sum of cumulative readings as completed events.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
42. Arithmetic must preserve meaningful units
PromQL arithmetic operates on values but does not check physical units. Ratios require compatible populations and windows; conversions require the correct factor. Dividing an event rate by another comparable event rate produces a dimensionless fraction, which can then be multiplied by 100.
Worked example: An error rate of two requests per second divided by a total rate of 80 gives 0.025. Multiplying by 100 displays 2.5%.
Mistake to avoid: Dividing an error count by a per-second request rate and labeling the result an error percentage.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
43. Vector matching controls binary calculations
Binary calculations between instant vectors require matching label sets, normally excluding the metric name. on restricts matching to named labels; ignoring excludes named labels from matching. Unmatched series do not acquire an automatic counterpart or silently become zero.
Worked example: A request-rate series labeled service="billing",instance="a" will not match a service-only capacity series by default. Matching on service addresses that label difference.
Mistake to avoid: Assuming arithmetic aligns series by their visual order or similar metric names.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
44. Many-to-one matching needs an unambiguous side
group_left permits multiple left-side series to match one right-side series for arithmetic or comparison; group_right handles the reverse. The one side must be unique for the matching labels. Group modifiers do not legitimize ambiguous many-to-many arithmetic.
Worked example: Per-method request rates divide by one service capacity using on(service) group_left. Two capacity series for the same service would make that intended one-side match ambiguous.
Mistake to avoid: Adding a group modifier without checking uniqueness on the side that must contain one match.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
45. Comparisons filter unless bool is requested
Comparison operators normally retain matching samples whose comparisons are true and discard those that are false. With bool, matching comparisons produce 1 or 0 instead. This distinction matters in alert expressions, where the presence of a result series makes that alert instance active.
Worked example: With up values 1 and 0, up == 0 returns only the failed target. up == bool 0 returns both targets, valued 0 and 1.
Mistake to avoid: Using a bool comparison in an alert while assuming zero-valued results cannot activate it.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
46. Set operators work with series membership
and, or, and unless operate on instant-vector membership under label matching, rather than calculating truth from numerical values. and retains matching left-side series; unless retains unmatched left-side series. or forms a union, keeping left-side samples where matches exist.
Worked example: A and on(service) B retains service samples from A when B contains matching service labels, even if the matching B sample has value zero.
Mistake to avoid: Reading a zero-valued sample on the right of and as a Boolean false condition.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
47. Classic histogram quantiles require bucket boundaries
histogram_quantile estimates a quantile from classic histogram bucket rates, retaining the le label during bucket aggregation. The estimate depends on bucket placement and interpolation within the selected bucket. It is not an exact reconstruction of individual observations.
Worked example: For 100 observations, cumulative counts are 80 at 0.1 seconds and 95 at 0.5 seconds. The 90th percentile estimates 0.1 + (10/15 × 0.4), about 0.367 seconds.
Mistake to avoid: Removing le before calculating a classic histogram quantile.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
48. Distribution means use sums and counts
For nonnegative duration observations, divide the rate of the distribution's sum by the rate of its count to obtain a mean duration. Aggregate sums and counts separately for a combined mean. Averaging instance means gives the wrong weighting when observation counts differ.
Worked example: One replica records 10 seconds across 10 requests; another records 90 seconds across 30. The combined mean is 100/40 = 2.5 seconds.
Mistake to avoid: Averaging the replica means of one and three seconds to claim a combined mean of two.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
49. Timestamp values and sample timestamps differ
A timestamp metric can store an event's Unix time as its value. time() minus that value measures event age. timestamp(metric) instead returns the sample's timestamp, answering when that measurement was recorded. These quantities can differ greatly for a regularly scraped but unchanged event timestamp.
Worked example: The evaluation time is 12,000 and last_success_timestamp_seconds holds 11,910. The last successful event is 90 seconds old, regardless of its latest scrape time.
Mistake to avoid: Using the newest scrape timestamp as evidence that a batch job recently succeeded.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
50. Missing series need explicit absence reasoning
An empty query result differs from a zero-valued sample. absent can detect an empty instant-vector expression; absent_over_time can detect a range with no samples. Define the expected population carefully, because absence checks cannot infer every expected target from data that never existed.
Worked example: absent(up{job="billing"}) signals when no matching up series exists. Existing billing targets with up equal to zero require a separate failed-scrape condition.
Mistake to avoid: Assuming an absence expression also detects every present but unsuccessful scrape.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
Alerting & Dashboarding
51. Dashboard ratios need visible populations
A dashboard should make the measured population, units, and time window understandable. Counts show volume, while ratios show relative outcomes. Presenting both can prevent a large service from appearing less reliable merely because it handles more traffic and therefore produces more failures.
Worked example: Route A has 20 failures in 100 requests, or 20%. Route B has 50 in 10,000, or 0.5%. A failure-count panel alone hides Route A's worse reliability.
Mistake to avoid: Ranking reliability by failure counts without showing the corresponding request volume.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
52. Choose displays that match the measurement
Time-series panels reveal trends; current-value panels summarize an evaluation time. Display counters as rates or increases when activity is the question, and label duration units clearly. Query windows and display resolution affect how much short-term variation remains visible.
Worked example: A raw completed-request counter continually climbs. A rate graph instead shows demand rising from four to nine requests per second and later falling.
Mistake to avoid: Interpreting the upward slope of every cumulative counter as continuously worsening system behavior.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
53. Recording rules store reusable query results
A recording rule periodically evaluates an expression and stores its output under a new metric name. It can make repeated dashboard or alert queries simpler and cheaper. Recorded results retain the expression's chosen dimensions; discarded dimensions cannot be recovered by querying the recorded metric.
Worked example: A rule records sum by (job) (rate(http_requests_total[5m])). Dashboards can reuse per-job rates, but cannot derive per-instance rates from that recorded result.
Mistake to avoid: Aggregating away a label in a recording rule and expecting it to remain available downstream.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
54. Alert expressions identify actionable conditions
An alert rule evaluates an expression and creates active alert instances for returned series. Choose conditions that support a meaningful response and preserve the labels needed to identify the affected population. A comparison without bool usually expresses a threshold by filtering.
Worked example: queue_depth{queue="payments"} > 100 returns the payments series when its depth is 140. The retained queue label identifies which backlog needs investigation.
Mistake to avoid: Using a bare nonzero metric as though alert activation depends on its value being true.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
55. Pending and firing states express persistence
An alert's for duration requires its condition to remain active for that alert instance before firing. During that period it is pending. If the condition disappears, that pending progression ends. Label changes can create another instance with its own persistence history.
Worked example: A rule with for: 5m sees a condition for three minutes, then a healthy evaluation. That episode does not satisfy five minutes of continuous activity.
Mistake to avoid: Adding several separate short episodes together to satisfy a continuous for duration.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
56. Alert labels identify; annotations explain
Labels identify alert instances and support routing, grouping, and matching. Annotations carry explanatory text such as a summary or diagnostic context. Frequently changing observations belong in annotations rather than identity labels, so one ongoing condition does not become many changing instances.
Worked example: Keep service="checkout" and severity="warning" as labels. Put the observed queue depth of 143 in an annotation that can update as the backlog changes.
Mistake to avoid: Putting the current measurement in a label and continually changing the alert's identity.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
57. Alertmanager routes alerts by labels
Alertmanager's routing configuration directs alerts to receivers using label matchers and a routing tree. Evaluate how an alert's actual labels interact with that tree, including fallback behavior and any continuation settings. A firing alert alone does not establish which receiver will handle it.
Worked example: An alert labeled team="payments" can match a payments route. If the label is absent, it follows another matching path or the configured fallback instead.
Mistake to avoid: Expecting a team route to match because the service's ownership is known outside the alert labels.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
58. Grouping reduces notification duplication
Alertmanager groups alerts according to configured labels, allowing related instances to share notifications. Grouping changes notification organization, not the number of firing conditions. Choose grouping labels that keep the message understandable while avoiding a separate notification for every low-level instance.
Worked example: Five failing replicas share service="search". Grouping by service can present their alerts together, while grouping by instance separates them into five groups.
Mistake to avoid: Assuming grouping resolves duplicate alert conditions or reduces the underlying number of firing instances.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
59. Inhibition suppresses related notifications
An inhibition rule suppresses notifications for matching target alerts when matching source alerts are present, subject to configured label relationships. It represents a dependency or causal hierarchy. Constrain matching carefully so a failure in one scope does not hide unrelated failures elsewhere.
Worked example: A cluster-unavailable alert inhibits instance-unreachable alerts only when their cluster labels match. An outage in cluster east does not suppress alerts for cluster west.
Mistake to avoid: Using overly broad inhibition that hides independent failures in other clusters or services.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
60. Silences mute notifications within a defined scope
A silence suppresses notifications for alerts matching its conditions during a defined period. It does not make the monitored condition healthy or stop rule evaluation. Narrow matchers preserve visibility of unrelated problems, and the alert can remain firing while notification delivery is muted.
Worked example: A maintenance silence matches service="reports" in environment="staging". Production reports alerts remain outside its scope, and staging conditions can still appear as firing.
Mistake to avoid: Treating a silenced alert as resolved or assuming a silence disables metric collection.
Source reference: Prometheus Certified Associate (PCA) - Linux Foundation - Education
Sources
Sources checked:
Browse all study guides