Monitoring Concepts

CPU, Memory, Disk, and Network usage — numeric values measured repeatedly over time. Metrics give you the big picture of how healthy your server and application are at any moment. They are the first signal that something is wrong.

• top — live CPU and memory view
• free -h — memory and swap
• df -h — disk usage per partition
• Grafana — visual dashboards for all metrics

Detect server overload. When CPU is at 94% or disk hits 100%, metrics tell you before a single client complaint arrives. This is the difference between proactive and reactive support.

Application and system logs — timestamped records of every event your software and OS write. Logs tell you exactly what happened and when, with the precise error message. Where metrics show a number, logs show the story behind it.

• tail -f app.log — live log stream
• grep "ERROR" app.log — search by pattern
• ELK Stack — Elasticsearch, Logstash, Kibana
• Loki — lightweight log aggregation

Find errors. When a metric alert fires, logs tell you the exact error — DB_CONNECTION_TIMEOUT, NullPointerException, SOCKET_TIMEOUT — so you know what to fix, not just that something broke.

Threshold-based notifications — rules that fire automatically when a metric crosses a defined limit. Alerts are the bridge between monitoring and human response. Without them you must watch screens constantly. With them the system watches for you.

• Alertmanager — routes and manages alerts
• email — low-urgency notification channel
• Slack webhook — team channel alerts
• PagerDuty — on-call phone escalation

Immediate action. A CRITICAL alert at 3 AM wakes the on-call engineer before any client notices. A WARNING alert during office hours creates a Slack message to investigate at the next opportunity. Severity routing means the right person gets the right urgency.

Service availability verification — automated checks that confirm your application and its dependencies (DB, MQ, external APIs) are reachable and responding. A health check endpoint returns a structured status telling you the health of every component in one call.

• curl /health — call the health endpoint
• systemctl status — check service status
• ping — basic network reachability
• netstat -an — check open connections

Ensure uptime. When a client reports failures, calling the /health endpoint is the fastest first step — it tells you whether the application, DB, and MQ are all up or which one is broken, in under 2 seconds.

Latency and response monitoring — Application Performance Monitoring tracks how long each API endpoint takes to respond, what the error rate is per endpoint, and how many requests per second each one handles. It watches the software layer, not just the server.

• Grafana APM — per-endpoint performance
• Datadog APM — traces and latency maps
• Jaeger — distributed request tracing
• New Relic — full-stack APM platform

Detect slow APIs. When the system feels slow but CPU and disk look fine, APM shows you that /raast/transfer has p99 latency of 8 seconds while all other endpoints are under 200ms. That specificity makes root cause identification immediate.

Visual status overview — dashboards pull together metrics, logs, and APM data into a single visual screen. Every key number is visible at once — CPU%, error rate, response time, queue depth — colour-coded so you can read system health in under 5 seconds.

• Grafana — most common, highly customisable
• Kibana — log-based dashboards (ELK)
• Datadog — all-in-one with built-in dashboards
• Splunk — enterprise dashboard for log data

Quick decision making. During an S1 bridge call, opening the dashboard gives you everything needed to answer "what is the current status" in seconds. You read from the dashboard — error rate, affected services, when it started — without opening a terminal.

Service reliability targets — SLA (Service Level Agreement) is the contractual commitment to a client on uptime and response time. SLO (Service Level Objective) is the internal target your team works to. Both define what "acceptable" looks like numerically — e.g. 99.9% uptime per month.

• Reports — monthly uptime and incident reports
• Grafana SLO panels — visual SLO tracking
• Incident log — duration of each outage
• Error budget — how much downtime remains

Measure performance. Every incident you handle contributes to or drains the SLA. A 15-minute outage on a 99.9% SLA uses a significant portion of the month's allowed downtime. Knowing the SLA makes every incident feel appropriately urgent.

Root cause analysis — the structured investigation after an incident to find not just what failed but why it failed. A good RCA uses evidence from logs and metrics to produce a timeline, a root cause statement, and next steps that prevent recurrence.

• Logs — timestamps, error messages, build-up
• Metrics — when did numbers change
• Jira — where the RCA is documented
• grep / Kibana — searching evidence

Prevent recurrence. An RCA that says "DB pool exhausted because of a connection leak" leads to a code fix. Without the RCA the same incident happens again in 2 weeks. Good RCAs turn reactive incidents into proactive improvements.