Your Service Is Healthy, but Its Data Isn’t: Rethinking Cloud-Native Health Checks
Some of the most difficult production issues do not start with a failed service. The application is running, the database is reachable, requests are completed successfully and the monitoring dashboard is green. Yet users report that something is wrong because the information they depend on has stopped changing.
I have seen this type of problem in data-intensive applications where information is expected to update continuously. An application can remain technically available while an upstream data source has stopped, processing has fallen behind or the latest information is no longer reaching the consumer.
From an infrastructure perspective, the service is healthy. From the user’s perspective, it may no longer be reliable.
For cloud-native applications that depend on frequently changing data, this creates an important distinction: service health and data health are not always the same thing.
A Green Health Check Can Still Hide a Failure
Consider a common data path:
Producer → Ingestion → Processing → Database → API → Consumer
If the producer stops sending updates, the rest of the architecture does not necessarily fail. The ingestion service can remain available, the database stays reachable and the API can continue returning the last stored value.
Every component can report itself as healthy even though the overall data flow has stopped progressing.
This is where traditional infrastructure monitoring can leave a gap. Knowing that a service can answer a request is important, but it does not tell us whether the information in that response is still current.
Make Data Freshness Part of Health
For applications built around continuously changing information, freshness should be treated as an application-health signal.
The idea is simple: understand how frequently information is expected to change and compare that expectation with the most recent successful update.
If data normally arrives every 30 seconds, an update that is slightly late may not matter. If several expected update cycles pass without anything new arriving, the application should recognize that condition.
The threshold depends on the workload. A real-time stream and an hourly batch process should not share the same definition of stale.
What matters is defining what “current” means instead of assuming that an available service means current data.
Degraded Is a Useful Production State
One mistake is treating every freshness problem as an outage.
If an upstream source becomes delayed, restarting an otherwise healthy service may accomplish nothing. Failing readiness can also remove a functioning application even though the last available information may still have value.
A better approach is to recognize intermediate states. Data can move from healthy to degraded and eventually to stale as the time since the last successful update increases.
Different consumers can respond differently. A dashboard might continue showing the last known value while indicating that it is delayed. A downstream process may reject information beyond a certain age. Support teams may only need an alert after the condition persists beyond the expected tolerance.
This allows the application to degrade predictably rather than turning every upstream delay into an outage.
Monitor the Flow, Not Only the Components
Cloud-native platforms provide extensive visibility into containers, services and infrastructure. Teams routinely monitor CPU, memory, request latency, error rates and availability.
Data-intensive applications need another layer of visibility: Is information still moving through the system?
Several signals are particularly useful:
Last successful update: When did new information arrive most recently?
Data age: How old is the newest information being served?
Expected frequency: How often should an update normally occur?
Processing lag: Is information taking longer than expected to move through the pipeline?
Time in degraded state: Is this a temporary delay or a continuing problem?
Looking at these signals together can narrow an investigation quickly. If an API and databases are healthy but data age continues increasing, engineers have a strong reason to investigate earlier stages of the pipeline instead of troubleshooting a healthy application.
Avoid Turning Freshness Monitoring Into Alert Noise
Once freshness becomes measurable, it is easy to create another problem: too many alerts.
Not every delayed update requires immediate action. Networks experience short interruptions, processing times vary and upstream systems may occasionally deliver information later than expected.
Alerting on every missed interval can quickly train teams to ignore notifications.
A better approach is to consider both severity and duration. A short delay may move the application into a degraded state without generating a high-priority alert. If the condition continues across several expected update cycles, it can escalate to stale and trigger the appropriate response.
Once new information begins flowing and the data path is progressing again, the condition should recover automatically.
The goal is not more alerts, but alerts that tell engineers something useful.
Design Freshness Into the Data Path
Freshness monitoring works best when it is part of application design rather than something added after a production incident.
Data producers should provide timestamps, sequence information or another way for consumers to determine whether information is progressing. Processing layers should make their latest successful activity observable. APIs should provide enough context for consumers to understand how current the information is.
This becomes increasingly important as the architecture grows. When several services exist between the producer and consumer, each component can report itself as healthy while information has already stopped moving somewhere upstream.
The complete data path therefore needs to be treated as part of application health.
Rethink What Healthy Means
Liveness tells us whether a process is alive. Readiness tells us whether a service can accept traffic. Infrastructure monitoring tells us whether the platform is functioning.
For data-dependent applications, there is one more question:
Is the information still current enough to trust?
This does not mean replacing existing cloud-native health checks. It means extending the health model to reflect what users depend on.
A service can remain available and return successful responses long after its information has stopped changing.
For me, one principle stands out:
A healthy service should also know when its data is unhealthy.
