Health Checking

GRPC provides Health Checking Protocol to implement health checks. You can see it’s latest definition here: grpc/health/v1/health.proto.

As you can see from the service definition, Health service should implement one or two methods: simple unary-unary Check method for synchronous checks and more sophisticated unary-stream Watch method to asynchronously wait for status changes. grpclib implements both of them.

grpclib also provides additional functionality to help write health checks, so users don’t have to write a lot of code on their own. It is possible to implement health check in two ways (you can use both ways simultaneously):

  • use ServiceCheck class by providing a callable object which can be called asynchronously to determine check’s status

  • use ServiceStatus class and change it’s status by using set method

ServiceCheck is a simplest and most generic way to implement periodic checks.

ServiceStatus is for a more advanced usage, when you are able to detect and change check’s status proactively (e.g. by detecting lost connection). And this way is more efficient and robust.

User Guide

Note

To test server’s health we will use grpc_health_probe command.

Overall Server Health

The most simplest health checks:

from grpclib.health.service import Health

health = Health()

server = Server(handlers + [health])

Testing:

$ grpc_health_probe -addr=localhost:50051
healthy: SERVING

Overall server status is always SERVING.

If you want to add real checks:

from grpclib.health.service import Health, OVERALL

health = Health({OVERALL: [db_check, cache_check]})

Overall server status is SERVING if all checks are passing.

Detailed Services Health

If you want to provide different checks for different services:

foo = FooService()
bar = BarService()

health = Health({
    foo: [a_check, b_check],
    bar: [b_check, c_check],
})

Testing:

$ grpc_health_probe -addr=localhost:50051 -service acme.FooService
healthy: SERVING
$ grpc_health_probe -addr=localhost:50051 -service acme.BarService
healthy: NOT_SERVING
$ grpc_health_probe -addr=localhost:50051
healthy: NOT_SERVING

  • acme.FooService is healthy if a_check and b_check are passing

  • acme.BarService is healthy if b_check and c_check are passing

  • Overall health status depends on all checks

You can also override checks list for overall server’s health status:

foo = FooService()
bar = BarService()

health = Health({
    foo: [a_check, b_check],
    bar: [b_check, c_check],
    OVERALL: [a_check, c_check],
})

Reference

grpclib.health.service.OVERALL = <grpclib.health.service._Overall object>

Represents overall health status of all services

class grpclib.health.service.Health(checks: Optional[Mapping[ICheckable, Collection[CheckBase]]] = None)

Health-checking service

Example:

from grpclib.health.service import Health

auth = AuthService()
billing = BillingService()

health = Health({
    auth: [redis_status],
    billing: [db_check],
})

server = Server([auth, billing, health])
async Check(stream: Stream[HealthCheckRequest, HealthCheckResponse]) → None

Implements synchronous periodic checks

class grpclib.health.check.ServiceCheck(func: Callable[[], Awaitable[Optional[bool]]], *, loop: Optional[AbstractEventLoop] = None, check_ttl: float = 30, check_timeout: float = 10)

Performs periodic checks

Example:

async def db_test():
    # raised exceptions are the same as returning False,
    # except that exceptions will be logged
    await db.execute('SELECT 1;')
    return True

db_check = ServiceCheck(db_test)
Parameters

  • func – callable object which returns awaitable object, where result is one of: True (healthy), False (unhealthy), or None (unknown)

  • loop – (deprecated) asyncio-compatible event loop

  • check_ttl – how long we can cache result of the previous check

  • check_timeout – timeout for this check

class grpclib.health.check.ServiceStatus(*, loop: Optional[AbstractEventLoop] = None)

Contains status of a proactive check

Example:

redis_status = ServiceStatus()

# detected that Redis is available
redis_status.set(True)

# detected that Redis is unavailable
redis_status.set(False)
Parameters

loop – (deprecated) asyncio-compatible event loop

set(value: Optional[bool]) → None

Sets current status of a check

Parameters

value – True (healthy), False (unhealthy), or None (unknown)