tutorial

Monitoring Apache ServiceComb Microservices with Vigilmon

Apache ServiceComb is a production-grade Java microservices framework — but service registry failures and circuit breaker trips are hard to catch without the right monitors. Here's how to monitor ServiceComb Service Center, instance health, and distributed Saga transactions with Vigilmon.

Apache ServiceComb is an open source Java microservices framework originally developed by Huawei and donated to the Apache Software Foundation in 2017, graduating as a top-level project in 2018. It provides a comprehensive runtime for Java microservices: service registration and discovery via Service Center, load balancing, circuit breaking, distributed Saga transactions via ServiceComb Pack, and runtime governance configuration via Kie. ServiceComb is widely used in Huawei Cloud and enterprise Java deployments across Asia. Vigilmon adds the external monitoring layer that keeps you informed when Service Center crashes, circuit breakers open, or Saga compensations spike.

What You'll Set Up

  • Apache Service Center health monitor (registry availability)
  • Microservice instance count alerting
  • Circuit breaker state monitoring per service dependency
  • ServiceComb Kie configuration service health check
  • Saga transaction health via heartbeat
  • Request throughput and latency endpoint monitors

Prerequisites

  • Apache ServiceComb Java Chassis 2.x+ or 3.x
  • ServiceComb Service Center running (standalone or embedded)
  • Optional: ServiceComb Kie and ServiceComb Pack deployed
  • A free Vigilmon account

Step 1: Monitor ServiceComb Service Center Health

Service Center is the service registry where every ServiceComb microservice registers its endpoints and discovers its peers. If Service Center goes down, all service discovery fails and no new RPC calls can be routed. Add an HTTP monitor for the Service Center health endpoint:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the Service Center health URL: http://service-center-host:30100/v4/default/registry/health.
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

ServiceComb Service Center exposes a /health endpoint at the registry API base path. A 200 response confirms the registry is accepting registrations and responding to discovery queries.

Step 2: Monitor Microservice Instance Counts

ServiceComb microservices register their instances on startup. A drop in registered instance count — below the minimum healthy count — means requests will be load-balanced to fewer backends, increasing latency and error rates. Add a keyword monitor that checks instance counts via the Service Center API:

  1. Click Add Monitor.
  2. Set Type to HTTP / HTTPS with Keyword check.
  3. Enter: http://service-center-host:30100/v4/default/registry/microservices/SERVICE_ID/instances.
  4. Set Keyword to "instances" to confirm the response contains instance data.
  5. Set Check interval to 1 minute.
  6. Click Save.

For production use, expose a custom /health/instances endpoint from each service that returns a structured count:

@RestSchema(schemaId = "instanceHealth")
@RequestMapping(path = "/health")
public class InstanceHealthEndpoint {

  @GetMapping("/instances")
  public Map<String, Object> instanceHealth() {
    return Map.of(
      "status", "UP",
      "registeredInstances", DiscoveryManager.INSTANCE.getInstanceCount("my-service")
    );
  }
}

Monitor this endpoint with a keyword check for "UP".

Step 3: Monitor Circuit Breaker State

ServiceComb circuit breakers (Hystrix-compatible) protect services from cascading failures. An open circuit breaker means the downstream service is failing — requests are being short-circuited before they reach the unhealthy backend. Expose circuit breaker state from your service's actuator endpoint and monitor it:

// Add a health indicator for circuit breakers
@Component
public class CircuitBreakerHealthIndicator implements HealthIndicator {

  @Override
  public Health health() {
    // Check Hystrix circuit breaker state
    boolean anyOpen = HystrixCircuitBreaker.Factory.getInstance()
        .getInstances().stream()
        .anyMatch(cb -> cb.isOpen());
    
    return anyOpen
        ? Health.down().withDetail("circuitBreaker", "OPEN").build()
        : Health.up().withDetail("circuitBreaker", "CLOSED").build();
  }
}
  1. Click Add Monitor in Vigilmon.
  2. Set Type to HTTP / HTTPS with Keyword check.
  3. Enter: http://your-service:8080/health/circuit-breaker.
  4. Set Keyword must NOT contain to "OPEN".
  5. Set Check interval to 1 minute.
  6. Click Save.

An open circuit breaker will cause the keyword check to fail, triggering an alert.

Step 4: Monitor ServiceComb Kie Configuration Service

ServiceComb Kie manages runtime governance configuration — rate limiting rules, circuit breaker thresholds, load balancing weights. If Kie becomes unavailable, services fall back to static configuration, which may be incorrect for the current load patterns. Add a health monitor for Kie:

  1. Click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter: http://kie-host:30110/v1/default/kie/kv?label=environment:production.
  4. Set Expected HTTP status to 200.
  5. Set Check interval to 2 minutes.
  6. Click Save.

The Kie API returns 200 when the configuration service is healthy and the key-value store is accessible.

Step 5: Monitor Saga Transaction Health (ServiceComb Pack)

If you use ServiceComb Pack for distributed Saga transactions, a rising compensation rate indicates distributed transaction failures requiring rollback. Add a heartbeat that fires only when the Saga success rate is within acceptable bounds:

@Component
public class SagaHealthReporter {

  @Scheduled(fixedDelay = 60000)
  public void reportSagaHealth() {
    double compensationRate = sagaMetrics.getCompensationRate();
    if (compensationRate < 0.05) { // <5% compensation rate is healthy
      vigilmonHeartbeat.fire("YOUR_SAGA_HEARTBEAT_ID");
    }
    // If compensation rate exceeds 5%, heartbeat stops and Vigilmon alerts
  }
}

Create a Vigilmon Heartbeat monitor with a 3-minute timeout. If Saga compensations exceed 5%, the heartbeat stops and you receive an alert before downstream users encounter failed transactions.

Step 6: Monitor Request Throughput and Latency

Microservice-to-microservice call performance is the key indicator of ServiceComb application health. Add HTTP monitors for each critical service's health and response time:

  1. Click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter your service's health endpoint: http://your-service:8080/health.
  4. Set Expected HTTP status to 200.
  5. Set Response time alert to 500ms (adjust to your SLA).
  6. Set Check interval to 1 minute.
  7. Click Save.

Repeat for each critical downstream service. Vigilmon tracks response time history and alerts you when latency breaches the configured threshold.

Step 7: Monitor API Contract Validation

ServiceComb validates requests and responses against OpenAPI contracts. A spike in contract violations indicates that a client or service is sending unexpected payloads — often caused by schema changes or version mismatches. Expose a contract health endpoint:

@GetMapping("/health/contracts")
public ResponseEntity<Map<String, Object>> contractHealth() {
  long violations = contractViolationCounter.get();
  double errorRate = violationRateTracker.getRate();
  
  if (errorRate > 0.01) { // >1% violation rate is suspicious
    return ResponseEntity.status(503).body(Map.of(
      "status", "DEGRADED",
      "violationRate", errorRate
    ));
  }
  return ResponseEntity.ok(Map.of("status", "OK", "violations", violations));
}

Monitor this endpoint with a Vigilmon HTTP check expecting status 200.

Alerting Configuration

Configure targeted alerts for each ServiceComb component:

| Alert | Condition | Recommended Channel | |-------|-----------|---------------------| | Service Center down | HTTP check fails | PagerDuty / SMS | | Instance count drop | Keyword check fails | Slack + Email | | Circuit breaker open | OPEN keyword detected | Slack | | Kie unavailable | HTTP check fails | Email | | Saga compensation spike | Heartbeat missing > 3 min | Slack | | Latency SLA breach | Response time > 500ms | Slack | | Contract violations | Status 503 | Email |

In Vigilmon, go to Alerts → Notification Channels and add your Slack webhook or email. Assign each monitor to the appropriate channel with a suitable escalation policy.

Conclusion

Apache ServiceComb's comprehensive microservices runtime requires monitoring at every layer: the service registry, individual instance pools, circuit breakers, governance configuration, and distributed transactions. Vigilmon's combination of HTTP uptime checks, keyword monitors, and heartbeat monitors gives you a complete picture of your ServiceComb cluster health. When Service Center goes down or a circuit breaker trips, you know before your users do.

Start with a free Vigilmon account and add your Service Center health monitor in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles