Circuit Breaker Stability Pattern

Table of Contents

  • https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern
  • automatically degrades service functions in response to a likely fault, preventing larger or cascading failures by eliminating recurring errors and providing reasonable error responses
  • 2 major components :
    • Circuit
    • Breaker
  • Consider the Scenario:->
    • User Request <-> Business Logic Layer <-> Data Layer
    • the Business Logic implements a Breaker if the Circuit (Data Layer in this case has failed)
    • will retry after sensible pauses and backoff strats
    • won't continue requesting if failures are assured.

1. Code

  • Consider the Circuit type as, that signifies the signature of the function that's interacting with the database. An error as one of its returns is necessary.

    type Circuit func(context.Context) (string, error)
    
  • Consider the Breaker function that accepts a function of type Circuit and some failure threshold information (number of consecutive failures to be tolerated in this case, before breaking (opening) the circuit), returning a wrapped circuit with the same function signature.

    func Breaker(circuit Circuit, failureThreshold uint) Circuit {
    
            consecutiveFailures := 0
            lastAttempt := time.Now()
    
            // for closures access over breaker's administrative data
            // failure count and retry times
            var m sync.RWMutex
    
            return func(context.Context) (string,error) {
                    //see exponential backoff
                    m.RLock()
    
                    d := consecutiveFailures - int(failureThreshold)
    
                    if d >= 0 { //circuit breaking in case of failure
                            shouldRetryAt := lastAttempt.Add(time.Second*2 << d)
                            if !time.Now().After(shouldRetryAt){
                                    m.Runlock()
                                    return "", errors.New("Service unreachable")
                            }
                    }
    
                    m.Runlock()
    
                    // continuing request if no track of failures
    
                    response, err := circuit(ctx)
    
                    m.Lock() // acquiring write locks for closure's commons
                    defer m.Unlock()
    
                    lastAttempt = time.Now()
    
                    if err != nil {
                            consecutiveFailures++
                            return response, err
                    }
    
                    consecutiveFailures = 0
                    return response, nil
            }
    }
    
    
Tags::cs:cloud: