Infrastructure

Meloqui provides infrastructure components for retry logic and rate limiting.

RetryManager

Provides retry logic with exponential backoff for transient errors.

import { RetryManager, RetryConfig } from 'meloqui';

const config: RetryConfig = {
  maxAttempts: 3,
  initialBackoffMs: 1000,
  maxBackoffMs: 10000,
  backoffMultiplier: 2
};

const retryManager = new RetryManager(config, (attempt, error, delayMs) => {
  console.log(`Retry ${attempt}: ${error.message} (waiting ${delayMs}ms)`);
});

const result = await retryManager.executeWithRetry(async () => {
  return await someApiCall();
});

Constructor

constructor(config: RetryConfig, onRetry?: OnRetryCallback)

Parameter	Type	Description
config	RetryConfig	Retry configuration
onRetry	OnRetryCallback	Optional callback before each retry

executeWithRetry

Execute an async operation with retry logic.

executeWithRetry<T>(operation: () => Promise<T>): Promise<T>

Only retries errors with isRetryable: true (e.g., RateLimitError, TimeoutError, ServerError).

OnRetryCallback

type OnRetryCallback = (
  attempt: number,  // 1-indexed retry attempt
  error: Error,     // Error that triggered retry
  delayMs: number   // Backoff delay before this retry
) => void;

RetryConfig

interface RetryConfig {
  /** Maximum retry attempts */
  maxAttempts: number;
  /** Initial backoff delay in ms */
  initialBackoffMs: number;
  /** Maximum backoff delay in ms */
  maxBackoffMs: number;
  /** Exponential backoff multiplier */
  backoffMultiplier: number;
}

Backoff Calculation

Delay = initialBackoffMs * (backoffMultiplier ^ attempt), capped at maxBackoffMs.

Example with default config:

• Attempt 1: 1000ms
• Attempt 2: 2000ms
• Attempt 3: 4000ms (capped at maxBackoffMs)

RateLimiter

Token bucket rate limiter for controlling API request rates.

import { RateLimiter, RateLimitConfig } from 'meloqui';

const limiter = new RateLimiter({
  requestsPerMinute: 60
});

// Waits if rate limit would be exceeded
await limiter.acquire();
await makeApiCall();

Constructor

constructor(config: RateLimitConfig)

acquire

Acquire permission to make a request. Waits if rate limits would be exceeded.

acquire(tokenCount?: number): Promise<void>

Parameter	Type	Description
tokenCount	number	Tokens this request consumes (default: 0)

RateLimitConfig

interface RateLimitConfig {
  /** Maximum requests per minute */
  requestsPerMinute?: number;
  /** Maximum tokens per minute (for token-based limiting) */
  tokensPerMinute?: number;
}

How It Works

Uses the token bucket algorithm with continuous refill:

• Bucket starts full (at the per-minute limit)
• Each request consumes one token
• Tokens refill continuously over time
• If bucket is empty, acquire() waits until tokens are available

Using with ChatClient

Both retry and rate limiting are built into ChatClient:

import { ChatClient } from 'meloqui';

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4o',
  retryConfig: {
    maxAttempts: 3,
    initialBackoffMs: 1000,
    maxBackoffMs: 10000,
    backoffMultiplier: 2
  },
  rateLimitConfig: {
    requestsPerMinute: 60
  }
});

Standalone Usage

For custom API clients or advanced scenarios:

import { RetryManager, RateLimiter, RateLimitError } from 'meloqui';

const limiter = new RateLimiter({ requestsPerMinute: 100 });
const retryManager = new RetryManager({
  maxAttempts: 3,
  initialBackoffMs: 500,
  maxBackoffMs: 5000,
  backoffMultiplier: 2
});

async function makeRequest() {
  return retryManager.executeWithRetry(async () => {
    await limiter.acquire();

    const response = await fetch('/api/data');
    if (response.status === 429) {
      throw new RateLimitError('Rate limited');
    }
    return response.json();
  });
}