Custom Providers

All providers in Meloqui implement the ProviderPlugin interface, including the built-in OpenAI, Anthropic, Google, and Ollama providers. You can create your own custom providers following the same pattern.

The ProviderPlugin Interface

import { ProviderPlugin, ProviderCapabilities } from 'meloqui';

interface ProviderPlugin {
  readonly name: string;
  readonly capabilities: ProviderCapabilities;

  chat(messages: Message[], options?: ChatOptions): Promise<ChatResponse>;
  stream(messages: Message[], options?: ChatOptions): AsyncIterableIterator<StreamChunk>;
  chatWithTools?(messages: Message[], tools: Record<string, CoreTool>, options?: ChatOptions): Promise<ChatResponse>;
  streamWithTools?(messages: Message[], tools: Record<string, CoreTool>, options?: ChatOptions): AsyncIterableIterator<StreamChunk>;
}

interface ProviderCapabilities {
  chat: boolean;
  streaming: boolean;
  toolCalling: boolean;
  vision: boolean;
  audio: boolean;
}

Creating a Custom Provider

import { ChatClient, ProviderPlugin, ProviderCapabilities } from 'meloqui';

class MyCustomProvider implements ProviderPlugin {
  readonly name = 'my-custom-llm';
  readonly capabilities: ProviderCapabilities = {
    chat: true,
    streaming: true,
    toolCalling: false,
    vision: false,
    audio: false
  };

  async chat(messages, options) {
    // Call your custom API here
    return { content: '...', role: 'assistant' };
  }

  async *stream(messages, options) {
    // Implement streaming
    yield { content: '...', role: 'assistant' };
  }
}

const client = new ChatClient({
  provider: new MyCustomProvider(),
  model: 'custom-model-v1'
});

Configuration Interface

When registering a custom provider, ChatClient will pass a configuration object to your provider's constructor. To support all features, your provider's config interface should include:

import { ILogger, logger as defaultLogger } from 'meloqui';

interface MyProviderConfig {
  apiKey: string;
  model: string;
  baseUrl?: string;
  /** Optional logger for debug/error output */
  logger?: ILogger;
}

class MyCustomProvider implements ProviderPlugin {
  private readonly logger: ILogger;

  constructor(config: MyProviderConfig) {
    // Fall back to default logger if none provided
    this.logger = config.logger ?? defaultLogger;
  }

  async chat(messages, options) {
    this.logger.debug('chat request', { provider: this.name, model: this.model });
    // ... implementation
  }
}

The logger field enables debug logging to flow from ChatClient through to your provider. If not provided, you can fall back to the default logger or skip logging entirely.

Capabilities

Declare your provider's capabilities honestly. The ChatClient uses these to:

• Validate operations before attempting them (e.g., throwing CapabilityError if tool calling is attempted on a provider that doesn't support it)
• Enable/disable features in the client

Capability	Description
chat	Basic chat completion
streaming	Streaming responses
toolCalling	Function/tool calling
vision	Image input support
audio	Audio input/output