RAG (Retrieval-Augmented Generation)

Meloqui provides components for building RAG pipelines that enhance LLM responses with relevant context.

Overview

RAG works by:

1. Chunking documents into smaller pieces
2. Converting chunks to vector embeddings
3. Storing embeddings for similarity search
4. Retrieving relevant chunks based on user queries
5. Injecting context into the LLM prompt

IDocumentStore

Interface for document storage and retrieval.

interface IDocumentStore {
  /** Add documents (automatically chunked and embedded) */
  addDocuments(docs: Document[]): Promise<void>;

  /** Search for relevant chunks */
  search(query: string, topK?: number): Promise<SearchResult[]>;

  /** Delete chunks from a source */
  delete(source: string): Promise<void>;

  /** Clear all documents */
  clear(): Promise<void>;
}

Document

A document to add to the store.

interface Document {
  /** The text content */
  content: string;
  /** Source identifier (e.g., filename) */
  source: string;
  /** Optional metadata */
  metadata?: Record<string, unknown>;
}

DocumentChunk

A chunk with its embedding.

interface DocumentChunk {
  /** Unique chunk ID */
  id: string;
  /** Chunk text content */
  content: string;
  /** Vector embedding */
  embedding: number[];
  /** Chunk metadata */
  metadata: {
    source: string;
    chunkIndex: number;
    [key: string]: unknown;
  };
}

SearchResult

Search result with similarity score.

interface SearchResult extends DocumentChunk {
  /** Similarity score (-1 to 1, higher is more similar) */
  score: number;
}

InMemoryDocumentStore

In-memory document store for development and small datasets.

import { InMemoryDocumentStore, OpenAIEmbeddings } from 'meloqui';

const embeddings = new OpenAIEmbeddings({
  apiKey: process.env.OPENAI_API_KEY
});

const store = new InMemoryDocumentStore({ embeddingProvider: embeddings });

await store.addDocuments([
  { content: 'Meloqui is an LLM SDK', source: 'docs/intro.md' }
]);

const results = await store.search('What is Meloqui?', 3);

InMemoryDocumentStoreConfig

interface InMemoryDocumentStoreConfig {
  /** Embedding provider for generating vectors */
  embeddingProvider: IEmbeddingProvider;
  /** Maximum chunk size in characters (default: 1000) */
  chunkSize?: number;
  /** Overlap between chunks in characters (default: 200) */
  chunkOverlap?: number;
  /** Optional logger for warnings */
  logger?: ILogger;
  /** Warning threshold (default: 10000) */
  warnThreshold?: number;
  /** Maximum chunks to store (default: 100000, set to 0 to disable) */
  maxChunks?: number;
}

Constants

const DEFAULT_MAX_CHUNKS = 100000;
const DEFAULT_WARN_THRESHOLD = 10000;

FileDocumentStore

File-based persistence for document stores.

import { FileDocumentStore, OpenAIEmbeddings } from 'meloqui';

const store = new FileDocumentStore({
  embeddingProvider: new OpenAIEmbeddings({ apiKey }),
  directory: './rag-data'
});

FileDocumentStoreConfig

interface FileDocumentStoreConfig {
  /** Embedding provider for generating vectors */
  embeddingProvider: IEmbeddingProvider;
  /** Directory to store chunk data */
  directory: string;
  /** Maximum chunk size in characters (default: 1000) */
  chunkSize?: number;
  /** Overlap between chunks in characters (default: 200) */
  chunkOverlap?: number;
}

IEmbeddingProvider

Interface for embedding providers.

interface IEmbeddingProvider {
  /** Embed single text */
  embed(text: string): Promise<number[]>;

  /** Embed multiple texts (batch) */
  embedBatch(texts: string[]): Promise<number[][]>;

  /** Embedding dimensions */
  readonly dimensions: number;
}

OpenAIEmbeddings

OpenAI embedding provider using text-embedding-3-small.

import { OpenAIEmbeddings } from 'meloqui';

const embeddings = new OpenAIEmbeddings({
  apiKey: process.env.OPENAI_API_KEY,
  model: 'text-embedding-3-small' // default
});

const vector = await embeddings.embed('Hello world');
console.log(`Dimensions: ${embeddings.dimensions}`); // 1536

OpenAIEmbeddingsConfig

interface OpenAIEmbeddingsConfig {
  /** API key (defaults to OPENAI_API_KEY env var) */
  apiKey?: string;
  /** Model name (default: 'text-embedding-3-small') */
  model?: string;
  /** Custom base URL for API-compatible endpoints */
  baseUrl?: string;
  /** Maximum texts per API request (default: 2048) */
  batchSize?: number;
  /** Override dimensions for custom/unknown models */
  dimensions?: number;
}

Text Chunking

chunkText

Split text into overlapping chunks for embedding.

import { chunkText } from 'meloqui';

const chunks = chunkText(longDocument, {
  chunkSize: 1000,    // max chars per chunk
  chunkOverlap: 200,  // overlap between chunks
  splitOn: 'sentence' // boundary type
});

ChunkOptions

interface ChunkOptions {
  /** Maximum chunk size in characters (default: 1000) */
  chunkSize?: number;
  /** Overlap between chunks (default: 200) */
  chunkOverlap?: number;
  /** Split boundary type (default: 'character') */
  splitOn?: SplitBoundary;
}

SplitBoundary

type SplitBoundary = 'character' | 'sentence' | 'paragraph';

Boundary	Description
character	Split at exact positions (may split mid-word)
sentence	Split at sentence boundaries (., ?, !)
paragraph	Split at double newlines

Similarity

cosineSimilarity

Calculate cosine similarity between two vectors.

import { cosineSimilarity } from 'meloqui';

const similarity = cosineSimilarity(vectorA, vectorB);
// Returns -1 to 1 (higher = more similar)

Using RAG with ChatClient

import { ChatClient, InMemoryDocumentStore, OpenAIEmbeddings } from 'meloqui';

const store = new InMemoryDocumentStore({
  embeddingProvider: new OpenAIEmbeddings({ apiKey })
});

await store.addDocuments([
  { content: 'Product docs...', source: 'product.md' },
  { content: 'API reference...', source: 'api.md' }
]);

const client = new ChatClient({
  provider: 'openai',
  model: 'gpt-4o',
  documentStore: store,
  ragOptions: {
    topK: 3,
    minScore: 0.5
  }
});

// Context is automatically retrieved and injected
const response = await client.chat('How do I use the API?');

RAGOptions

interface RAGOptions {
  /** Number of chunks to retrieve (default: 3) */
  topK?: number;
  /** Minimum similarity threshold (default: 0) */
  minScore?: number;
  /** Custom context injection template */
  contextTemplate?: string;
  /** Throw on RAG failure (default: false) */
  failOnError?: boolean;
  /** Error callback */
  onError?: (error: Error) => void | Promise<void>;
}