SDK Overview

Complete API reference for building graphs with Flutch SDK.

What is Flutch SDK?

Flutch SDK is a production-ready infrastructure package that transforms your graph logic into a fully functional microservice. Instead of building REST APIs, streaming logic, monitoring, and other infrastructure from scratch, you focus solely on your graph implementation - the SDK handles everything else.

The Problem It Solves

Building production AI agents requires more than just graph logic:

• REST API with streaming support
• Health checks and monitoring
• State management and persistence
• Interactive callbacks
• Versioning and deployment
• Security and error handling

The SDK provides all of this out of the box, so you can focus on your AI logic, not infrastructure.

Framework-Agnostic Architecture

While the SDK currently provides first-class support for LangGraph.js with TypeScript and NestJS, the architecture is designed to support other graph frameworks:

• LlamaIndex (coming soon)
• Custom graph implementations
• Other AI frameworks

This ensures your investment in the SDK remains valuable as the ecosystem evolves.

Getting Started

Installation

bash
actions.copy
npm install @flutchai/flutch-sdk

Quick Start

typescript
actions.copy
import { Injectable } from '@nestjs/common';
import { AbstractGraphBuilder } from '@flutchai/flutch-sdk';
import { StateGraph, START, END } from '@langchain/langgraph';

@Injectable()
export class MyGraphBuilder extends AbstractGraphBuilder<'myGraph'> {
  async buildGraph(): Promise<CompiledGraph> {
    return new StateGraph(MyState)
      .addNode('generate', this.generateNode.execute)
      .addEdge(START, 'generate')
      .addEdge('generate', END)
      .compile();
  }
}

Register and run:

typescript
actions.copy
@Module({
  imports: [
    UniversalGraphModule.register({
      graphType: "myGraph",
      builder: MyGraphBuilder,
    }),
  ],
})
export class AppModule {}

Your service is now running with REST API, streaming, health checks, and metrics.

Invoking Your Graph

Request Payload

When calling your graph, send a request with the following structure:

typescript
actions.copy
interface IGraphRequestPayload {
  // Unique identifier for this request
  requestId: string;

  // Your graph type identifier
  graphType: string;

  // Conversation messages
  messages: Array<{
    role: 'user' | 'assistant' | 'system';
    content: string;
  }>;

  // Optional: Thread ID for conversation continuity
  threadId?: string;

  // Optional: User and company context
  userId?: string;
  companyId?: string;

  // Optional: Channel information (web, telegram, etc.)
  channel?: string;
  platform?: string;

  // Optional: Graph-specific settings
  graphSettings?: {
    modelName?: string;
    temperature?: number;
    systemPrompt?: string;
    enabledTools?: string[];
    // ... any custom settings
  };
}

Non-Streaming Request

bash
actions.copy
curl -X POST http://localhost:3000/generate \
  -H "Content-Type: application/json" \
  -d '{
    "requestId": "req-123",
    "graphType": "myGraph",
    "messages": [
      { "role": "user", "content": "Hello, how are you?" }
    ],
    "userId": "user-456",
    "graphSettings": {
      "modelName": "gpt-4",
      "temperature": 0.7
    }
  }'

Response:

json
actions.copy
{
  "text": "Hello! I'm doing well, thank you for asking.",
  "attachments": [],
  "metadata": {
    "requestId": "req-123",
    "timestamp": "2025-11-03T12:00:00.000Z",
    "tokensUsed": 42
  }
}

Streaming Request (SSE)

bash
actions.copy
curl -X POST http://localhost:3000/stream \
  -H "Content-Type: application/json" \
  -d '{
    "requestId": "req-789",
    "graphType": "myGraph",
    "messages": [
      { "role": "user", "content": "Tell me a story" }
    ],
    "threadId": "thread-abc"
  }'

Response (Server-Sent Events):

bash
actions.copy
event: stream_event
data: {"type":"chunk","content":"Once"}

event: stream_event
data: {"type":"chunk","content":" upon"}

event: stream_event
data: {"type":"chunk","content":" a time"}

event: final
data: {"text":"Once upon a time...","attachments":[],"metadata":{}}

Flutch Platform Integration

The SDK is designed to work standalone or integrate with the Flutch Platform for enterprise capabilities:

Simple Platform Connection

typescript
actions.copy
UniversalGraphModule.register({
  graphType: "myGraph",
  builder: MyGraphBuilder,
  flutch: {
    apiKey: process.env.FLUTCH_API_KEY,
    endpoint: "https://api.flutch.ai"
  }
})

Platform Features

Tracing & Analytics

• Full distributed tracing across graph executions
• Performance metrics and bottleneck analysis
• Cost tracking per request and model
• Token usage analytics and optimization

Multi-Channel UI

• Deploy to Web, Telegram, Slack, WhatsApp
• Unified UI components across all channels
• Automatic channel-specific adaptations
• Custom branding and theming

Governance & Limits

• Rate limiting and quota management
• User-level and organization-level controls
• Budget alerts and automatic cutoffs
• Detailed usage reports

Testing & Quality Assurance

• A/B testing for different graph versions
• Automated acceptance testing
• AI-powered test case generation
• Regression testing and validation

Deployment Options

• Self-Hosted: Run the SDK standalone with your own infrastructure
• Platform-Connected: Connect to Flutch Platform for advanced features
• Hybrid: Use platform features selectively (e.g., only tracing or only multi-channel UI)

Your graphs work the same way regardless of deployment choice.

Core Concepts

1. Graph Builder

Your entry point for defining graph logic. Extend AbstractGraphBuilder:

typescript
actions.copy
@Injectable()
export class MyGraphBuilder extends AbstractGraphBuilder<'myGraph'> {
  async buildGraph(): Promise<CompiledGraph> {
    // Define your graph logic here
    // Access built-in services via dependency injection
    return compiledGraph;
  }
}

Key features:

• Implement buildGraph() method with your graph logic
• Use dependency injection for services (models, tools, messages)
• Access built-in components provided by the SDK

The builder is where you define your graph logic. The SDK handles everything else.

2. Automatic Service Setup

When you register your graph builder, the SDK automatically provides:

• REST Controllers: Graph execution endpoints (/generate, /stream)
• Health Checks: /health endpoint with NestJS Terminus
• Metrics: Prometheus metrics at /metrics
• Streaming: Server-Sent Events for real-time responses
• State Management: Redis-backed persistence
• Security: Helmet headers, compression, validation
• Registry: /registry endpoints for discovering registered graphs

3. State Management

Define your workflow state with type safety:

typescript
actions.copy
import { Annotation } from '@langchain/langgraph';

const MyState = Annotation.Root({
  messages: Annotation<BaseMessage[]>({
    reducer: (x, y) => x.concat(y),
  }),
  result: Annotation<string>,
  metadata: Annotation<Record<string, any>>,
});

State is automatically persisted and versioned using Redis checkpointing.

4. Models

Initialize AI models with ModelInitializer:

typescript
actions.copy
const model = await this.modelInitializer.initChatModel({
  modelName: 'gpt-4',
  temperature: 0.7,
});

Supported providers:

• OpenAI
• Anthropic
• Google (Gemini)
• Mistral AI
• Cohere

Also supports embedding and rerank models.

5. Tools

Execute MCP tools with McpRuntimeClient:

typescript
actions.copy
// Get available tools
const tools = await this.mcpClient.get_tools(['web_search', 'calculator']);

// Execute a tool
const result = await this.mcpClient.execute_tool('web_search', {
  query: 'latest AI news'
});

6. Graph Settings

Graph settings are provided in the request payload and accessible in your graph:

typescript
actions.copy
interface GraphRequestPayload {
  // ... other fields
  graphSettings?: {
    modelName?: string;
    temperature?: number;
    systemPrompt?: string;
    enabledTools?: string[];
    // Any custom settings your graph needs
  };
}

In your graph nodes:

typescript
actions.copy
async processNode(state: MyState, config: RunnableConfig) {
  const settings = config.configurable?.graph_settings;

  // Use settings to configure your graph behavior
  const modelName = settings?.modelName || 'gpt-4';
  const systemPrompt = settings?.systemPrompt || 'Default prompt';
  const enabledTools = settings?.enabledTools || [];

  // ... your logic
}

Settings are injected at runtime, making your graphs flexible and environment-aware. Each request can have different settings without changing your graph code.

7. Interactive Callbacks

Add interactive buttons and user interactions to your graph responses:

typescript
actions.copy
import { Callback, WithCallbacks } from '@flutchai/flutch-sdk/callbacks';

@Injectable()
export class MyNode {
  @Callback('approve')
  async handleApproval(callbackData: any, user: any) {
    // Handle approval action
    return { approved: true };
  }

  @WithCallbacks(['approve', 'reject'])
  async execute(state: MyState) {
    return {
      text: 'Please approve or reject this action',
      callbacks: [
        { id: 'approve', label: 'Approve' },
        { id: 'reject', label: 'Reject' }
      ]
    };
  }
}

Callbacks are:

• Token-based with security checks
• Idempotent (same callback can't be triggered twice)
• Platform-aware (Web, Telegram, etc.)
• Automatically tracked and logged

Graph Versioning

The SDK provides robust versioning capabilities for managing graph evolution:

Version Management

typescript
actions.copy
@Injectable()
export class MyGraphV1Builder extends AbstractGraphBuilder<'myGraph::1.0.0'> {
  readonly version = '1.0.0' as const;

  async buildGraph(): Promise<CompiledGraph> {
    // Version 1.0.0 implementation
  }
}

@Injectable()
export class MyGraphV2Builder extends AbstractGraphBuilder<'myGraph::2.0.0'> {
  readonly version = '2.0.0' as const;

  async buildGraph(): Promise<CompiledGraph> {
    // Version 2.0.0 implementation with new features
  }
}

Running Multiple Versions

Register multiple versions in your module:

typescript
actions.copy
@Module({
  imports: [
    UniversalGraphModule.register({
      graphType: "myGraph",
      builders: [
        MyGraphV1Builder,  // Legacy version
        MyGraphV2Builder,  // Current version
        MyGraphV3Builder,  // Beta version
      ],
      defaultVersion: "2.0.0"
    }),
  ],
})
export class AppModule {}

Version Selection

In the request payload:

json
actions.copy
{
  "requestId": "req-123",
  "graphType": "myGraph::2.0.0",  // Specific version
  "messages": [...]
}

Or use default version:

json
actions.copy
{
  "requestId": "req-123",
  "graphType": "myGraph",  // Uses defaultVersion (2.0.0)
  "messages": [...]
}

Version Utilities

Use GraphTypeUtils for version parsing and management:

typescript
actions.copy
import { GraphTypeUtils } from '@flutchai/flutch-sdk';

// Parse versioned graph type
const parsed = GraphTypeUtils.parse("myGraph::2.0.0");
// → { name: "myGraph", version: "2.0.0" }

// Build graph type string
const fullType = GraphTypeUtils.build("myGraph", "2.0.0");
// → "myGraph::2.0.0"

// Extract version
const version = GraphTypeUtils.getVersion("myGraph::1.5.0");
// → "1.5.0"

// Check version compatibility
const isCompatible = GraphTypeUtils.isCompatible("2.0.0", "2.1.0");
// → true (minor version change)

Versioning Benefits

1. Gradual Rollout: Deploy new versions while keeping old ones running
2. A/B Testing: Test new versions with subset of users
3. Safe Rollback: Instantly rollback by changing default version
4. Migration Path: Users can migrate at their own pace
5. Breaking Changes: Introduce breaking changes without disrupting existing users

Best Practices

• Use semantic versioning (MAJOR.MINOR.PATCH)
• Increment MAJOR for breaking changes
• Increment MINOR for new features
• Increment PATCH for bug fixes
• Keep at least 2 versions running during migration
• Deprecate old versions gradually with warnings

Available Endpoints

Your graph service automatically exposes these endpoints:

Graph Execution

• POST /generate - Non-streaming generation
• POST /stream - Server-Sent Events streaming
• POST /cancel/:requestId - Cancel running generation

Service Management

• GET /health - Health check
• GET /graph-types - List supported graph types
• GET /registry - View registered graphs
• GET /registry/stats - Registry statistics

Monitoring

• GET /metrics - Prometheus metrics

Interactive Features

• POST /callback - Handle user callbacks

OpenAPI Documentation

• GET /api - Swagger UI (in development mode)
• GET /api-json - OpenAPI JSON schema