Data Flow Patterns

Overview

Every user interaction in OpenSail follows a consistent request/response pattern that flows through the frontend, orchestrator, database, and (optionally) the container runtime. This page documents the lifecycle of each major flow: general API requests, agent chat, file operations, container management, Git operations, deployments, and streaming patterns. If you are new to the codebase, start with the General API Request Flow to understand the common pattern, then explore the specific flows relevant to your work.

General API Request Flow

All user interactions follow this eight-step lifecycle.

User interaction

The user performs an action in the browser (click, type, navigate).

Frontend sends request

The React app sends an HTTP or WebSocket request to the Orchestrator. Authentication is included via Authorization: Bearer {jwt} header or session cookie.

Orchestrator validates auth

FastAPI middleware decodes the JWT token, verifies the user session, and checks permissions (RBAC).

Database query or update

The Orchestrator queries or updates PostgreSQL using async SQLAlchemy.

Perform operation

Depending on the request type, the Orchestrator delegates to the appropriate subsystem:

File operation: Container filesystem (direct in Docker, pod exec in K8s)
Container operation: Docker Compose or Kubernetes API
AI chat: LiteLLM proxy to OpenAI/Anthropic
Deployment: Vercel/Netlify/Cloudflare API

Build response

The Orchestrator assembles the JSON response from the operation result and database state.

Return to frontend

The response is sent back to the React app over the same HTTP connection (or as SSE/WebSocket events for streaming).

UI update

The frontend updates its state and re-renders the relevant components.

Request Flow Diagram

+----------+
|  User    |
| Browser  |
+----+-----+
     |
     | 1. User interaction (click, type, etc.)
     v
+----------------+
|   Frontend     |
|  (React App)   |
+----+-----------+
     |
     | 2. HTTP/WebSocket request
     |    Authorization: Bearer {jwt} OR Cookie: {session}
     v
+----------------+
| Orchestrator   |
| (FastAPI API)  |
+----+-----------+
     |
     | 3. Validate authentication
     | 4. Query/update database
     v
+----------------+
|  PostgreSQL    |
|   Database     |
+----+-----------+
     |
     | 5. Database response
     v
+----------------+
| Orchestrator   |  6. Perform operation:
| (FastAPI API)  |     File op -> Container filesystem
+----+-----------+     Container op -> Docker/K8s API
     |                 AI chat -> LiteLLM -> AI provider
     | 7. Return JSON  Deployment -> Vercel/Netlify API
     v
+----------------+
|   Frontend     |
|  (React App)   |  8. Update UI with response data
+----------------+

Agent Chat Flow

The agent chat is the most complex data flow, involving LLM calls, tool execution, and real-time streaming to the frontend via Server-Sent Events (SSE).

User types a message

The user enters a message in the chat UI (e.g., “Create a React component for a todo list”).

Frontend opens SSE connection

The frontend sends POST /api/chat/stream with { project_id, message, chat_id } and opens an EventSource for streaming.

Load chat history

The chat router loads previous messages from the database and builds conversation context.

Create agent instance

agent/factory.py instantiates a tesslate-agent with the appropriate system prompt, available tools (read_file, write_file, bash_exec, etc.), and LLM model.

Agent execution loop

The tesslate-agent enters a loop:

Call the LLM with system prompt + conversation history
If the LLM returns tool calls, execute them (e.g., write_file, bash_exec)
Stream each tool execution event to the frontend
Call the LLM again with tool results
Repeat until the LLM produces a final text response

Stream final response

The agent streams its final message to the frontend, which renders it in real-time in the chat UI.

Agent Tool Execution Example

User prompt: “Create a React component for a todo list”

LLM Call 1:
  Input:  System prompt + user message
  Output: Tool call: write_file("src/TodoList.tsx", "import React...")

Tool Execution (write_file):
  Docker mode: Write to users/{user_id}/{project_slug}/src/TodoList.tsx
  K8s mode:    Exec into file-manager pod, write to /app/src/TodoList.tsx

Stream Event to Frontend:
  {
    "type": "tool_execution",
    "tool": "write_file",
    "args": { "path": "src/TodoList.tsx" },
    "result": "File created successfully"
  }

LLM Call 2:
  Input:  Previous context + tool result
  Output: "I've created a TodoList component in src/TodoList.tsx..."

Stream Event to Frontend:
  {
    "type": "message",
    "content": "I've created a TodoList component..."
  }

Available Agent Tools

Tool	File	Purpose
`read_file` / `write_file`	`agent/tools/file_ops/read_write.py`	Read and write files in the project
`patch_file` / `multi_edit`	`agent/tools/file_ops/edit.py`	Edit specific file sections
`bash_exec`	`agent/tools/shell_ops/bash.py`	Execute shell commands
`shell_exec` / `shell_open`	`agent/tools/shell_ops/session.py`	Persistent shell sessions
`web_fetch`	`agent/tools/web_ops/fetch.py`	HTTP requests for web content
`todos`	`agent/tools/planning_ops/todos.py`	Task planning and tracking
`get_project_info`	`agent/tools/project_ops/metadata.py`	Query project information

File Operations Flow

File reads and writes differ depending on deployment mode. In Docker mode, the orchestrator accesses the filesystem directly. In Kubernetes mode, it executes commands inside the file-manager pod.

Read File
Write File

User clicks file in browser
    |
    v
Frontend: GET /api/projects/{id}/files/{path}
    |
    v
Orchestrator validates auth, gets project
    |
    v
Check deployment mode:
  Docker mode:
    file_path = "users/{user_id}/{slug}/{path}"
    content = open(file_path).read()
  Kubernetes mode:
    namespace = "proj-{project_id}"
    pod = file-manager pod
    content = kubectl exec cat /app/{subdir}/{path}
    |
    v
Return { content } to frontend
    |
    v
Display in Monaco editor

User edits file in Monaco editor
    |
    v
Frontend: PUT /api/projects/{id}/files/{path}
          Body: { content: "..." }
    |
    v
Orchestrator validates auth, gets project
    |
    v
Check deployment mode:
  Docker mode:
    file_path = "users/{user_id}/{slug}/{path}"
    open(file_path, 'w').write(content)
  Kubernetes mode:
    Encode content as base64
    kubectl exec mkdir -p {dir_path}
    kubectl exec sh -c "echo '{base64}' | base64 -d > {path}"
    |
    v
Update project.last_activity
    |
    v
Return { status: "success" } to frontend

Container Operations Flow

Container start and stop operations are non-blocking. The Orchestrator returns immediately and the frontend polls for status updates.

Start Project Containers

User clicks Start

Frontend sends POST /api/projects/{id}/start.

Validation and background task

The Orchestrator validates auth, checks that the project is not already running, queues a background task for container setup, and returns { "status": "starting" } immediately.

Frontend polls for status

The frontend polls GET /api/projects/{id}/status every 2 seconds.

Background task executes (Kubernetes mode)

Create namespace (proj-{uuid})
Create PVC (shared storage, e.g. 10Gi RWO)
Restore from VolumeSnapshot if hibernated (or hydrate from S3 for legacy projects)
Create file-manager pod (always running)
For each container: create Deployment + Service + Ingress
Create NetworkPolicy for isolation
Update project status in database to “running”
Return container URLs

Frontend detects running state

Status poll detects “running”. The frontend displays container URLs and enables the live preview iframe.

Stop Project Containers

User clicks Stop (or navigates away)

Frontend sends POST /api/projects/{id}/stop.

Background task: dehydrate and delete

Create VolumeSnapshot from PVC (under 5 seconds)
Wait for snapshot readiness
Delete namespace (cascades to all resources: Deployments, Services, Ingress, PVC, NetworkPolicy)
Update project status to “hibernated”

Frontend detects stopped state

Status poll detects “stopped” or “hibernated”. Live preview is disabled and the Start button appears.

In Kubernetes mode, hibernation creates an EBS VolumeSnapshot that preserves the entire filesystem state including node_modules. No npm install is needed on restore. Projects restore in under 10 seconds thanks to EBS lazy-loading.

Git Operations Flow

Clone Repository

User clicks "Import from GitHub"
    |
    v
Frontend: POST /api/git/clone
          Body: { repo_url, project_id }
    |
    v
Orchestrator validates auth, queues background task
Returns { "status": "cloning" }
    |
    v
Background task:
  Kubernetes mode:
    Generate git clone script
    Execute in file-manager pod via kubectl exec
  Docker mode:
    Clone directly to filesystem: users/{user_id}/{slug}/
    |
    v
Frontend polls, detects "cloned"
Refreshes file tree

Commit and Push

User (or agent) clicks "Commit"
    |
    v
Frontend: POST /api/git/commit
          Body: { message, project_id }
    |
    v
Orchestrator:
  Kubernetes mode:
    kubectl exec git config user.name {name}
    kubectl exec git config user.email {email}
    kubectl exec git add .
    kubectl exec git commit -m {message}
    kubectl exec git push
  Docker mode:
    Execute git commands on filesystem
    |
    v
Return { "status": "pushed", "commit_hash": "..." }

Deployment Flow (External Providers)

External deployments to Vercel, Netlify, or Cloudflare follow a consistent non-blocking pattern.

User initiates deployment

Frontend sends POST /api/deployments with provider name, project ID, and configuration.

Retrieve OAuth credentials

The Orchestrator decrypts the user’s stored DeploymentCredential for the chosen provider.

Background build and deploy

Build the project locally (e.g., npm run build)
Push to Git if needed (create/update GitHub repo)
Call provider API to create deployment
Poll provider API until deployment status is “READY”
Save deployment record to database

Notify frontend

A WebSocket message or status poll delivers the live URL to the frontend, which displays a success message with a link to the deployed application.

WebSocket and SSE Streaming Patterns

OpenSail uses two streaming mechanisms for real-time communication.

Server-Sent Events (Agent Chat)
Polling (Status Checks)
WebSocket (Bidirectional)

Backend (FastAPI):

async def stream_agent_response(project_id, message):
    async for event in agent.run(message):
        yield f"data: {json.dumps(event)}\n\n"

@router.post("/stream")
async def chat_stream(request: ChatRequest):
    return StreamingResponse(
        stream_agent_response(request.project_id, request.message),
        media_type="text/event-stream"
    )

Frontend (EventSource):

const eventSource = new EventSource('/api/chat/stream', {
  body: JSON.stringify({ message, project_id }),
  method: 'POST'
});

eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);
  switch (data.type) {
    case 'tool_execution':
      displayToolExecution(data.tool, data.args);
      break;
    case 'message':
      displayAgentMessage(data.content);
      break;
    case 'error':
      displayError(data.error);
      break;
  }
};

Use cases: Agent chat streaming, build output streaming.

Frontend:

const pollStatus = async () => {
  const status = await api.get(`/api/projects/${id}/status`);
  if (status.state !== 'running') {
    setTimeout(pollStatus, 2000); // Poll every 2 seconds
  }
};

Use cases: Container startup status, deployment progress, background task status.

Backend (FastAPI):

@router.websocket("/ws/{chat_id}")
async def websocket_endpoint(websocket: WebSocket, chat_id: str):
    await websocket.accept()
    try:
        while True:
            data = await websocket.receive_text()
            await websocket.send_text(response)
    except WebSocketDisconnect:
        pass

Use cases: Live terminal sessions, real-time log streaming, hibernation notifications.

Performance Optimizations

Non-Blocking Operations

Long-running operations return immediately and execute in the background. The frontend polls for status.

# Return immediately, execute in background
@router.post("/")
async def create_project(data: ProjectCreate, background_tasks: BackgroundTasks):
    project = await db_create_project(data)
    background_tasks.add_task(setup_containers, project)
    return project  # Frontend polls /status

Database Query Optimization

Use selectinload() to prevent N+1 queries and load related objects in a single query.

# Single query with joined loading
project = await db.execute(
    select(Project)
    .options(
        selectinload(Project.containers),
        selectinload(Project.containers)
            .selectinload(Container.connections)
    )
    .where(Project.id == project_id)
)

Streaming vs. Polling Decision Guide

Pattern	When to Use	Example
SSE (Server-Sent Events)	Unidirectional, real-time data from server	Agent chat responses
Polling	Simple status checks, stateless	Container startup status
WebSocket	Bidirectional, real-time communication	Live terminal, shell sessions

Key Source Files

File	Purpose
`orchestrator/app/routers/projects.py`	Project CRUD, file operations, container lifecycle
`orchestrator/app/routers/chat.py`	Agent chat and streaming
`orchestrator/app/routers/git.py`	Git operations
`orchestrator/app/routers/deployments.py`	External deployments
`orchestrator/app/agent/stream_agent.py`	Streaming AI agent implementation
`orchestrator/app/agent/factory.py`	Agent creation from config
`orchestrator/app/agent/tools/`	Agent tool implementations
`orchestrator/app/services/orchestration/kubernetes_orchestrator.py`	K8s container management
`orchestrator/app/services/s3_manager.py`	S3 hydration/dehydration

Get Started

Install and Run

Core Concepts

Working with Projects

Agents and Skills

Ship and Share

Self-Hosting

Developer Guide

Data Flow Patterns

Overview

General API Request Flow

Request Flow Diagram

Agent Chat Flow

Agent Tool Execution Example

Available Agent Tools

File Operations Flow

Container Operations Flow

Start Project Containers

Stop Project Containers

Git Operations Flow

Clone Repository

Commit and Push

Deployment Flow (External Providers)

WebSocket and SSE Streaming Patterns

Performance Optimizations

Key Source Files

​Overview

​General API Request Flow

​Request Flow Diagram

​Agent Chat Flow

​Agent Tool Execution Example

​Available Agent Tools

​File Operations Flow

​Container Operations Flow

​Start Project Containers

​Stop Project Containers

​Git Operations Flow

​Clone Repository

​Commit and Push

​Deployment Flow (External Providers)

​WebSocket and SSE Streaming Patterns

​Performance Optimizations

​Key Source Files

Overview

General API Request Flow

Request Flow Diagram

Agent Chat Flow

Agent Tool Execution Example

Available Agent Tools

File Operations Flow

Container Operations Flow

Start Project Containers

Stop Project Containers

Git Operations Flow

Clone Repository

Commit and Push

Deployment Flow (External Providers)

WebSocket and SSE Streaming Patterns

Performance Optimizations

Key Source Files