> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openclaas.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Local GPU Backend

> Run CLaaS with local GPU training and vLLM inference

# Local GPU Backend

The Local backend runs SDPO training and vLLM inference on your own hardware. It requires a GPU with >= 24 GB VRAM.

## Requirements

* NVIDIA GPU with >= 24 GB VRAM (e.g. RTX 3090, RTX 4090, A5000, L40S)
* [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) (for Docker)
* Docker and Docker Compose
* Python 3.11+ and [uv](https://docs.astral.sh/uv/)

## Installation

<Steps>
  <Step title="Clone and install">
    ```bash theme={null}
    git clone https://github.com/kfallah/CLaaS.git
    cd CLaaS
    uv sync --extra local
    ```
  </Step>

  <Step title="Configure environment">
    ```bash theme={null}
    cd docker
    cp .env.local.example .env
    ```

    Edit `.env` and set `TELEGRAM_BOT_TOKEN` (required). Optionally set `HF_TOKEN` for gated models.
  </Step>

  <Step title="Start the stack">
    ```bash theme={null}
    docker compose --profile local up --build
    ```

    The first run downloads Qwen3-8B (\~16 GB). The vLLM health check takes 10-20 minutes on first start.
  </Step>

  <Step title="Verify">
    ```bash theme={null}
    curl http://localhost:8000/v1/models -H "Authorization: Bearer sk-local"
    curl http://localhost:8080/
    curl http://localhost:8080/v1/lora
    ```
  </Step>
</Steps>

## Services

| Service          | Port  | Description                                             |
| ---------------- | ----- | ------------------------------------------------------- |
| `vllm`           | 8000  | Qwen3-8B with LoRA serving and sleep/wake support       |
| `claas-api`      | 8080  | CLaaS feedback API and distill worker                   |
| `openclaw-local` | 18789 | OpenClaw gateway with Telegram bot                      |
| `init-local`     | —     | One-shot: creates LoRA adapter + writes OpenClaw config |

## Configuration

These variables are set in the `.env` file.

| Variable                 | Required | Default         | Description                           |
| ------------------------ | -------- | --------------- | ------------------------------------- |
| `TELEGRAM_BOT_TOKEN`     | Yes      | —               | Bot token from @BotFather             |
| `HF_TOKEN`               | No       | —               | HuggingFace token (gated models only) |
| `MODEL`                  | No       | `Qwen/Qwen3-8B` | Base model ID                         |
| `GPU_MEMORY_UTILIZATION` | No       | `0.70`          | VRAM fraction for vLLM                |
| `MAX_MODEL_LEN`          | No       | `32768`         | Max sequence length                   |

For the full Hydra config and all environment variables, see the [Configuration Reference](/configuration).

## Verification

```bash theme={null}
# Check vLLM models
curl http://localhost:8000/v1/models -H "Authorization: Bearer sk-local"

# Check CLaaS API
curl http://localhost:8080/

# List LoRA adapters
curl http://localhost:8080/v1/lora

# Test feedback loop
curl -X POST http://localhost:8080/v1/feedback \
  -H "Content-Type: application/json" \
  -d '{
    "lora_id": "openclaw/assistant-latest",
    "prompt": "hi",
    "response": "hello",
    "feedback": "good",
    "training": {"teacher_mode": "self"}
  }'
```

Send a DM to your Telegram bot. It should respond using the `openclaw-assistant-latest` LoRA model.

<Accordion title="Manual Setup (without Docker)">
  If you prefer not to use Docker, you can run each service manually:

  ```bash theme={null}
  # 1. Start vLLM with LoRA support
  vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --port 8000 \
    --enable-lora --lora-modules my-lora=/loras/user/my-lora-init

  # 2. Start the CLaaS API
  uv run uvicorn claas.api:web_app --host 0.0.0.0 --port 8080

  # 3. Initialize a LoRA adapter
  curl -X POST http://localhost:8080/v1/lora/init \
    -H "Content-Type: application/json" \
    -d '{"lora_id": "user/my-lora"}'

  # 4. Send feedback
  curl -X POST http://localhost:8080/v1/feedback \
    -H "Content-Type: application/json" \
    -d '{
      "lora_id": "user/my-lora-init",
      "prompt": "Write a function to calculate factorial",
      "response": "def factorial(n): ...",
      "feedback": "Good recursive solution"
    }'
  ```

  <Warning>
    When running uvicorn directly, use `claas.api:web_app`, **not** `claas.api:app`. The `app` object is a Modal `App` and is not ASGI-compatible.
  </Warning>
</Accordion>