49 lines
1.8 KiB
Markdown
49 lines
1.8 KiB
Markdown
# SkyrimNet LLM Stack — LIVE
|
|
|
|
All four GPUs live. llama.cpp on dioscuri/comfy-dev. vLLM on theia.
|
|
Chat endpoint: `/v1/chat/completions` with `{"chat_template_kwargs": {"enable_thinking": false}}`
|
|
|
|
## Instances
|
|
|
|
| Instance | Node | GPU | Port | Model | Quant | VRAM | Vision |
|
|
|----------|------|-----|------|-------|-------|------|--------|
|
|
| **nyx** | theia | Blackwell 6000 | 31000 | self-trained ~41B | BF16 | ~84GB | ✅ |
|
|
| **alpha1** | dioscuri | Ada #1 | 31001 | Qwen3.6-27B-Fable-711 | IQ4_XS | 16.6GB | ❌ |
|
|
| **alpha2** | dioscuri | Ada #2 | 31002 | Qwen3.6-27B-Fable-711 | IQ4_XS | 16.6GB | ❌ |
|
|
| **gamma** | comfy-dev | RTX 3090 | 31003 | Qwen3.6-27B-Fable-711-MTP | Q4_K_S MTP | 19.9GB | ✅ |
|
|
|
|
## Role Mapping
|
|
|
|
- **nyx (31000):** default dialogue, diary, agent_helper
|
|
- **alpha1 (31001):** meta, combat, action_evaluation
|
|
- **alpha2 (31002):** gamemaster, overflow
|
|
- **gamma (31003):** memory, character_profile, vision, universal_translator
|
|
|
|
## Endpoints for SkyrimNet
|
|
|
|
```yaml
|
|
default: 10.0.40.21:31000/v1
|
|
diary: 10.0.40.21:31000/v1
|
|
agent_helper: 10.0.40.21:31000/v1
|
|
meta: 10.0.40.22:31001/v1
|
|
combat: 10.0.40.22:31001/v1
|
|
action_evaluation: 10.0.40.22:31001/v1
|
|
gamemaster: 10.0.40.22:31002/v1
|
|
memory: 10.0.30.124:31003/v1
|
|
character_profile: 10.0.30.124:31003/v1
|
|
vision: 10.0.30.124:31003/v1
|
|
universal_translator: 10.0.30.124:31003/v1
|
|
```
|
|
|
|
## Notes
|
|
|
|
- Throughput: gamma ~20 tok/s, alpha1 ~10 tok/s, alpha2 ~14 tok/s
|
|
- alpha1/alpha2 identical models (SHA256 confirmed)
|
|
- gamma has MTP quant; alpha1/alpha2 have non-MTP
|
|
- All llama.cpp services run as nyx-organs:nimmerverse-agents
|
|
- OStimNet: Fable-Fusion needs explicit x-rated vocabulary in system prompts
|
|
|
|
## Service Files
|
|
|
|
`~/tmp/skyrimnet-analysis/services/` — llama-gamma.service, llama-alpha1.service, llama-alpha2.service
|