ABVX Agent Skills capability

Local Inference Tuning

Choose and tune the right local LLM engine for the machine in front of you, from Apple Silicon MLX to GGUF and CUDA serving stacks.

GitHub skill Skills catalog

What it does

Audits hardware, accelerators, RAM/VRAM, model inventory, and approved storage volumes.
Selects the right engine for the hardware and model format instead of defaulting to one stack.
Builds a model-fit estimate before downloading or converting weights.
Defines cache location, batching, KV cache, concurrency, memory, and endpoint settings.
Verifies with smoke benchmarks and OpenAI-compatible API checks.

Positioning

A practical inference setup checklist for local/private agent fallback work.

The source of truth lives in markoblogo/abvx-agent-skills; ABVX Lab only catalogs the capability and points to install paths.

Quickstart

Install and use

gh skill install markoblogo/abvx-agent-skills local-inference-tuning --agent codex --scope user

Use local-inference-tuning. Audit this machine and design a local LLM inference setup with engine choice, model fit, cache policy, tuned flags, smoke benchmark, and OpenAI-compatible endpoint recipe.