Install Maki and Run Your First Agent

By the end of this chapter you will have an agent answering a question on your own machine. You need the setup from chapter 1: Maki installed, Ollama running and qwen3.5:4b pulled. If you would rather use llama.cpp or vLLM, the last section shows how; the program is the same.

The whole program

This is ch02_first_agent/scout.py. Run it with python ch02_first_agent/scout.py.

"""Chapter 2: your first agent.

Run it with:  python ch02_first_agent/scout.py
Needs a model server running (Ollama by default, see chapter 1).
To use llama.cpp or vLLM instead:  SCOUT_BACKEND=llamacpp python ch02_first_agent/scout.py
"""
import os

from maki import init_backend
from maki.agents import Agent

DEFAULT_SPEC = "ollama:qwen3.5:4b"   # 3.4 GB. Bigger machine: "ollama:gemma4:26b".


def make_backend(spec=None):
    """The model Scout talks to, named by a "provider:model" string.

    Set SCOUT_BACKEND to use another server, for example "llamacpp" or
    "vllm:Qwen/Qwen2.5-1.5B-Instruct". Nothing else in the program changes.
    """
    spec = spec or os.environ.get("SCOUT_BACKEND", DEFAULT_SPEC)
    # think=False is an Ollama setting (explained in the chapter); other backends lack it.
    options = {"think": False} if spec.split(":")[0] == "ollama" else {}
    return init_backend(spec, **options)


def build_agent(backend=None):
    """Scout, a research assistant. Pass a backend to swap the model."""
    return Agent(
        name="Scout",
        maki_instance=backend or make_backend(),
        role="research assistant",
        instructions="Answer in three short sentences. Say so when you are unsure.",
    )


def main(backend=None):
    scout = build_agent(backend)
    answer = scout.execute_task("In plain words, what is a multi-agent LLM application?")
    print(answer)
    return answer


if __name__ == "__main__":
    main()

The first run is the slowest, because Ollama has to load the model into memory; on the author's Mac it took about 25 seconds in all. Later runs are faster, because the model is already loaded. You should see three short sentences about multi-agent applications. The exact words change every run.

What each part does

make_backend() builds the connection to the model from one string, "provider:model". The default is "ollama:qwen3.5:4b", which means Ollama on your machine. Maki has the same kind of object for other servers and for hosted models, and an agent does not care which one you give it. That is what lets the last section switch servers without touching the agent.

For Ollama the connection is made with think=False, which turns off the model's "thinking" mode. qwen3.5 is a thinking model: before it answers, it writes out its reasoning. For a simple task that is slow, and when a page is long it can use up the whole budget on reasoning. Chapter 4 shows what goes wrong without it.

Agent(...) is where the behaviour lives:

name identifies the agent in logs.
role is a short description of what it is.
instructions is what you want from it. Keep these specific: "answer in three short sentences" is something you can check, "be helpful" is not.
maki_instance is the model object you just made.

execute_task(...) sends one task and returns the answer as text.

Try it

Change the instructions to "Answer in one sentence." and run it again. Then change the question. You are not changing any code that talks to the model, only the agent's behaviour.

Then try a bigger model. If you have gemma4:26b pulled, change DEFAULT_SPEC to "ollama:gemma4:26b" and run the script again. Maki does not change; only the object it uses to reach the model does.

When it does not work

Connection refused: Ollama is not running. Start it, then run python ch01_setup/check_setup.py.
Model not found: run ollama pull qwen3.5:4b.
Very slow: the first run loads the model. If every run is slow, the model is too large for your memory; use the 4B model.

An agent that only answers from what the model already knows is a chatbot. In the next chapter you give it a tool, so it can look things up.

Use llama.cpp or vLLM instead of Ollama

Ollama is the easiest way to start, but it is not the only way to run a model on your own hardware. Maki talks to two other local servers the same way, and Scout does not change: you only tell make_backend() which one to use, through the SCOUT_BACKEND environment variable.

Server	What it is	Pick it when
Ollama	A model manager with a one-command download	You want the simplest start. This is the course default
llama.cpp	A lean server for GGUF model files	You already have GGUF files, or want few moving parts
vLLM	A high-throughput server for GPUs	You have a GPU machine, and many requests at once

Both servers speak the same OpenAI-style API, so Maki needs one extra package for either, which is only the openai client library.

llama.cpp

pip install "maki-framework[llamacpp]"
llama serve -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0

llama serve downloads the model the first time and then answers on http://127.0.0.1:8080/v1. In another terminal:

SCOUT_BACKEND=llamacpp python ch02_first_agent/scout.py

A server started with one model ignores the model name, so llamacpp alone is enough. If you started the server without a model (router mode), name the one you want: SCOUT_BACKEND=llamacpp:ggml-org/gemma-3-4b-it-qat-GGUF:Q4_0.

vLLM

pip install "maki-framework[vllm]"
vllm serve Qwen/Qwen2.5-1.5B-Instruct

vLLM answers on http://127.0.0.1:8000/v1. Here the model name matters: it must match the name the server exposes, which is the model you gave vllm serve, or the name you set with --served-model-name.

SCOUT_BACKEND=vllm:Qwen/Qwen2.5-1.5B-Instruct python ch02_first_agent/scout.py

If the server is somewhere else

Maki reads the server's address from MAKI_LLAMACPP_BASE_URL or MAKI_VLLM_BASE_URL, set before you start the script. If the server was started with an API key, set LLAMACPP_API_KEY or VLLM_API_KEY.

Check it first

Chapter 1's check understands these too: run SCOUT_BACKEND=llamacpp python ch01_setup/check_setup.py and it tells you whether the server is answering.

What is different from Ollama

think=False is an Ollama setting. make_backend() only passes it to Ollama. If a thinking model on another server is slow or rambles, see the server's own option for turning thinking off.
The model is chosen when you start the server, not by the script, except in llama.cpp's router mode and with vLLM's model name.
The later chapters work the same way, since they use the same make_backend(). A model that is too small for tool calls is the usual reason chapter 3 misbehaves, on any server.

A note on what has been checked. The Maki side of both examples runs in the course's tests against a stand-in server that speaks the same API. The llama serve and vllm serve commands follow Maki's documentation and were not run for this chapter, so if a command or model name has changed in the meantime, the servers' own documentation wins.