Skip to main content

Local AI models with Ollama

Why local AI models?​

Ollama runs the model locally. Sign-in and chat storage use the cloud and require internet. There is no automatic switch to a cloud model.

That applies to normal mode. In offline mode everything stays on this computer: the app then also works without internet and new chats are stored only locally, encrypted.

Why Ollama?​

Ollama is the leading platform for running open-source AI models locally. Key features:

  • ✅ Easy installation and setup
  • ✅ Thousands of available models
  • ✅ Lightweight and fast
  • ✅ Cross-platform (Windows, macOS, Linux)
  • ✅ Simple model management
  • ✅ OpenAI-compatible API

Installation​

1. Download and install Ollama​

Visit ollama.ai and download the installer for your operating system.

2. Verify installation​

After installation, verify Ollama is working:

ollama --version

For Windows, restart your terminal after installation.

3. Pull a model​

Download a model (example with qwen2.5):

ollama pull qwen2.5:7b

This will download the model. Depending on your internet speed, this may take a few minutes.

Model installation​

Quick start​

For the best balance between quality and performance, we recommend:

ollama pull qwen3-vl:4b-instruct

Choose the -instruct variant: it answers right away. The plain qwen3-vl:4b thinks at length before every step and is many times slower on a laptop. For small local models the app uses a short instruction and only the tools that fit your question. If the same step keeps failing, the AI stops and tells you what worked and what did not.

This is the recommended model because it:

  • ✅ Requires only 4GB of RAM
  • ✅ Includes vision capabilities (see images)
  • ✅ Offers good speed and quality balance
  • ✅ Works well on most hardware
  • ✅ Fully open and free to use

Install other models​

You can install additional models:

# Other popular models
ollama pull llama2:7b       # Excellent all-purpose model
ollama pull mistral:7b      # Fast and capable
ollama pull neural-chat:7b  # Great for conversations

Hardware recommendations​

Ollama works on various hardware. Here's what you need for different models:

Model SizeRAM neededGraphics CardPerformance
3-4B4GB minimumNot requiredFast (5-10 tokens/sec)
7B8GB recommendedOptional (faster)Good (2-5 tokens/sec)
13B+16GB+ recommendedGPU strongly recommendedSlower without GPU

GPU acceleration: If you have an NVIDIA GPU, Ollama will automatically use it for faster inference.

Configuration in the desktop app​

After installing Ollama and models:

  1. Open the AI-School Desktop application
  2. Go to Settings → Local Models
  3. Check that Ollama is detected
  4. Select your model from the dropdown
  5. You're ready to use local AI!

On the start screen, the model selector appears directly below the chat input, so you can choose a local model before your first message. That selection is used from the first message and can still be changed later in the chat.

Available models​

Popular models available via Ollama:

Vision models (can see images)​

  • qwen3-vl:4b-instruct (recommended) - Fast vision model
  • llama2-vision:13b - More powerful vision model
  • minicpm-v:latest - Compact vision model

Text models​

  • qwen2.5:7b - Excellent for all tasks
  • llama2:7b - Classic, well-tested
  • mistral:7b - Fast and efficient
  • neural-chat:7b - Conversational focus
  • openchat:7b - Good all-rounder

Specialized models​

  • codegemma:7b - For programming tasks
  • sqlcoder:7b - SQL database queries
  • dolphin-mixtral:8x7b - Powerful mixture model

Start with qwen3-vl:4b-instruct and explore other models based on your needs!

Reading documents and scans locally​

With a local model, the desktop app reads documents on your own computer. Nothing is uploaded.

  • Work folder: in an approved folder the AI reads Word, PowerPoint, Excel, PDF, OpenDocument (ODT, ODP, ODS) and RTF. Old .doc and .ppt files are only read when LibreOffice is installed.
  • Scans and photos: scanned PDF pages, images (PNG, JPG, WebP, TIFF) and images inside Word and PowerPoint files are read with text recognition (OCR). The OCR comes with the app and recognises Dutch, English, German and French. The chat shows which page is being read; a long scan can take a few minutes. At most 100 pages per document are read with OCR.
  • Uncertain recognition: when text recognition is uncertain, the AI says so. Check names, numbers and amounts in the original. Handwriting, tables and diagrams are not always recognised correctly.
  • Vision model: if the selected model has vision (such as qwen3-vl:4b-instruct), the model also looks at images where text recognition reads little or is uncertain. Without a GPU this takes longer.
  • iPhone photos (HEIC): save them as JPEG or PNG first.

With a cloud model the app also reads documents in the work folder on your computer, but the text it reads is then sent to that model as part of the conversation.

Working without internet, including attachments in the chat: see Offline mode.

On this page