DevOpsLog.pl

The Art of Technical Troubleshooting

Category: Containerization

  • Posted on

Local LLM Setup with Docker Containers

This guide describes my setup for running a local LLM using Docker containers. The model and the frontend UI run in separate containers, connected via a Docker network. Stack Overview: Final Architecture (Docker Version) Install Docker Run Docker without sudo:https://devopslog.pl/2026/05/06/using-docker-without-sudo/(A reboot is required after configuration.) Model Storag & Recommended Engine Best place for the latest
Read More
  • Posted on

Local AI like ChatGPT via Docker Model Runner

The most powerful and convenient way to deploy AI with Docker is Docker Model Runner (DMR). High-level architecture and major components. Docker Model Runner – is a runtime process working outside the containers that provides models direct access to host hardware. OpenAI-compatible endpoints – the same API endpoints as the official OpenAI API uses. Provides
Read More
  • Posted on

Docker Model Runner: context size

To change the default context size of a model, which is typically 4096 tokens, use the following command: Example usage: NOTE! If you run a model after increasing the context size, RAM usage can immediately spike to the maximum amount reserved for that context size, even if the conversation only contains a small number of
Read More
  • Posted on

using Docker without sudo

Have you seen this error: Run Docker commands without sudo 1. Add the docker group if it doesn’t already exist 2. Add the connected user $USER to the docker group Optionally change the username to match your preferred user. IMPORTANT: Log out and log back in so that your group membership is re-evaluated. 3. Restart
Read More
  • Posted on

Docker and AI in Containers using Ollama

Real Use Case Using Docker as an environment and Ollama as a containerized application with a specific model, e.g. Phi-3. What is Ollama A tool that lets you run LLMs locally. It is often based on the llama.cpp engine.It handles: It runs as a server on port 11434. Ollama inside Docker Docker creates a container
Read More
  • Posted on

Docker Compose – Running a Multi-Container App

In short, Compose is a tool (now shipped by default with Docker) that allows you to easily define and run multi-container applications using a YAML file. The structure of the YAML file is defined by Compose. The Compose file should be called compose.yaml or compose.yml. Its structure includes the following top-level keys: Digging deeper:The core
Read More
  • Posted on

Docker Self Healing – Restart Policies

Restart policies are rules for restarting containers. There are four:●​ no (default)●​ on-failure●​ always●​ unless-stopped In the table: Y = Restart, N = No Restart How to use it To apply these policies, use --restart <policy> when running a container: or when update:

PROJECTS