DevOpsLog.pl

The Art of Technical Troubleshooting

Category: AI

  • Posted on

Local LLM Setup with Docker Containers

This guide describes my setup for running a local LLM using Docker containers. The model and the frontend UI run in separate containers, connected via a Docker network. Stack Overview: Final Architecture (Docker Version) Install Docker Run Docker without sudo:https://devopslog.pl/2026/05/06/using-docker-without-sudo/(A reboot is required after configuration.) Model Storag & Recommended Engine Best place for the latest
Read More
  • Posted on

Local AI like ChatGPT via Docker Model Runner

The most powerful and convenient way to deploy AI with Docker is Docker Model Runner (DMR). High-level architecture and major components. Docker Model Runner – is a runtime process working outside the containers that provides models direct access to host hardware. OpenAI-compatible endpoints – the same API endpoints as the official OpenAI API uses. Provides
Read More
  • Posted on

Docker and AI in Containers using Ollama

Real Use Case Using Docker as an environment and Ollama as a containerized application with a specific model, e.g. Phi-3. What is Ollama A tool that lets you run LLMs locally. It is often based on the llama.cpp engine.It handles: It runs as a server on port 11434. Ollama inside Docker Docker creates a container
Read More

PROJECTS