Docker Model Runner: context size
To change the default context size of a model, which is typically 4096 tokens, use the following command:
docker model configure --context-size <num> <model_name>Example usage:
docker model configure --context-size 8192 ai/qwen2.5-coderNOTE!
If you run a model after increasing the context size, RAM usage can immediately spike to the maximum amount reserved for that context size, even if the conversation only contains a small number of tokens, like 500, because the runtime often reserves memory upfront for the full possible KV cache capacity rather than only for the currently used tokens.