012、基于docker容器平台构建运行AI大模型，灵活微调各类大模型

字数统计: 201字   |   阅读时长≈ 1分

使用Docker运行AI大模型

1
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

如何在容器中运行基于GPU的大模型

需要安装驱动的，所以安装驱动吧
Nvidia安装地址: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#基于Debain系列服务器
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
    | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
    | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
    | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit




#基于RedHat系列服务器
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo \
    | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo yum install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker