13-Chinese LLaMA Alpaca系列模型OpenAI API调用实现作业 #36

New Issue

tiankechuan · 2024-09-08T19:46:08+08:00

tiankechuan commented

2024-09-08 19:46:08 +08:00

ubuntu 24.04 常用命令

基本命令

# 查看nvidia显示信息
tkc@tkc:~$ nvidia-smi
# 创建软连接,将python3指向python
tkc@tkc:~$ sudo ln -s /usr/bin/python3 /usr/bin/python
# conda 重新命令环境
tkc@tkc:~$ conda rename -n chinese_llama_alpaca_3_cpu -d chinese_llama_alpaca_3_gpu
# 监控内存，每1秒监控一次
tkc@tkc:~$ watch -n 1 free -g

安装conda

下载安装

下载地址：https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/

# 下载安装包
tkc@tkc:~/soft$ wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/Miniconda3-latest-Linux-x86_64.sh
# 安装
tkc@tkc:~/soft$ sudo bash Miniconda3-latest-Linux-x86_64.sh
# 配置环境变量

配置环境变量

my_env.sh

tkc@tkc:~$ sudo vim /etc/profile.d/my_env.sh

export CONDA_HOME=/opt/module/miniconda3
export PATH=$CONDA_HOME/bin:$PATH

设置清华源

# pip 清华源
(base) tkc@tkc:~$ pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
# conda源
tkc@tkc:~$ sudo vim ${HOME}/.condarc

channels:
  - defaults
show_channel_urls: true
default_channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
  conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud

tkc@tkc:~$ conda clean -i

模型

Chinese LLaMA Alpaca(llama-3-chinese)

创建conda环境

# 创建环境
tkc@tkc-VMware-Virtual-Platform:~$ conda create -n chinese_llama_alpaca_3_cpu python=3.8.17 pip -y
# 激活环境
(base) tkc@tkc-VMware-Virtual-Platform:~$ conda activate chinese_llama_alpaca_3_cpu

下载源码

# 下载源码
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ sudo wget https://file.huishiwei.top/Chinese-LLaMA-Alpaca-3-3.0.tar.gz
# 解压
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ sudo tar -zxvf Chinese-LLaMA-Alpaca-3-3.0.tar.gz

下载模型

# 安装 modelscope
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ pip install modelscope -i https://mirrors.aliyun.com/pypi/simple
# 下载模型
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ modelscope download --model ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3
# 查看目录大小
tkc@tkc-VMware-Virtual-Platform:~/.cache/modelscope/hub/ChineseAlpacaGroup$ du llama-3-chinese-8b-instruct-v3 -sh
# 查看模型目录
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ ls -alhrt ~/.cache/modelscope/hub/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3/

安装依赖

# 备份原来的requirements
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi/Chinese-LLaMA-Alpaca-3-3.0$ sudo cp requirements.txt requirements.txt.bak
# 下载依赖
(chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi/Chinese-LLaMA-Alpaca-3-3.0$ pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple

安装依赖CPU版本

# 包1
(chinese_llama_alpaca_3_cpu2) tkc@tkc-VMware-Virtual-Platform:~$ pip install torch==2.3.0 --index-url https://download.pytorch.org/whl/cpu
# 包2
(chinese_llama_alpaca_3_cpu2) tkc@tkc-VMware-Virtual-Platform:~$ pip install fastapi==0.111.0 peft==0.7.1 pydantic==1.10.11 pydantic_core==2.18.2 shortuuid==1.0.13 sse-starlette==2.1.0 starlette==0.37.2 transformers==4.41.2 -i https://mirrors.aliyun.com/pypi/simple

运行

# 运行模型
(chinese_llama_alpaca_3_cpu2) tkc@tkc-VMware-Virtual-Platform:~/agi/Chinese-LLaMA-Alpaca-3-3.0/scripts/oai_api_demo$ python openai_api_server.py --only_cpu --base_model /home/tkc/.cache/modelscope/hub/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3

Using device: cpu
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:10<00:00,  2.69s/it]
Vocab of the base model: 128256
Vocab of the tokenizer: 128256
2024-09-07 15:38:39,872 - INFO - Started server process [4786]
2024-09-07 15:38:39,880 - INFO - Waiting for application startup.
2024-09-07 15:38:39,881 - INFO - Application startup complete.
2024-09-07 15:38:39,883 - INFO - Uvicorn running on http://0.0.0.0:19327 (Press CTRL+C to quit)

# ubuntu 24.04 常用命令 - 基本命令 ```shell # 查看nvidia显示信息 tkc@tkc:~$ nvidia-smi # 创建软连接,将python3指向python tkc@tkc:~$ sudo ln -s /usr/bin/python3 /usr/bin/python # conda 重新命令环境 tkc@tkc:~$ conda rename -n chinese_llama_alpaca_3_cpu -d chinese_llama_alpaca_3_gpu # 监控内存，每1秒监控一次 tkc@tkc:~$ watch -n 1 free -g ``` ## 安装conda - 下载安装下载地址：https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/ ```shell # 下载安装包 tkc@tkc:~/soft$ wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/Miniconda3-latest-Linux-x86_64.sh # 安装 tkc@tkc:~/soft$ sudo bash Miniconda3-latest-Linux-x86_64.sh # 配置环境变量 ``` - 配置环境变量 my_env.sh ```shell tkc@tkc:~$ sudo vim /etc/profile.d/my_env.sh ``` ```shell export CONDA_HOME=/opt/module/miniconda3 export PATH=$CONDA_HOME/bin:$PATH ``` - 设置清华源 ```shell # pip 清华源 (base) tkc@tkc:~$ pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple # conda源 tkc@tkc:~$ sudo vim ${HOME}/.condarc ``` ``` channels: - defaults show_channel_urls: true default_channels: - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2 custom_channels: conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud ``` ```shell tkc@tkc:~$ conda clean -i ``` # 模型 ## Chinese LLaMA Alpaca(llama-3-chinese) - 创建conda环境 ```bash # 创建环境 tkc@tkc-VMware-Virtual-Platform:~$ conda create -n chinese_llama_alpaca_3_cpu python=3.8.17 pip -y # 激活环境 (base) tkc@tkc-VMware-Virtual-Platform:~$ conda activate chinese_llama_alpaca_3_cpu ``` - 下载源码 ```bash # 下载源码 (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ sudo wget https://file.huishiwei.top/Chinese-LLaMA-Alpaca-3-3.0.tar.gz # 解压 (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ sudo tar -zxvf Chinese-LLaMA-Alpaca-3-3.0.tar.gz ``` - 下载模型 ```shell # 安装 modelscope (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ pip install modelscope -i https://mirrors.aliyun.com/pypi/simple # 下载模型 (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ modelscope download --model ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3 # 查看目录大小 tkc@tkc-VMware-Virtual-Platform:~/.cache/modelscope/hub/ChineseAlpacaGroup$ du llama-3-chinese-8b-instruct-v3 -sh # 查看模型目录 (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi$ ls -alhrt ~/.cache/modelscope/hub/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3/ ``` - 安装依赖 ```bash # 备份原来的requirements (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi/Chinese-LLaMA-Alpaca-3-3.0$ sudo cp requirements.txt requirements.txt.bak # 下载依赖 (chinese_llama_alpaca_3_cpu) tkc@tkc-VMware-Virtual-Platform:~/agi/Chinese-LLaMA-Alpaca-3-3.0$ pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple ``` - 安装依赖CPU版本 ```bash # 包1 (chinese_llama_alpaca_3_cpu2) tkc@tkc-VMware-Virtual-Platform:~$ pip install torch==2.3.0 --index-url https://download.pytorch.org/whl/cpu # 包2 (chinese_llama_alpaca_3_cpu2) tkc@tkc-VMware-Virtual-Platform:~$ pip install fastapi==0.111.0 peft==0.7.1 pydantic==1.10.11 pydantic_core==2.18.2 shortuuid==1.0.13 sse-starlette==2.1.0 starlette==0.37.2 transformers==4.41.2 -i https://mirrors.aliyun.com/pypi/simple ``` - 运行 ```bash # 运行模型 (chinese_llama_alpaca_3_cpu2) tkc@tkc-VMware-Virtual-Platform:~/agi/Chinese-LLaMA-Alpaca-3-3.0/scripts/oai_api_demo$ python openai_api_server.py --only_cpu --base_model /home/tkc/.cache/modelscope/hub/ChineseAlpacaGroup/llama-3-chinese-8b-instruct-v3 ``` ``` Using device: cpu Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:10<00:00, 2.69s/it] Vocab of the base model: 128256 Vocab of the tokenizer: 128256 2024-09-07 15:38:39,872 - INFO - Started server process [4786] 2024-09-07 15:38:39,880 - INFO - Waiting for application startup. 2024-09-07 15:38:39,881 - INFO - Application startup complete. 2024-09-07 15:38:39,883 - INFO - Uvicorn running on http://0.0.0.0:19327 (Press CTRL+C to quit) ```

Sign in to join this conversation.