大模型提示词进阶版作业 #121

New Issue

GANGUAGUA · 2024-09-24T16:28:37+08:00

GANGUAGUA commented

2024-09-24 16:28:37 +08:00

下载中文数据

从 huggingface mirror 网站上找到想要的中文数据集并复制

在notebook里下载数据集：

huggingface-cli download jtjt520j/CSpider_sql_create_context --repo-type dataset --revision main --local-dir-use-symlinks False --local-dir /code/jtjt520j/CSpider_sql_create_context

设置环境变量

export HF_HOME=/code/huggingface-cache/
export HF_ENDPOINT=https://hf-mirror.com
export http_proxy=http://10.10.9.50:3000
export https_proxy=http://10.10.9.50:3000
export https_proxy=http://10.10.9.50:3000

API-Key 准备

按照对应的环境变量设置：

export ZHIPUAI_API_KEY=xxx_xxx

训练前提问大模型

使用FastChat部署codellama-7b-hf模型

# 按顺序依次执行

# 新建终端，输入如下命令

python -m fastchat.serve.controller --host 0.0.0.0

# 新建终端，输入如下命令

python -m fastchat.serve.model_worker --model-path /dataset/CodeLlama-7b-hf/ --host 0.0.0.0 --num-gpus 4 --max-gpu-memory 15GiB

# 新建终端，输入如下命令

python -m fastchat.serve.openai_api_server --host 0.0.0.0

提问大模型

curl -X POST http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "CodeLlama-7b-hf",
"prompt": "You are a powerful text-to-SQL model. Your job is to answer questions about a database. You are given a question and context regarding one or more tables. You must output the SQL query that answers the question. ### Input: 部门的最高预算和最低预算是多少？ ### Context: CREATE TABLE department (budget_in_billions INTEGER) ### Response:",
"max_tokens": 41,
"temperature": 0.5
}'

大模型回答不正确

微调开始

在第一台机器上输入：

NPROC_PER_NODE=4 NNODES=2 PORT=12345 ADDR=10.244.34.79 NODE_RANK=0 xtuner train llama2_7b_chat_qlora_sql_e3_copy.py --work-dir /code/xtuner-workdir --deepspeed deepspeed_zero3_offload

在第二台机器上输入：

NPROC_PER_NODE=4 NNODES=2 PORT=12345 ADDR=10.244.34.79 NODE_RANK=1 xtuner train llama2_7b_chat_qlora_sql_e3_copy.py --work-dir /code/xtuner-workdir --deepspeed deepspeed_zero3_offload

微调结果

## 下载中文数据 从 huggingface mirror 网站上找到想要的中文数据集并复制 ![](https://cdn.nlark.com/yuque/0/2024/png/48118617/1727162953058-de075f12-6753-4e27-9e6a-aa8abcad4cc7.png) 在notebook里下载数据集： ```plain huggingface-cli download jtjt520j/CSpider_sql_create_context --repo-type dataset --revision main --local-dir-use-symlinks False --local-dir /code/jtjt520j/CSpider_sql_create_context ``` ## 设置环境变量 ```plain export HF_HOME=/code/huggingface-cache/ export HF_ENDPOINT=https://hf-mirror.com export http_proxy=http://10.10.9.50:3000 export https_proxy=http://10.10.9.50:3000 export https_proxy=http://10.10.9.50:3000 ``` ## API-Key 准备 ![](https://cdn.nlark.com/yuque/0/2024/png/48118617/1727163681534-60c2a58b-8abc-40d5-a08d-2aa9a7a0699c.png) 按照对应的环境变量设置： ```plain export ZHIPUAI_API_KEY=xxx_xxx ``` ## 训练前提问大模型 ### **使用FastChat部署codellama-7b-hf模型** # 按顺序依次执行 # 新建终端，输入如下命令 ```plain python -m fastchat.serve.controller --host 0.0.0.0 ``` # 新建终端，输入如下命令 ```plain python -m fastchat.serve.model_worker --model-path /dataset/CodeLlama-7b-hf/ --host 0.0.0.0 --num-gpus 4 --max-gpu-memory 15GiB ``` # 新建终端，输入如下命令 ```plain python -m fastchat.serve.openai_api_server --host 0.0.0.0 ``` ### 提问大模型 ```plain curl -X POST http://localhost:8000/v1/completions \ -H "Content-Type: application/json" \ -d '{ "model": "CodeLlama-7b-hf", "prompt": "You are a powerful text-to-SQL model. Your job is to answer questions about a database. You are given a question and context regarding one or more tables. You must output the SQL query that answers the question. ### Input: 部门的最高预算和最低预算是多少？ ### Context: CREATE TABLE department (budget_in_billions INTEGER) ### Response:", "max_tokens": 41, "temperature": 0.5 }' ``` 大模型回答不正确 ![](https://cdn.nlark.com/yuque/0/2024/png/48118617/1727165958030-8d5947c4-7502-4fa2-929b-92e4b0603cc9.png) ## 微调开始在第一台机器上输入： ```plain NPROC_PER_NODE=4 NNODES=2 PORT=12345 ADDR=10.244.34.79 NODE_RANK=0 xtuner train llama2_7b_chat_qlora_sql_e3_copy.py --work-dir /code/xtuner-workdir --deepspeed deepspeed_zero3_offload ``` 在第二台机器上输入： ```plain NPROC_PER_NODE=4 NNODES=2 PORT=12345 ADDR=10.244.34.79 NODE_RANK=1 xtuner train llama2_7b_chat_qlora_sql_e3_copy.py --work-dir /code/xtuner-workdir --deepspeed deepspeed_zero3_offload ``` ## 微调结果 ![](https://cdn.nlark.com/yuque/0/2024/png/48118617/1727165059362-3e8ecade-1c39-4e06-b0e9-ec9bd6dd3789.png)

Sign in to join this conversation.