LLama3仿OpenAI实验，命令行测试成功，但是NextChat不行 #241
Labels
No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: HswOAuth/llm_course#241
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
1命令行测试结果成功
(base) root@autodl-container-eded47b3e0-b7a6df75:~# curl http://172.17.0.10:19327/v1/chat/completions
-H "Content-Type: application/json"
-H "Authorization: Bearer x"
-d '{
"model": "llama-3-chinese",
"messages": [
{
"role": "user",
"content": "你是谁"
}
],
"max_tokens": 4096
}'
{"id":"chatcmpl-zMQyHvkJ5j9e92qHWaxzVN","object":"chat.completion","created":1729404828,"model":"llama-3-chinese","choices":[{"index":0,"message":{"role":"user","content":"你是谁"}},{"index":1,"message":{"role":"assistant","content":"我是一名人工智能语言模型，我的名字叫做Open Assistant。我被设计用来回答各种问题，并与用户进行对话。我可以提供信息、解答疑问和帮助完成任务等等。请随时向我提问，我会尽力为您服务！"}}]}
2但是NextChat测试不行，如图
3IP查找如下，请老师帮忙看看，可能是哪里的问题
(base) root@autodl-container-eded47b3e0-b7a6df75:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.0.10 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:ac:11:00:0a txqueuelen 0 (Ethernet)
RX packets 5317 bytes 4901794 (4.9 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5776 bytes 7502271 (7.5 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 7891 bytes 15887669 (15.8 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7891 bytes 15887669 (15.8 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
是AutoDL上租用3090卡，在自己的电脑上运行NextChat
检查一下有没有打通道，你在服务器上是可以访问本机的接口的，但是你电脑只能访问服务器的ssh端口
打了隧道以后，nextchat里面的ip要填写为http://127.0.0.1:19327
进行问答时，服务器返回一下问题
Exception in thread Thread-2:
Traceback (most recent call last):
File "/root/autodl-tmp/conda/envs/chinese_llama_alpaca_3/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/root/autodl-tmp/conda/envs/chinese_llama_alpaca_3/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/root/autodl-tmp/conda/envs/chinese_llama_alpaca_3/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/autodl-tmp/conda/envs/chinese_llama_alpaca_3/lib/python3.8/site-packages/transformers/generation/utils.py", line 1758, in generate
result = self._sample(
File "/root/autodl-tmp/conda/envs/chinese_llama_alpaca_3/lib/python3.8/site-packages/transformers/generation/utils.py", line 2455, in _sample
unfinished_sequences = unfinished_sequences & ~stopping_criteria(input_ids, scores)
File "/root/autodl-tmp/conda/envs/chinese_llama_alpaca_3/lib/python3.8/site-packages/transformers/generation/stopping_criteria.py", line 507, in call
is_done = is_done | criteria(input_ids, scores, **kwargs)
TypeError: unsupported operand type(s) for |: 'Tensor' and 'NoneType'
解决了，问题出在 定义 StopOnEvent 的时候最后一行代码，return false与上面的return true并列了，使得程序报错。应该去掉前面的空格