HswOAuth/llm_course
19
16
Fork 0
Code Issues 608 Pull Requests Actions Packages Projects Releases Wiki Activity

13-24.9.5-私有化LLM仿OpenAI API接口的高可用工程实践 课后作业 #30

New Issue
Open
opened 2024-09-06 23:35:11 +08:00 by 12422372205cs · 0 comments
12422372205cs commented 2024-09-06 23:35:11 +08:00
Copy Link

环境：
windows宿主机
docker搭建ubuntu完成实验

Tip：
配置完OneAPI令牌后，NextChat需要重启聊天框才生效。

BUG：
--load_in_8bit 在高并发下会报错（ output += torch.matmul(subA, state.subB RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x2 and 1x1024)）

作业完成

image
image

环境： windows宿主机 docker搭建ubuntu完成实验 Tip： 配置完OneAPI令牌后，NextChat需要重启聊天框才生效。 BUG： --load_in_8bit 在高并发下会报错（ output += torch.matmul(subA, state.subB RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x2 and 1x1024)） ## 作业完成 ![image](/attachments/4ebf1233-c549-4567-b5b4-9c41396eb535) ![image](/attachments/3b2a87e0-e17c-4f4f-85eb-7e6c4924a177)
image.png
59 KiB
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: HswOAuth/llm_course#30
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?