HswOAuth/llm_course
19
16
Fork 0
Code Issues 608 Pull Requests Actions Packages Projects Releases Wiki Activity

【求助帖】关于调用qianwen2.5模型加速的问题 #345

New Issue
Open
opened 2024-11-04 14:49:11 +08:00 by guozhushu · 1 comment
guozhushu commented 2024-11-04 14:49:11 +08:00
Copy Link

最近尝试调用qianwen2.5 70B 模型的api，请求一次，响应结果为45秒，请问有什么办法进行加速处理呢？

最近尝试调用qianwen2.5 70B 模型的api，请求一次，响应结果为45秒，请问有什么办法进行加速处理呢？
21970855250cs commented 2024-11-04 23:49:26 +08:00
Copy Link

调用的是阿里的api，还是自己部署的模型？
如果是阿里的线上模型，可能是网络模型；
如果是自己部署的模型，硬件以及部署方式需要提供下看看

调用的是阿里的api，还是自己部署的模型？ 如果是阿里的线上模型，可能是网络模型； 如果是自己部署的模型，硬件以及部署方式需要提供下看看
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: HswOAuth/llm_course#345
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?