HswOAuth/llm_course
19
16
Fork 0
Code Issues 608 Pull Requests Actions Packages Projects Releases Wiki Activity

关于VLM课程中使用xinference部署本地模型 #303

New Issue
Open
opened 2024-10-26 21:51:51 +08:00 by 21970855250cs · 1 comment
21970855250cs commented 2024-10-26 21:51:51 +08:00
Copy Link

使用glm-4v 9b模型部署接入fastgpt是可以的，
模型部署需要使用8bit量化模型（大概需要21G显存），否则会出现OOM
image

image image
使用glm-4v 9b模型部署接入fastgpt是可以的， 模型部署需要使用8bit量化模型（大概需要21G显存），否则会出现OOM <img width="1440" alt="image" src="/attachments/5500637e-30be-4d36-ae3f-e7e8fe57e2a1"> <img width="1440" alt="image" src="/attachments/b7d89a2b-de61-4df5-b151-437a49348c2f"> <img width="1440" alt="image" src="/attachments/1fb7a9fc-8219-4b90-9741-385917f22d19">
image.png
241 KiB
image.png
774 KiB
image.png
287 KiB
👍 1
21970855250cs commented 2024-10-27 11:44:30 +08:00
Author
Copy Link

另外关于gradio_gpt4o.py脚本，已经添加了 记录历史会话信息的逻辑
修改后可以将历史会话信息记录下来
image

另外关于gradio_gpt4o.py脚本，已经添加了 记录历史会话信息的逻辑 修改后可以将历史会话信息记录下来 <img width="1043" alt="image" src="/attachments/b5c79200-54f0-48f5-9a55-c2f174835b87">
image.png
1.7 MiB
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: HswOAuth/llm_course#303
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?