【求助】微调训练出错，报错信息和截屏如下 #372
Labels
No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: HswOAuth/llm_course#372
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
[INFO|configuration_utils.py:670] 2024-11-12 16:28:15,297 >> loading configuration file /root/autodl-tmp/modelscope/hub/ZhipuAI/glm-4-9b-chat/config.json
[INFO|configuration_utils.py:670] 2024-11-12 16:28:15,300 >> loading configuration file /root/autodl-tmp/modelscope/hub/ZhipuAI/glm-4-9b-chat/config.json
[INFO|configuration_utils.py:739] 2024-11-12 16:28:15,301 >> Model config ChatGLMConfig { "_name_or_path": "/root/autodl-tmp/modelscope/hub/ZhipuAI/glm-4-9b-chat", "add_bias_linear": false, "add_qkv_bias": true, "apply_query_key_layer_scaling": true, "apply_residual_connection_post_layernorm": false, "architectures": [ "ChatGLMModel" ], "attention_dropout": 0.0, "attention_softmax_in_fp32": true, "auto_map": { "AutoConfig": "configuration_chatglm.ChatGLMConfig", "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForCausalLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSequenceClassification": "modeling_chatglm.ChatGLMForSequenceClassification" }, "bias_dropout_fusion": true, "classifier_dropout": null, "eos_token_id": [ 151329, 151336, 151338 ], "ffn_hidden_size": 13696, "fp32_residual_connection": false, "hidden_dropout": 0.0, "hidden_size": 4096, "kv_channels": 128, "layernorm_epsilon": 1.5625e-07, "model_type": "chatglm", "multi_query_attention": true, "multi_query_group_num": 2, "num_attention_heads": 32, "num_hidden_layers": 40, "num_layers": 40, "original_rope": true, "pad_token_id": 151329, "padded_vocab_size": 151552, "post_layer_norm": true, "rmsnorm": true, "rope_ratio": 500, "seq_length": 131072, "tie_word_embeddings": false, "torch_dtype": "bfloat16", "transformers_version": "4.45.0", "use_cache": true, "vocab_size": 151552 }
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,307 >> loading file tokenizer.model
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,307 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,307 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,307 >> loading file tokenizer_config.json
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,307 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2478] 2024-11-12 16:28:15,839 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO|configuration_utils.py:670] 2024-11-12 16:28:15,840 >> loading configuration file /root/autodl-tmp/modelscope/hub/ZhipuAI/glm-4-9b-chat/config.json
[INFO|configuration_utils.py:670] 2024-11-12 16:28:15,841 >> loading configuration file /root/autodl-tmp/modelscope/hub/ZhipuAI/glm-4-9b-chat/config.json
[INFO|configuration_utils.py:739] 2024-11-12 16:28:15,843 >> Model config ChatGLMConfig { "_name_or_path": "/root/autodl-tmp/modelscope/hub/ZhipuAI/glm-4-9b-chat", "add_bias_linear": false, "add_qkv_bias": true, "apply_query_key_layer_scaling": true, "apply_residual_connection_post_layernorm": false, "architectures": [ "ChatGLMModel" ], "attention_dropout": 0.0, "attention_softmax_in_fp32": true, "auto_map": { "AutoConfig": "configuration_chatglm.ChatGLMConfig", "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForCausalLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSequenceClassification": "modeling_chatglm.ChatGLMForSequenceClassification" }, "bias_dropout_fusion": true, "classifier_dropout": null, "eos_token_id": [ 151329, 151336, 151338 ], "ffn_hidden_size": 13696, "fp32_residual_connection": false, "hidden_dropout": 0.0, "hidden_size": 4096, "kv_channels": 128, "layernorm_epsilon": 1.5625e-07, "model_type": "chatglm", "multi_query_attention": true, "multi_query_group_num": 2, "num_attention_heads": 32, "num_hidden_layers": 40, "num_layers": 40, "original_rope": true, "pad_token_id": 151329, "padded_vocab_size": 151552, "post_layer_norm": true, "rmsnorm": true, "rope_ratio": 500, "seq_length": 131072, "tie_word_embeddings": false, "torch_dtype": "bfloat16", "transformers_version": "4.45.0", "use_cache": true, "vocab_size": 151552 }
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,845 >> loading file tokenizer.model
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,845 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,845 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,845 >> loading file tokenizer_config.json
[INFO|tokenization_utils_base.py:2212] 2024-11-12 16:28:15,845 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2478] 2024-11-12 16:28:16,365 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[WARNING|arrow_dataset.py:3098] 2024-11-12 16:28:17,203 >> num_proc must be <= 7. Reducing num_proc to 7 for dataset of size 7.
损失
在提供的日志和截图里面没有看到相关的报错信息，日志中的WARNING可以忽略
那怎么会训练出错呢
请问这个dataset是您自己设置的嘛 还是根据课程操作手册来弄的？
哪个dataset?json文件吗？这个是按操作手册来写的
为方便协助调试，请点击”预览数据集“按钮，看看数据库是否合法？最好是录制整个出错的过程，包括控制台的报错，谢谢。
请老师看看，多谢
另外推荐你查看这个帖子：#308，有可能是您启动了多个webui，在正常情况下，webui默认是监听7860端口，只有在你已经移动了一个webui，7860被占用的时候，才会选择使用7861端口。
隧道没问题，原来可以训练的json文件现在也加载不了，到底怎么回事啊
该json文件的格式，与上课时候提供的JSON文件，内部的文件组织形式不一样。建议，先一模一样按照老师的教程做，然后再使用自己的数据集。另外：数据集格式需要和老师的格式一样。
我的json文件直接用的你的文件，格式也是按你们的做的，现在连调取都调取不了，到底是什么鬼啊，这么多天了怎么都解决不了
请问是否按照教程，更新了dataset_info.json文件？另外请将数据集命名为my_demo.json，确保和老师上课时所有操作一致。您可以上传dataset_info.json给我，我排查一下是否有问题
谢谢，可以了，但还有个问题，我要添加别的对话信息，老师提供的json文件里键值对不够表达，我怎么添加别的关键词？怎么保证格式的正确？
只要保持和老师的json文件字段名一样即可，也就是键值对中的键保持和老师的文件一直即可。