HswOAuth/llm_course
19
16
Fork 0
Code Issues 608 Pull Requests Actions Packages Projects Releases Wiki Activity

模型训练数据加载问题 #436

New Issue
Closed
opened 2024-12-20 21:34:57 +08:00 by elysia · 1 comment
elysia commented 2024-12-20 21:34:57 +08:00
Copy Link

有一个神经网络的训练算法，用的arima，然后数据量大概有几亿条时序数据（大概半年的数据量），这么大的数据量应该如何去加载训练

有一个神经网络的训练算法，用的arima，然后数据量大概有几亿条时序数据（大概半年的数据量），这么大的数据量应该如何去加载训练
12019701659cs commented 2024-12-23 14:13:34 +08:00
Copy Link
  1. ARIMA 不是神经网络。
  2. 如果使用神经网络来处理大量时序数据，可以考虑分批加载 (Batch Loading)、内存映射 (Memory Mapping)、分布式训练之类的方法。
1. ARIMA 不是神经网络。 2. 如果使用神经网络来处理大量时序数据，可以考虑分批加载 (Batch Loading)、内存映射 (Memory Mapping)、分布式训练之类的方法。
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: HswOAuth/llm_course#436
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?