HswOAuth/llm_course
19
16
Fork 0
Code Issues 608 Pull Requests Actions Packages Projects Releases Wiki Activity

第30次课-爬虫失败 #671

New Issue
Open
opened 2025-04-01 15:15:01 +08:00 by 15555422276 · 3 comments
linhui commented 2025-04-01 16:12:21 +08:00
Copy Link

这个应该是天眼查的cookie未设置正确，导致爬取的数据为空

这个应该是天眼查的cookie未设置正确，导致爬取的数据为空
15555422276 commented 2025-04-01 16:33:32 +08:00
Author
Copy Link

这个应该是天眼查的cookie未设置正确，导致爬取的数据为空

我在淘宝上买了一个小时，爬取了10%就给禁掉了，有什么办法吗

> 这个应该是天眼查的cookie未设置正确，导致爬取的数据为空 我在淘宝上买了一个小时，爬取了10%就给禁掉了，有什么办法吗
linhui commented 2025-04-01 17:01:47 +08:00
Copy Link

因为天眼查对反爬有比较严格的限制，可以考虑两种方式

  1. 每爬一条数据的时候让程序sleep的时间更久一些，如下图所示，可以改成30秒爬取一次image
  2. 可以去购买天眼查的官方API获取它的数据，https://open.tianyancha.com/api_list
因为天眼查对反爬有比较严格的限制，可以考虑两种方式 1. 每爬一条数据的时候让程序sleep的时间更久一些，如下图所示，可以改成30秒爬取一次![image](/attachments/7ddd3ce0-aba7-40d0-9cda-1ff2e73ad3a1) 2. 可以去购买天眼查的官方API获取它的数据，https://open.tianyancha.com/api_list
image.png
12 KiB
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: HswOAuth/llm_course#671
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?