NIUBOYI»Home › › 脚本分享 › 中文 | English
微信扫一扫 分享朋友圈

已有 7270 人浏览分享

脚本分享    Douyin comment scraping script: an easy way to scrape comments and save them to CSV

2024-7-9 00:22:587270 0
```python
#!/usr/bin/env python3
# coding:utf-8
import datetime
import csv
from DrissionPage import ChromiumPage

def save_comments_to_csv(comments, writer):
    for comment in comments:
        text = comment['text']
        nickname = comment['user']['nickname']
        create_time = datetime.datetime.fromtimestamp(comment['create_time']).strftime('%Y-%m-%d %H:%M:%S')
        digg_count = comment['digg_count']
        writer.writerow({'Nickname': nickname, 'Likes': digg_count, 'Time': create_time, 'Comment': text})
        print({'Nickname': nickname, 'Likes': digg_count, 'Time': create_time, 'Comment': text})

def main():
    with open('data.csv', mode='w', encoding='utf-8', newline='') as f:
        csv_writer = csv.DictWriter(f, fieldnames=['Nickname', 'Likes', 'Time', 'Comment'])
        csv_writer.writeheader()

        driver = ChromiumPage()
        driver.listen.start('aweme/v1/web/comment/list/')
        driver.get('https://v.douyin.com/ijUq5C6m/')
        # driver.get('https://v.douyin.com/ijUsDWgh/')

        for page in range(10):
            print(f'Collecting data from page {page + 1}')
            driver.scroll.to_bottom()
            resp = driver.listen.wait()
            json_data = resp.response.body
            save_comments_to_csv(json_data.get('comments', []), csv_writer)

if __name__ == '__main__':
    main()
```
  • WeChat Official Account

  • Scan to Visit Mobile Site

QQ|NIUBOYI ( 京ICP备17025393号 )|网站地图

GMT+8, 2026-10-30 19:46 , Processed in 0.043495 second(s), 38 queries .

POWERED BY NIUBOYIAI! 主理人:NIUBOYIAI  点亮哥

Copyright © 2001-2020, NIUBOYI AI.