COVID-19-TweetIDs collects and archives Twitter tweet IDs associated with the COVID-19 pandemic (SARS-CoV-2). The project leverages Twitter's APIs to gather Tweets based on keywords and accounts, providing a publicly available dataset for research. We are primarily utilizing the Twitter Streaming API to capture real-time data and the Search API to retrieve historical data. The data is released for non-commercial research purposes, and we meticulously adhere to Twitter's Terms of Service.
This repository distinguishes itself by offering a continuously updated and expanding dataset of COVID-19 related tweets. It provides detailed information on data collection methodology, error handling, and hydration techniques. The inclusion of statistics such as language breakdowns and missing data periods enhances the dataset's transparency and usability for researchers.
- Data Collection: Historical and real-time tweets collected via Twitter APIs.
- Data Format: Tweet IDs organized into year/month folders, with daily files.
- Hydration Tools: Includes instructions and scripts for hydrating raw tweet IDs.
- Documentation: Provides a detailed description of the data, usage agreements, and statistics.
- API Usage: Details configurations and best practices for accessing Twitter's API.
- Data Updates: Regularly updated with new data entries.
- Language Breakdown: Includes detailed language distribution of tweets.
The project is actively maintained with ongoing data collection and updates at least every 2-3 weeks. The development team is responsive to API changes and has migrated data collection infrastructure to AWS to increase data volume. The repository includes detailed documentation and issue tracking, indicating a commitment to data quality and accessibility. The project is not formally archived and is expected to be maintained for the foreseeable future.
Researchers, public health analysts, and social scientists can utilize this dataset to study public sentiment, information diffusion, and the spread of misinformation related to COVID-19. It provides a valuable resource for analyzing social media trends during the pandemic, supporting a deeper understanding of its impact. The complete dataset allows for broad analysis and allows for investigation of real-time, short-term trends.