Botometer Twitter
Merih Angın
Merih Angın is an Assistant Professor at the International Relations Department of Koç University. Previously, she was a Postdoctoral Fellow at the Weatherhead Center for International Affairs of Harvard University, a postdoctoral research fellow at the Blavatnik School of Government of the University of Oxford, and a visiting scholar at the Mortara Center for International Studies of the Edmund A. Walsh School of Foreign Service at Georgetown University. Dr. Angın holds a PhD degree in International Relations/Political Science from the Graduate Institute of International and Development Studies (IHEID), an M.Sc. degree in International Relations from METU, and a Bachelor’s degree in Economics from Bilkent University. Her research interests lie in the areas of international political economy, international organizations, international development, international financial institutions, investment arbitration, political economy of privatization, migration, quantitative methods, agent-based modelling, machine learning, artificial intelligence and computational social sciences. Her research on IMF lending has recently been awarded the European Commission’s Marie Skłodowska-Curie Actions Individual Fellowship, as well as the Scientific and Technological Research Council of Turkey’s International Fellowship for Outstanding Researchers for 3 years.Botometer is a supervised machine learning tool, which means it learns the characteristics of likely automated and human accounts from annotated datasets. As time goes by, bots on Twitter evolve and novel behaviors emerge, which requires us to retrain the model with new annotated datasets periodically. His thesis focuses on the analysis of manipulation and threats on social media and he was awarded the 2018 University Distinguished Ph.D. Dissertation Award. He has developed a system called Botometer to detect social bots on Twitter and his team ranked top 3 worldwide at. Botometer® checks Twitter accounts for possibly automated activity.
Photo by Morning Brew on Unsplash
A couple years back, I have writtena short Python code that retrieves all the followers for a specific account and check for bots usingBotometer. However, the Botometer has gone through somemajor changes which renders the original code obsolete. It is time to create something new!
Many new tools have become available since, includingStreamlit. How to burn a dmg on a pc. According to Streamlit’s developers:
Streamlit’s open-source app framework is the easiest way for data scientists and machine learning engineers to create beautiful, performant apps in only a few hours! All in pure Python. All for free.
Although it might be an overkill, I have decided to create a Streamlit app that serves as a wrapper over the new Botometer API. It essentially provides two simple functions: 1) Collect all the followers for a given account and store them in an SQLite database and 2) Loop through each account and post the data to the Botometer API and receive bot check data. SQLite database is natively supported by Python so no need to run a separate service.
How to use
Download and set up the Streamlit app
The app is built with Python 3.7.4 (although other Python versions such as 3.7.x and 3.6.x should also work). It is easier to start fresh with a new virtual environment (e.g. “conda create –name botcheck python=3.7” if you are using conda), then running:
Now, you should be able to see the app at:http://localhost:8501
Get the Twitter and Botometer Rapid API credentials
Please seethis page for setting up the credentials. There is a free plan for testing that allows us to check up to 500 accounts per day. If you need more quota, their other paid plans would be ideal.
Botometer Pro
Use the app to retrieve followers and check for bots
Potometer And Transpiration
- Select the JSON file that contains the Twitter and Botometer credentials.
credentials.json
is a template of the credential file, which can be found with the app. Credential loaded will be cached for later use. - Provide an existing database. If not provided, the app will look for database in the temp folder. If no cached database available, a new database will be created.
- Specify the account whose followers need to be checked, and set the timeframe you want to keep the bot check data.
- Run “Retrieve Twitter followers”, which uses Tweepy API to retrieve all the followers of a specific account. If a database is provided or cached in the temp folder, the app will only add new followers that are not already in the database. If no database is available, the app will create a new database from scratch.
- Run “Check bot”, which call the Botometer Rapid API to check for bots in the database. It will not re-check the followers unless the bot data is expired. A download link will be available at the bottom of the sidebar when the process is completed. Alternatively, you can click the “Download bot check results” button to get the download link.
Results
Please checkthis section for the details of the output from bot check. You may also find thisblog post andthis paper from the developer of Botometer useful.