Applied Studies with Botometer

Question

Hi,

I have testet three datasets with Botometer and meassured the percentage of the accounts that had a botmeter score above 3.

- In dataset one 4.9 % of the account scored above 3.
- In dataset two 5.6 % of the account scored above 3.
- In dataset three 15.9 % of the account scored above 3.

The third dataset seems quite more "robotic" than the others. But my question is how much more? Is 15.9% accounts many, compared to what is expected?

I have seen many studies discussing the precision of Botometer, but I am more interested in applied studies to be able to compare my results. Any suggestions?

Best regards

Observatory on Social Media · Answer

It is hard to tell because "what is expected" depends largely on the domains of your datasets. We advise that you inspect these accounts to gauge these percentages and determine how to interpret them.

Comparing the percentages (or better, the distributions) of the different groups is IMHO much more meaningful than labeling individual accounts. You may not be able to say "dataset 3 has high bot activity", but you should be able to say "dataset 3 displays higher levels of automation than dataset 1".

Finally, the score threshold of 3 is somewhat arbitrary. Consider setting a threshold on the CAP, so that it is easier to interpret (as a p-value) as explained in the FAQ on the Botometer website.

Hope this helps,
-Botometer Team

Botometer Pro

9.3 / 10

1,134ms

100%

N/A

Applied Studies with Botometer