JungHwan Yang, a professor of communication at the University of Illinois Urbana-Champaign, is a co-principal investigator on a research project and a website — socialpolls.org — that examine the informal polls about the U.S. presidential election posted on the social media platform X, formerly called Twitter.
Social media users may be exposed to numerous informal polls during the election. How much faith should they place in the results?
We examined nearly 2,000 polls posted on X that ostensibly gauged support for U.S. presidential candidates during the 2016 and 2020 elections.
While social media polls appear like traditional opinion surveys, they are not accurate measures of public opinion. Instead, they are indicators of political engagement among specific subsets of the population and serve a distinct function.
This distinction is important because the observed opinions on social media are inherently biabosed.
What are some of ways in which social media polls are biased?
We found that these polls are biased in terms of their content, authorship and audience.
First, they are not scientific. Traditional polls are carefully designed as to who is asked the questions and how the surveys are distributed. Social media polls are purely based on people’s voluntary engagement because they are not asking questions of a representative sample of the electorate.
For example, if former President Donald Trump is posting a poll, it is very likely that his followers will vote rather than people who do not follow him.
Based on certain events, the level of public engagement also can change drastically. For example, the failed assassination attempt on Trump could mobilize his supporters on a social media platform.
Sometimes the survey questions are biased. Creators do not ask the question in an objective way because they want a certain outcome.
By amplifying some negative aspects of a candidate, they can also skew the responses. For example, they might have asked, “Since Joe Biden’s cognitive health is deteriorating, who do you think the next president of the United States should be?”
And the inclusion of a third or fourth candidate in a multiple-choice format can change the results, as can the order of the questions and the responses.
What have you found in your research about those who create these polls?
We scraped data from each poll’s author profile as well as those of their followers. We collected all the profiles of poll authors and their followers who retweeted and tagged each poll as a favorite. Finally, we used machine learning algorithms coupled with human verification to identify these social media users’ demographic and political characteristics.
The social media polls in the 2016 and 2020 election cycles were predominantly crafted by older males and manifested a pronounced bias for Trump, whereas traditional polls favored the Democratic candidates to win.
Those who post these social media polls control when they put them on sites and when they close the posts, giving them a lot of control over generating the results they want.
Your study indicates that the creators of these polls purchase votes from bot farms overseas. How prevalent is that? And do social media companies — some of which claim they block false information — address it?
It is common to see social media polls accumulate thousands of votes very quickly.
Bot farms and astroturfing campaigns can create fake accounts that follow the instructions of the people who purchase their services. Bad actors who want a certain outcome can buy a lot of votes from bots. As far as we know, social media companies do not monitor these activities and are not transparent about the extent to which this occurs on their platforms.
False impressions created by social polls can reduce the quality of political information on social media and further erode trust in these platforms.
How do these polls affect social media users’ perceptions of the election information reported by mainstream media?
In a second study we found that some poll questions allude to mistrust in mainstream media. For example, the poll authors might say, “Given that the traditional media polls are biased, let’s measure the real public opinion with this.” Poll questions hint about conspiracies with voter fraud, rigged elections and follow that kind of narrative.
It is important to acknowledge the potential impact of these polls in general discussions among the public and in the media. We know that many conservative people in the U.S. have very low trust in mainstream media and information coming from them — but they trust the social polls they engage with.
For example, they might think “I voted on an X poll last week, and in that poll, Trump is winning by 75%. But in the poll published in The New York Times, Trump’s winning by only 55% — so mainstream media are biased.”
If you consider all the things happening right now in the media, the government, politics and social media, this can be impactful.
What is the purpose of the Social Polls website your team created?
The Social Polls website is a platform to communicate our research findings to the public. We are examining polls conducted over social media to see how the results change daily. We also publish social media poll results after we adjust for bias using sample reweighting methods. And we are constantly updating as we approach the election.
There is a similar website called fivethirtyeight.com, which is run by statistician Nate Silver, but it is mostly collecting traditional opinion polls that are more systematic and scientific by design.
The paper “Analyzing support for U.S. presidential candidates in Twitter polls” is available online.