Artificial Intelligence: a guide for students

F&M Library Guide: Evaluate Sources

Accuracy and source Check

❏ Verify all facts, statistics, and data points with multiple reputable sources (academic journals, government publications, respected news organizations); don’t rely on a single source 

❏ If the AI cites sources, check those sources directly; if the AI cannot cite a source, disregard the information 

❏ Watch for outdated information; confirm the publication date of source materials 

❏ Try the same prompts in multiple AI tools and traditional search engines and compare the results 

Bias Check

❏ Ensure the content presents multiple perspectives and does not omit certain viewpoints or promote stereotypes 

❏ Check original sources to see if they favor a specific viewpoint, ideology or group 

❏ Use critical thinking to evaluate information that seems slanted to serve business, government or advocacy interests, or influence buying decisions 

Logical consistency check

❏ Ensure the arguments being presented flow logically and make sense 

❏ Watch for broad generalizations from limited evidence 

❏ Look carefully for contradictions or misleading jumps in reasoning 

❏ Assess the depth of reasoning and avoid simplistic analysis 

Emotional and manipulative language check

❏ Look for content that uses neutral, fact-based language; be wary of language that is overly dramatic or inflammatory 

❏ Watch for loaded words designed to provoke emotions (fear, anger, excitement) or influence opinions or actions

Generative AI tools create natural human language responses to human language prompts.  Prompt responses given by generative AI may be influenced by biased or inaccurate content in its training data or include inaccuracies created by the generative AI tool.  An initial evaluation of the outputs of generative AI can be similar to how information found online can be evaluated, with a series of questions.

Who is the author? Could their view be biased in any way?

Text or images generated by AI tools have no human author, but they are trained on materials created by humans with human biases.  Unlike humans, AI tools cannot reliably distinguish between biased material and unbiased material when using information to construct their responses.

What was the intended audience?

Generative AI tools can be used to generate content for any audience based on the user’s prompt.  Does an internet search of the generative AI tool's answer bring up results that are directed as specific audiences?

What is the intended purpose of the content?  Was it created to inform, to make money, to entertain?   

Generative AI tools can create convincing text and images that can be used to propagate many different ideas without being clear that the information or images could be false.

Where was it published?  Was it in a scholarly publication, a website, or an organization page?

Generative AI has already been used to create content for websites and news outlets. Considering whether the source is scholarly, has a good reputation and a clear history of providing reliable information is useful for figuring out whether the information you find is useful or misleading.

Does it provide sources for the information?

Articles, news outlets, and websites that provide sources could be an indicator of reliability.  Further assessing the sources by following the links and citations to verify the information will help confirm that the information you find is reliable.

Hallucinations and Fake News

Generative AI natural language processing tools, language models, or chatbots like ChatGPT have been shown to hallucinate or provide completely unsubstantiated information. While this has improved since it's inception, there are still some cases of this. Text generated by AI can also seem very confident, so it can be very difficult to ascertain what information generated by AI is trustworthy and what information is not. As the end user of a generative AI's deliverables, it's up to you to discern quality information and identify issues in an Generative AI's response.

--University of South Florida Libraries