AI at Puget Sound: For Staff

Large‑language models predict the next word rather than retrieve verified data; when the context runs thin they invent material. This hallucination risk can surface as fake citations, imaginary statistics, or fabricated quotations.

Why Does AI Hallucinate?

Generative AI systems produce inaccurate content for three main reasons:

1. Training Data Sources: AI models learn from internet data that contains misinformation, so they reproduce falsehoods without checking facts.
2. How Generative Models Work: These systems predict what comes next based on patterns, focusing on plausible-sounding content rather than accuracy.
3. Design Limitations: The technology can't distinguish between true and false information, so it generates new inaccurate content by combining patterns.

AI hallucinations result from flawed training data, pattern-based generation, and fundamental technology limits.

How to keep control

• Treat AI output as a starting point, to help explore data and evidence, never a definitive source.
• Ask the model for DOIs or URLs, then check them; fake links break on inspection.
• Cross‑check facts in databases or peer‑reviewed literature.
• Document any claim you keep so readers can verify it.

Take‑away: always verify with an independent source.

Deeper dive: Park, D. M., & Lee, H. J. (2024). Literature Review of AI Hallucination Research Since the Advent of ChatGPT: Focusing on Papers from arXiv. Informatization Policy, 31(2), 3-38.

AI echoes its training data, which often favor dominant groups. This can appear as the omission of minoritized scholars, stereotyped language, or skewed sentiment analysis.

Why Does AI Produce Biased Content?

Generative AI systems produce biased content for three main reasons:

1. Training Data Sources: AI models learn from internet data that reflects existing social and cultural biases from human-created content.
2. How Generative Models Work: These systems reproduce patterns from their training data, including discriminatory language and perspectives they've observed.
3. Design Limitations: The technology can't identify or correct biases, so it amplifies existing prejudices by combining biased patterns in new ways.

AI bias results from biased training data, pattern reproduction, and the inability to recognize discriminatory content.

Spot and reduce bias

1. Prompt for multiple perspectives or under‑represented voices.
2. Ask the model to list its main sources and examine their diversity.
3. Compare AI summaries with scholarly reference sources or peer-reviewed sources.
4. Note in your assignment how you checked for bias.

Bottom line: inclusive prompts and critical reading can help reduce the impact of algorithmic echo chambers.

Deeper dive: Zhou, M., Abhishek, V., Derdenger, T., Kim, J., & Srinivasan, K. (2024). Bias in generative AI. arXiv preprint arXiv:2403.02726.

Public AI services often store prompts and reserve the right to review them. Some retrain on user data. Uploading sensitive material can violate privacy laws or research ethics.

Why Does AI Pose Privacy Risks?

AI systems create privacy concerns for users in three main ways:

1. Training Data Collection: AI models are trained on vast quantities of data, much of which is composed of copyrighted material, and using copyrighted works to train AI models may constitute prima facie infringement.
2. User Data Retention: Every query, instruction, or conversation with ChatGPT is stored indefinitely unless deleted by the user, including sensitive data like personal details and proprietary information.
3. Inadvertent Data Exposure: There is a risk of inadvertently capturing and exposing sensitive user data, particularly in chatbots where personal or confidential information is often disclosed.

AI privacy risks stem from unauthorized use of copyrighted training data, indefinite storage of user conversations, and potential exposure of sensitive information shared with AI tools.

Safeguards

• Strip names, IDs, and unpublished data.
• Avoid uploading copyrighted texts.
• Use university‑approved tools for institutional data.
• Consult your course professor or faculty advisor regarding high‑risk cases.

Key point: treat public AI tools like social media—anything you post could become public.

Deeper dive: Golda, A., Mekonen, K., Pandey, A., Singh, A., Hassija, V., Chamola, V., & Sikdar, B. (2024). Privacy and security concerns in generative AI: a comprehensive survey. IEEE Access.