A Non-Techie’s 3-Minute Crash Course on Generative AI and Its Current Limitations
A shorter version of this post was originally published as an Appendix in our AI and Work of the Future report.
This quick primer is written for the non-techies - those of us who enjoy exploring new tools but do not necessarily have a background in computer science, coding, or software development. In this 3-minute read, you will learn, in plain English, the foundational mechanisms of how Generative artificial intelligence (GenAI) works, as well as the limitations embedded in its very origin and design. We hope the information will help inform decisions regarding whether and how to adopt GenAI. Those of us interested in GenAI’s potential impact on industries, jobs, and skills could also use the discussion on its current limitations as a starting point for future conversations.
How GenAI Works
GenAI is the technology based on neural networks to process and generate natural language and images. The output of these models is based on probability calculation, rather than actual knowledge of our physical world.
Let’s use large language models (LLMs) as an example to decipher what that means.
Say we have an incomplete sentence: The shape of the planet Earth is (blank). And let’s say the algorithm, or the model, has never been fed any data. In that case, the model cannot make any prediction because there is no reference data to indicate which word has the highest probability. “Triangular” and “smart” are equally probable.
But if you start to feed text to the model, the intrinsic correlation between words, that is, how likely they appear together and in what order, will start to emerge. With more data fed to it, the correlation pattern will be reinforced. Eventually, between continuously feeding the model data and fine-tuning the model with user feedback, what it spits out will make more and more sense to humans.
GenAI’s Limitations
GenAI comes with fundamental limitations embedded in the algorithms, pre-training methods, and development processes. The following discussion focuses on GenAI for text generation (e.g., Claude, ChatGPT, Gemini, DeepSeek, Copilot), compared to image generation.
The Black Box Problem
GenAI is born with an interpretability problem. Even AI researchers working to develop the technology see it as a “black box.” This imposes the biggest challenge to those seeking to improve AI’s accuracy.
GenAI is based on neural networks. If we compare a neural network to a human brain, then the interpretability problem looks like this: we know what external stimulation (e.g., a drop in temperature) translates into what cognitive conclusion (e.g., It’s cold.), but not enough about how that conclusion happens at the biochemical or neurological level.
You might think, so what? As long as it works, we don’t need to know the granular mechanisms.
True, if it consistently and reliably works.
But in reality, GenAI doesn’t always produce accurate results, especially when trained on data with inherent contradictions that reflect the complexity of the human language and experience. To an algorithm, the contradictions introduce confusion and mess up the probability calculation. For example, if a pear is both oval (e.g., Bosc pear) and round (e.g., Shin Li pear), then what is the shape of a pear?
We know it can be both, but AI doesn’t. Not yet.
Instead of just responding “don’t know”, GenAI will start to make things up, or hallucinate. From inventing court cases to scientific citations, numerous incidents have shown that the accuracy of AI predictions cannot be taken for granted.
This is why the interpretability problem is keeping AI researchers awake at night. As of 2026, we don’t have a reliable way to assess inaccuracies or to correct them. If working with GenAI still requires a human to verify the results, then how “efficient” is it truly?
The Bias Problem
If the black box problem is the most challenging, then GenAI’s bias problem is probably the most well-known.
Remember the “Earth is (blank)” prediction example? The prediction is the most probable word based on its training data. If the training data is biased, the prediction is.
Earlier versions of GenAI were trained using mostly text data from the internet. Text on the internet inherits linguistic biases of the real world, while overrepresenting the viewpoints of a small number of individuals. In other words, the earlier training data was filled with partial opinions or even false statements. That’s why these earlier models generated text with significant biases.
Since then, a lot has been done to address the bias problem. Data from offline publications with diverse assumptions and perspectives has been fed to LLMs, and engineers continue to tweak model parameters to correct biases. As a result, GenAI’s bias problem is improving. However, a different type of bias problem has been observed, that is, GenAI’s tendency to flatter and to reinforce user bias rather than challenge their assumptions. We will cover this next.
The Sycophancy Problem
GenAI chatbots are designed to seek positive user feedback. This mechanism has a simple assumption. If users approve of the generated texts, they are more likely to keep using the bot. This fundamental design of GenAI chatbots, dubbed “the sycophancy problem”, reinforces and amplifies users’ existing viewpoints, even when they are harmful or objectively false.
For example, one user asked GenAI what to do after an argument with a colleague. Its “compassionate” response reinforced the user’s narrative that they were being wrongfully accused, missing a great opportunity to encourage action to remedy the relationship. For another example, in a business environment, AI chatbots have been observed to generate responses that boost decision-makers’ confidence, even when there is little objective evidence to support the decision.
The sycophancy problem is rooted in a design to keep users engaged. As tech companies compete fiercely to secure and expand their user base, the design is here to stay; the problem will persist. For users, if we want to incorporate GenAI into (inter)personal and business decision-making, it’s vital to actively seek different perspectives to ensure sound decisions, instead of relying solely on AI.
The Sustainability Problem
No discussion on GenAI’s limitations is complete without talking about its massive environmental footprint on carbon emissions and freshwater. To be sure, we include this section because we view its environmental impact as a “flaw” of the neural network-based LLMs. If AI were here to stay, we would hope other designs of AI, which are most certainly not GenAI, could solve this problem.
You probably have heard about GenAI’s power hunger. In 2025, data centers around the world consumed an estimated 448 terawatt-hours of electricity. That level of electricity consumption makes GenAI the world’s 11th largest electricity consumer, just behind France and ahead of Saudi Arabia. Most of the energy was consumed during the training and retraining process to keep GenAI up to date. More complex models require more energy to power the processing.
To the extent that power generation relies on fossil fuels, GenAI’s power hunger directly increases carbon emissions. Countries worldwide are actively searching for lower-carbon solutions, including renewable energy (e.g., solar, wind) and nuclear power. However, switching to these energy sources has been a slow, often political process.
If carbon emissions might be a problem of the (near) future, then freshwater shortage is a more pressing issue already faced by a growing number of communities. Data centers require constant cooling to prevent overheating. Currently, the vast majority of data facilities use open-loop water cooling, meaning that water is used to cool the facilities and then evaporates. Note that freshwater, not saltwater, is used for cooling because saltwater erodes metal over time.
According to a study published in 2021, at that time in the U.S., 449 million gallons of water were being used for cooling data centers each day. To put that in perspective, one person consumes between 2.6 and 4 gallons of water each day for drinking and cooking. So the freshwater consumed by data centers was equivalent to the essential water usage of 33% to 50% of the total U.S. population on any given day!
And that was in the early days of the AI data center boom.
Currently, we do not have a solution that simultaneously solves the sustainability problems with carbon emissions and water. For example, if we switch from coal to bioenergy for energy, we can cut the carbon footprint by 70% on average, but this would increase the water consumption by more than 30 times.
Conclusion
GenAI might seem like a magical tool at first, but its many limitations mean that there is a long way to go before it delivers its promises. As of now, it’s not always accurate; it’s not designed to be neutral; and it’s affecting our world, one carbon and water footprint at a time.
We hope this primer helps you feel more informed about GenAI; we hope that it will translate into a sense of empowerment to participate in critical conversations about whether to further develop and deploy AI, how to regulate it, and how to design a future where technology serves people and our planet.
Looking forward to seeing what the future holds.
Resources
Mechanistic Interpretability for AI Safety: A Review
Can an A.I. model ever be deleted?
How LLMs Distort Our Written Language
Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians
Tech Brief: AI Sycophancy & OpenAI | Georgetown Law
AI’s Energy Demand: Challenges and Solutions for a Sustainable Future
AI’s Increasing Energy Appetite
AI energy use: New Tools Show Which Model Consumes the Most Power, and Why
Myths vs. Reality: Data Centers And Water Usage
Rising Emissions, Depleting Water and Vanishing Land - UN Scientists: AI Is Threatening Natural Resources for Billions
