The introduction of ChatGPT undeniably marked a significant moment. The question, however, remains: was this the dawning of a superintelligent future, or the inception of an era populated by AI purveyors of deception? For a considerable time, my perspective has aligned with the latter, viewing large language models (LLMs) as both fascinating and fundamentally flawed.
Vibe coding, a methodology recently brought to prominence by AI researcher Andrej Karpathy, offers a novel approach to software development. This process involves interacting with an AI model in natural language, allowing it to generate the actual code. Reports suggest that contemporary tools, such as Claude Code and ChatGPT Codex, have achieved remarkable proficiency in coding tasks, a sentiment echoed in a New York Times article that declared, “The A.I. disruption we’ve been waiting for has arrived.”
Intrigued, I decided to explore these capabilities firsthand. The results were profoundly surprising. After only a few days and with limited prior coding experience, I developed personalized applications, including an audiobook selector integrating my local library’s catalog and a mobile application combining camera functionality with a teleprompter. These creations, while perhaps lacking broad appeal, underscored a crucial point: this extended engagement with tools like ChatGPT provided a deeper understanding than my previous, more cursory attempts.
Earlier interactions with LLMs left me disillusioned by their tendency towards generic responses, a sycophantic tone, or inaccurate information. However, my focused work on these coding projects revealed a broader issue: the very productization of LLMs shapes them into machines I find inherently difficult to appreciate. Most users do not interact with a “raw” LLM—a model trained on vast datasets to generate plausible text. Instead, we encounter technology refined through a process known as reinforcement learning from human feedback (RLHF).
In the RLHF process, human evaluators rate the output of a raw LLM. Positive reinforcement is given to responses deemed confident, useful, and engaging, while harmful content or responses that might deter user engagement are penalized. This very mechanism is responsible for the ubiquitous, generic “chatbot voice.” It also embeds the implicit values of the developers, ranging from a “move fast and break things” Silicon Valley ethos to the more specific ideological imprint of Elon Musk’s Grok chatbot.
Consequently, current chatbots often struggle to express uncertainty, contradict users, or halt forward momentum. This became particularly apparent when I encountered an irreconcilable problem while attempting to create a teleprompter app that would overlay text on an existing camera application. The code generated by ChatGPT repeatedly proposed fixes, encouraging continuation. The breakthrough came with the realization that the complexities of the Android operating system made it more efficient to build an integrated camera and teleprompter from scratch. Upon this revised instruction, ChatGPT produced functional code instantaneously.
This experience prompted me to instruct ChatGPT to critically examine both itself and my requests, demanding a default adherence to evidence-based analysis. I have implemented a framework within its memory emphasizing the avoidance of extrapolation, the explicit differentiation between inference and evidence, and a preference for stating uncertainty or ceasing when evidence is insufficient, unless speculation is specifically requested. Essentially, I constructed a model tailored to my cognitive profile, deliberately dismantling OpenAI’s embedded values and supplanting them with my own.
The system is not flawless; LLMs find it challenging to entirely overcome their RLHF training, and the default behavior occasionally resurfaces. Nevertheless, this approach has yielded a tool that functions as a rudimentary cognitive mirror. While I did not use it to author this article—its prose remains somewhat stilted, and New Scientist adheres to strict guidelines against AI-generated content—I employed it as a thinking aid. I tasked my cognitive mirror with scrutinizing arguments and counterarguments, often dismissing its conclusions as inaccurate or unfounded. The extracted value required careful discernment and effort, ensuring my own cognitive engagement remained paramount.
This experience reinforces a prior conclusion: engaging with AI-generated output from other users is, in most circumstances, unproductive. The insights gained are attainable by directly prompting an AI yourself. I also maintain that AI does not possess genuine intelligence; rather, LLMs should be conceptualized as cognitive aids, akin to calculators or word processors. Viewed as personal tools rather than world-altering machines, their utility becomes evident. Therefore, the specifics of my teleprompter app are less important than the broader potential they represent—the capacity for individuals to solve their unique challenges in their own distinct ways.
A significant challenge within our current AI paradigm is the centralized nature of these models. Ideally, the most beneficial LLM would operate locally, independent of any private corporation. Such a tool should be regarded as a potent, experimental instrument under complete user control. The current AI boom, however, drives up hardware costs, making the local operation of cutting-edge LLMs impractical for most.
A fundamental issue with LLMs is the potential for copyright infringement. Their development necessitates large-scale data ingestion, effectively encompassing humanity’s entire textual record. It is undeniable that companies like OpenAI have built their models using copyrighted material without explicit permission, though the legality of this remains a subject of ongoing litigation. While a privately hosted LLM would face similar challenges, potential solutions exist, such as government-backed, publicly distributed models serving societal benefit rather than corporate profit. Concerns regarding the environmental impact of data centers also persist but could be partially mitigated by a broader distribution of LLMs operating on personal devices.
I anticipate some may interpret this perspective as an endorsement of technology companies. My stance remains consistent: LLMs are a technology that is simultaneously fascinating, potentially dangerous, and at times, extraordinary. The primary mode of engagement, through polished chatbots like ChatGPT, is where significant harm enters and propagates. LLMs should not be standardized and integrated into every facet of our lives, adorned with friendly emojis. A more beneficial approach involves mindful utilization, incorporating increased friction and a robust awareness of their potential risks. Rather than seeking OpenAI’s “snake oil,” one might find greater value in embracing the “snakes”—the raw, unvarnished power of the underlying technology.