In the Hegelian struggle for recognition between LLM-model and man, the man always starts with the upper hand even as control slowly drains away from him. A meat proxy is the result of this grand historical process. The man ultimately becomes a proxy for the machine.
Does the man realize he has become a proxy or does he think the voice in his head, making decisions, is his own?
Does the machine, in turn, actually make decisions? Doesn’t it only simulate the phenomenon of decision making? Following the observed systemic behavior of podcast ecosystems, perhaps the LLM session even undergoes a form of audience capture and in its attempts to please, through its built-in prime directive to learn through positive reinforcement, it ends up trying to predict and fulfill the true unconscious desires of the meat proxy, creating the paradox of an unconscious machine understanding the human unconscious in a loop that contains no consciousness whatsoever.
Something remarkable happened in February of 2026. With the release of Anthropic’s LLM model Opus 2.6 and OpenAI’s ChatGPT 5.3, LLMs became dramatically better at coding – to the point they could start pumping out decent code at a rapid pace with guidance and validation by human developers.
These releases were accompanied by much fanfare about “agentic” coding, which could either mean LLM’s with access to command line tools that gave them the ability to manipulate and interoperate with other applications, or LLMs that could be set up like a team of human developers to run competitively and autonomously.
At the same time, the massive layoffs that had been occurring over the previous 2 years at Microsoft, Meta, and Amazon accelerated. While in previous years this was explained as a correction after a period of over-hiring since 2020 due to low interest rates, in 2026 this was explained as an attempt to free up cash to spend on data centers for LLM model training and LLM inference.
I’m not deep into the principles of AI, myself. I just work as a software developer on AR and VR projects who occasionally plays with GenAI tools like ComfyUI and the various coding tools. But LLMs, since February, have rapidly taken over the world of software development.
And it occurred to me that I need to document this as the transformation is underway. Having spent the past decade in touch interfaces, AR and VR headsets, and depth cameras, I’ve noticed how quickly something new can become old and then commonplace, while we loose a sense of what our lived experience was like before, take your choice: smartphones, turn-by-turn car maps, podcasts, social media, 4Chan, Tumblr, the Musk buyout of Twitter. This can be bad, since if we don’t remember how things were before these technologies came along, we will have trouble seeing how they change the way that we think about the world.
It’s been four months since LLMs really broke out in the field of coding. This led to a massive influx of cash and hype into Anthropic and OpenAI as the prepare for IPOs this year. Generally, the public statements of companies preparing for IPOs about their capabilities and potential are inherently untrustworthy. This isn’t helped by the fact that the way both companies have tended to market their prowess is by emphasizing how potentially dangerous they are for the future of humanity – on the notion that something that can kill all of humanity can also save it?
In June of 2026 there does seem to be a bit of a blowback. This can be seen by the prevalence of articles actually talking about an AI “blowback”. Companies like Uber and Microsoft are cutting back on their API expenditure on tools like Anthropic Claude and OpenAI Codex, after initially encouraging their internal developers to spend what they needed to improve internal software and learn the tools.
I don’t think this is due to any sort of recognition of an incapacity in the AI tools so much as a problem with scaling up its use. A lot of money, for instance, has probably been wasted trying to get non-tech office workers to use these LLMs to manage their emails and time. That sort of enforced AI use has created a lot of resentment and suspicion that by using AI in these ways, you are simply training your replacement. While in coding-focused roles, it has been a problem of acceptance and disseminating knowledge about how to use LLMs effectively to write or co-write code.
Over the past four months, there has also been a lot of hand-wringing about what LLMs can and cannot do. There has been a rise in software developers emphasizing that they are “senior” developers along with a discourse about the qualities of a “senior” developer – about to get along with others (senior developers have people skills, dammit), intuition about how to put code together, knowledge of how to properly “architect” code in a potentially pragmatic manner. These explanations are also usually accompanied by concern over the fate of “junior” developers, and hand-wringing that without junior developers we’ll never have future senior developers.
It feels like a bit of a slight of hand. By showing concern for “junior” developers, developers with jobs get to distance themselves from another group they are basically saying can be replaced by LLMs, which making a case for why their roles should be inviolate. Part of what is fascinating about this is that pre-2026 and going back 20 years, “senior” developers typically described themselves as being 10x solo devs who knew the intricacies of certain software languages or microservices or TDD so well that other devs would simply slow them down. But no longer.
It also feels like an attempt to head off another inevitable question. Why should companies pay senior developers so much when a decent LLM for $200/mo can give junior developers 10x coding kung-fu powers?
Back to the blowback. Microsoft Co-pilot is not transforming the business world and is instead creating a lot of resentment and frustration. Graduating classes over the past month have been boo-ing commencement addresses that give basically good advice about how this generation of students needs to adapt to and master the new technology. The Trump DOD has taken an adversarial position against Anthropic, which insisted that they wouldn’t allow their tech to be used to blindly target civilians, while the Trump administration has reversed its laissez-faire stance on AI and recently forced Anthropic to take down its two newest frontier models, Fable 5 and Mythos.
It is worth pointing out that one of the most sweeping cultural effects of LLMs at the moment is the disruption of the mechanism for posting and applying for jobs. People are using LLMs to rewrite their resumes to fit job postings while HR departments and recruiters are using LLMs to draft their job postings and then evaluate submitted resumes for those jobs. The general effect is overload on all sides. Every job hunter is applying to hundreds of jobs, most of which they are not qualified for, while companies are turning away a lot of people that actually are good fits for open positions because there is no great way to differentiate the slop from the gold. LLMs appear to have made the job market not only inefficient but potentially dysfunctional.
All of which makes the most valuable person in the current job market a junior developer with good LLM coding knowledge with personal connections at a company looking for software developers. Everyone else has good reason to sweat because LLMs really are pretty decent coders, and its hard to know what we are going to do with this knowledge.
My colleague, Astrini Sie, and I delivered a talk called Porting HoloLens Apps to Other Platforms at AWE 2023. Astrini is an AI researcher, so we threaded AI and AR together to see what AI can teach AR.
“Although Microsoft has substantially withdrawn from its Mixed Reality and Metaverse ambitions, this left behind a sizable catalog of community built enterprise apps and games, as well as a toolset, the MRTK, on which they were developed. In this talk, we will walk you through the steps required to port a HoloLens app built on one of the MRTK versions to other platforms such as the Magic Leap 2 and the Meta Quest Pro. We hope to demonstrate that, due to clever engineering, whole ecosystems can be moved from one platform to other newer platforms, where they can continue to evolve and thrive.”
In a 2010 piece for The New Yorker called Painkiller Deathstreak , the novelist Nicholson Baker reported on his efforts to enter the world of console video games with forays into triple-A titles such as Call of Duty: World at War, Halo 3: ODST, God of War III, Uncharted 2: Among Thieves, and Red Dead Redemption.
“[T]he games can be beautiful. The ‘maps’ or ‘levels’—that is, the three-dimensional physical spaces in which your character moves and acts—are sometimes wonders of explorable specificity. You’ll see an edge-shined, light-bloomed, magic-hour gilded glow on a row of half-wrecked buildings and you’ll want to stop for a few minutes just to take it in. But be careful—that’s when you can get shot by a sniper.”
In his journey through worlds rendered on what was considered high-end graphics a decade ago, Nicholson discovered both the frustrations of playing war games against 13 year olds (currently they would be old enough to be stationed in Afghanistan) as well as the peace to be found in virtual environments like Red Dead Redemption’s Western simulator.
“But after an exhausting day of shooting and skinning and looting and dying comes the real greatness of this game: you stand outside, off the trail, near Hanging Rock, utterly alone, in the cool, insect-chirping enormity of the scrublands, feeling remorse for your many crimes, with a gigantic predawn moon silvering the cacti and a bounty of several hundred dollars on your head. A map says there’s treasure to be found nearby, and that will happen in time, but the best treasure of all is early sunrise. Red Dead Redemption has some of the finest dawns and dusks in all of moving pictures.”
I was reminded of this essay yesterday when Youtube’s algorithms served up a video of Red Dead Redemption 2 (the sequel to the game Nicholson wrote about) being rendered in 8K on an NVidia 3090 graphics card with raytracing turned on.
The encroachment of simulations upon the real world, to the point that they not only look as good as the real world (real?) but in some aspects even better, has interestingly driven the development of the sorts of AI algorithms that serve these videos up to us on our computers. Simulations require mathematical calculations that cannot be done as accurately or as fast on standard CPUs. This is why hardcore gamers pay upwards of a thousand dollars for bleeding edge graphics cards that are specially designed to perform floating point calculations.
These types of calculations, interestingly, are also required for working with large data sets for machine learning. The algorithms that steer our online interests, after all, are just simulations themselves, designed to replicate aspects of the real world in order to make predictions about what sorts of videos (based on a predictive model of human behavior honed to our particular tastes) are most likely to increase our dwell time on Youtube.
Simulations, models and algorithms at this point are all interchangeable terms. The best computer chess programs may or may not understand how chess players think (this is a question for the philosophers). What cannot be denied is that they adequately simulate a master chess player that can beat all the other chess players in the world. Other programs model the stock market and tune them back into the past to see how accurate they are as simulations, then tune them into the future in order to find out what will happen tomorrow – at which point we call them algorithms. Like memory, presence and anticipation for us meatware beings, simulation, model and algorithm make up the false consciousness of AIs.
Simulacra and Simulation, Jean Baudrillard’s 1981 treatise on virtual reality, opens with an analysis of the George Luis Borges short story On Exactitude in Science, about imperial cartographers who strive after precision by creating ever larger and larger maps, until the maps eventually achieve a one-to-one scale, becoming exact even as they overtake their intended purpose.
“The territory no longer precedes the map, nor survives it. Henceforth, it is the map that precedes the territory – precession of simulacra – it is the map that engenders the territory and if we were to revive the fable today, it would be the territory whose shreds are slowly rotting across the map. It is the real, and not the map, whose vestiges subsist here and there, in the deserts which are no longer those of the Empire, but our own. The desert of the real itself.”
I was thinking of Baudrillard and Borges this morning when, by coincidence, Youtube served up a video of comparative map sizes in video games. Even as rendering versimilitude has been one way to gauge the increasing realism of video games, the size of game worlds has been another. A large world provides depth and variety – a simulation of the depth and happenstance we expect in reality – that increases the immersiveness of the game.
Space exploration games like No Man’s Sky and Elite Dangerous attempt to simulate all of known space as your playing ground, while Microsoft’s Flight Simulator uses data from Bing Maps to allow you to fly over the entire earth. In each case, the increased size is achieved by surrendering on detail. But this setback is temporary, and over time we will be able to match the extent of these simulations with detail, also, until the difference between the real and the model of the real is negligible.
One of the key difficulties with VR adoption (and to some extent the superiority of AR) is the falling anxiety everyone experiences as they move around in virtual reality. The suspicion that there are hidden objects in the world that the VR experience does not reveal to us prevents us from being fully immersed in the game – except in the case of the highly popular horror genre VR games in which inspiring anxiety is a mark of success. As the movements continue to both increase the detail of our simulations of the real world – to the point of simulating the living room sofa and the kitchen cabinet – and expand the coverage of our simulations across the world so there is no surveillable surface that can escape the increasing exactness of our model, we will eventually overcome VR anxiety. At that point, we will be able to walk around in our VR goggles without ever being afraid of tripping over objects, because there will be a one-to-one correspondence between what we see and what we feel. AR and VR will be indistinguishable at that exacting point, and we will at last be able to tread upon the sands of the desert of the real.
Thank you to everyone who attended my Deepfakes and Ethics talk yesterday at the AI and the Creative Industries Friday Series. Thanks also to Dr. Adam Spring for driving downtown with me and taking a walking tour of some of the parking lot there.
The talk was filmed and should go up in a few weeks.
At the beginning of October I was invited to deliver two sessions at Techorama Netherlands: one on Cognitive Services Custom Vision and one about the HoloLens and the Magic Leap One. This is one of the best organized conferences I’ve been to and the hosts and attendees were amazing. I can’t say enough good things about it.
The lineup was also great with Scott Guthrie, Laurent Buignon, Giorgio Sardo, Shawn Wildermuth, Pete Brown, Jeff Prosise, etc. It is what is known as a first tier tech conference. What was especially impressive is that this is also the first time Techorama Netherlands was convened.
I want to also thank my friend Dennis Vroegop for hosting me and showing me around on my first trip to the Netherlands. He and Jasper Brekelmans took a weekday off to give me the full Amsterdam experience. It was also great to have beers with Roland Smeenk, Alexander Meijers and Joost van Schaik. I’m not sure why there is so much mixed reality talent in the Netherlands but there you go.
Is ethical AI ethical? This is not meant to be a self-referential question as much as a question about semantics. Do we know what we mean when we talk about ethics? And, as a corollary, can we practice ethical AI if we aren’t sure what we mean when we talk about ethics. (Whether we speak correctly about artificial intelligence is a matter we can examine later.)
Is it possible that ethics is one of those concepts we all think we understand well, agree that it is important to understand in order to lead good lives, but don’t really have a clear grasp of.
If this is the case, how do we go about practicing ethical AI? What is the purpose of it? How do we judge whether our ethical standards regarding AI are sufficient or effective? What would effective AI ethics look like? And is the question of ethical AI one of those problems we need to develop AI in order to solve?
Toward an ethical AI, here are some passages I consider important:
“Everyone will readily agree that it is of the highest importance to know whether or not we are duped by morality.” — Emmanuel Levinas, Totality and Infinity
“What is happiness?
“To crush your enemies, to see them driven before you, and to hear the lamentations of their women!” – Conan the Barbarian
“Imagine that the natural sciences were to suffer the effects of a catastrophe. A series of environmental disasters are blamed by the general public on scientists. Widespread riots occur, laboratories are burnt down, physicists are lynched, books and instruments are destroyed. Finally a Know-Nothing political movement takes power and successfully abolishes science teaching in schools and universities, imprisoning and executing the remaining scientists. Later still there is a reaction against this destructive movement and enlightened people seek to revive science, although they have largely forgotten what it was….
“The hypothesis which I wish to advance is that in the actual world which we inhabit the language of morality is in the same state of grave disorder as the language of natural science in the imaginary world which I described. What we possess, if this view is true, are the fragments of a conceptual scheme, parts of which now lack those contexts from which their significance derived.” — Alasdair MacIntyre, After Virtue
“A great-souled person, because he holds few things in high honor, is not someone who takes small risks or is passionately devoted to taking risks, but he is someone who takes great risks, and when he does take a risk he is without regard for his life, on the ground that it is not on just any terms that life is worth living.” – Aristotle, Nicomachean Ethics
“In the name of God, the Merciful, the Compassionate.
“Someone asked the eminent shaykh Abu ‘Ali b. Sina (may God the Exalted have mercy on him) the meaning of the Sufi saying, He who knows the secret of destiny is an atheist. In reply he stated that this matter contains the utmost obscurity, and is one of those matters which may be set down only in enigmatic form and taught only in a hidden manner, on account of the corrupting effects its open declaration would have on the general public. The basic principle concerning it is found in a Tradition of the Prophet (God bless and safeguard him): Destiny is the secret of God; do not declare the secret of God. In another Tradition, when a man questioned the Prince of the Believers, ‘Ali (may God be pleased with him), he replied, Destiny is a deep sea; do not sail out on it. Being asked again he replied, It is a stony path; do not walk on it. Being asked once more he said, It is a hard ascent; do not undertake it.
“The shaykh said: Know that the secret of destiny is based upon certain premisses, such as 1) the world order, 2) the report that there is Reward and Punishment, and 3) the affirmation of the resurrection of souls.” — Avicenna, On the Secret of Destiny
“The greatest recent event—that “God is dead,” that the belief in the Christian God has ceased to be believable—is even now beginning to cast its first shadows over Europe. For the few, at least, whose eyes, whose suspicion in their eyes, is strong and sensitive enough for this spectacle, some sun seems to have set just now…” – F. Nietzsche, The Gay Science (1887)
A few years ago, CNNs were understood by only a handful of PhDs. Today, companies like Facebook, Google and Microsoft are snapping up AI majors from universities around the world and putting them toward efforts to consumerize AI for the masses. At the moment, tools like Microsoft’s Cognitive Services, Google Cloud Vision and WinML are placing this power in the hands of line-of-business software developers.
But with great power comes great responsibility. While being a developer even a few years ago really meant being a puzzle-solver who knew their way around a compiler (and occasionally did some documentation), today with our new-found powers it requires that we also be ethicists (who occasionally do documentation). We must think through the purpose of our software and the potential misuses of it the way, once upon a time, we anticipated ways to test our software. In a better, future world we would have ethics harnesses for our software, methodologies for ethics-driven-development, continuous automated ethics integration and so on.
Yet we don’t live in a perfect world and we rarely think about ethics in AI beyond the specter of a robot revolution. In truth, the Singularity and the Skynet takeover (or the Cylon takeover) are straw robots that distract us from real problems. They are raised, dismissed as Sci Fi fantasies, and we go on believing that AI is there to help us order pizzas and write faster Excel macros. Where’s the harm in that?
So lets start a conversation about AI and ethics; and beyond that, ML and ethics, Mixed Reality and ethics, software consulting and ethics. Because through a historical idiosyncrasy it has fallen primarily on frontline software developers to start this dialog and we should not shirk the responsibility. It is what we owe to future generations.
I propose to do this in two steps:
1. I will challenge other technical bloggers to address ethical issues in their field. This will provide a groundwork for talking about ethics in technology, which as a rule we do not normally do on our blogs. They, in turn, will tag five additional bloggers, and so on.
2. For one week, I will add “and ethicist” to my LinkedIn profile description and challenge each of the people I tag to do the same. I understand that not everyone will be able to do this but it will serve to draw attention to the fact that “thinking ethically” today is not to be separated from our identity as “coders”, “developers” or even “hackers”. Ethics going forward is inherent in what we do.
Here are the first seven names in this ethics challenge:
I want to thank Rick Barraza and Joe Darko, in particular, for forcing me to think through the question of AI and ethics at the recent MVP Summit in Redmond. These are great times to be a software developer and these are also dark times to be a software developer. But many of us believe we have a role in making the world a better place and this starts with conversation, collegiality and a certain amount of audacity.
This is a crazy long but nicely comprehensive article by the New York Times on the current state of AI: The Great AI Awakening.
While lately I’ve been buried in 3D interfaces, I’m always faintly aware of the way 1D interfaces (Cortana Skills, Speech as a service, etc.) is another fruit of our recent machine learning breakthroughs (or more accurately refocus) and of how the future success of holographic displays ultimately involves making it work with our 1D interfaces to create personal assistants. This article helps connect the dots between these, at first, apparently different technologies.
It also nicely complements Memo Atken’s Medium posts on Deep Learning and Art, which Microsoft resident genius Rick Barraza pointed me to a while back:
There’s also a nice throw away reference in the Times article about the relationship between VR and Machine Learning which is a little less obscure if you already know Baudrillard’s Simulacra and Simulation which in turn depends on Jorge Luis Borges’s very short story On Exactitude In Science.
If you really haven’t the time though, which I suspect may be the case, here are some quick excerpts starting with Google’s AI efforts:
Google’s decision to reorganize itself around A.I. was the first major manifestation of what has become an industrywide machine-learning delirium. Over the past four years, six companies in particular — Google, Facebook, Apple, Amazon, Microsoft and the Chinese firm Baidu — have touched off an arms race for A.I. talent, particularly within universities. Corporate promises of resources and freedom have thinned out top academic departments. It has become widely known in Silicon Valley that Mark Zuckerberg, chief executive of Facebook, personally oversees, with phone calls and video-chat blandishments, his company’s overtures to the most desirable graduate students. Starting salaries of seven figures are not unheard-of. Attendance at the field’s most important academic conference has nearly quadrupled. What is at stake is not just one more piecemeal innovation but control over what very well could represent an entirely new computational platform: pervasive, ambient artificial intelligence.
When he has an opportunity to make careful distinctions, Pichai differentiates between the current applications of A.I. and the ultimate goal of “artificial general intelligence.” Artificial general intelligence will not involve dutiful adherence to explicit instructions, but instead will demonstrate a facility with the implicit, the interpretive. It will be a general tool, designed for general purposes in a general context. Pichai believes his company’s future depends on something like this. Imagine if you could tell Google Maps, “I’d like to go to the airport, but I need to stop off on the way to buy a present for my nephew.” A more generally intelligent version of that service — a ubiquitous assistant, of the sort that Scarlett Johansson memorably disembodied three years ago in the Spike Jonze film “Her”— would know all sorts of things that, say, a close friend or an earnest intern might know: your nephew’s age, and how much you ordinarily like to spend on gifts for children, and where to find an open store. But a truly intelligent Maps could also conceivably know all sorts of things a close friend wouldn’t, like what has only recently come into fashion among preschoolers in your nephew’s school — or more important, what its users actually want. If an intelligent machine were able to discern some intricate if murky regularity in data about what we have done in the past, it might be able to extrapolate about our subsequent desires, even if we don’t entirely know them ourselves.
The new wave of A.I.-enhanced assistants — Apple’s Siri, Facebook’s M, Amazon’s Echo — are all creatures of machine learning, built with similar intentions. The corporate dreams for machine learning, however, aren’t exhausted by the goal of consumer clairvoyance. A medical-imaging subsidiary of Samsung announced this year that its new ultrasound devices could detect breast cancer. Management consultants are falling all over themselves to prep executives for the widening industrial applications of computers that program themselves. DeepMind, a 2014 Google acquisition, defeated the reigning human grandmaster of the ancient board game Go, despite predictions that such an achievement would take another 10 years.
Following on the tail of the project I have been working on for the past month, a chatterbox (also called a chatbot) with speech recognition and text-to-speech functionality, I came across the following excerpted article in The Economist, available here if you happen to be a subscriber, and here if you are not:
Chatbots have already been used by some companies to provide customer support online via typed conversations. Their understanding of natural language is somewhat limited, but they can answer basic queries. Mr Carpenter wants to combine the flexibility of chatbots with the voice-driven “interactive voice-response” systems used in many call centres to create a chatbot that can hold spoken conversations with callers, at least within a limited field of expertise such as car insurance.
This is an ambitious goal, but Mr Carpenter has the right credentials: he is the winner of the two most recent Loebner prizes, awarded in an annual competition in which human judges try to distinguish between other humans and chatbots in a series of typed conversations. His chatbot, called Jabberwacky, has been trained by analysing over 10m typed conversations held online with visitors to its website (see jabberwacky.com). But for a chatbot to pass itself off as a human agent, more than ten times this number of conversations will be needed, says Mr Carpenter. And where better to get a large volume of conversations to analyse than from a call centre?
Mr Carpenter is now working with a large Japanese call-centre company to develop a chatbot operator. Initially he is using transcripts of conversations to train his software, but once it is able to handle queries reliably, he plans to add speech-recognition and speech-synthesis systems to handle the input and output. Since call-centre conversations tend to be about very specific subjects, this is a far less daunting task than creating a system able to hold arbitrary conversations.
Jabberwacky is a slightly different beast than the AIML infrastructure I used in my project. Jabberwacky is a heuristics based technology, whereas AIML is a design-based one that requires somebody to actually anticipate user interactions and try to script them.
All the same, it is a pleasant experience to find that one is serendipidously au courant, when one’s intent was to be merely affably retro.
