If there are a million people, then there will be someone who flips heads 20 times in a row. Even when there is no difference in how many chances people are given and no difference in how lucky they are, and the playing field is perfectly equal, some people will still end up looking lucky. […]
Stop Leaving Your Success to Luck: A Researcher’s Guide to Hacking Randomness Read Post »
If you are unsure what to do in the age of AI, it is good to look to the past.When you think about what kind of life you have lived up to now, a path opens up. AI levels people’s abilities.With coding agents, the gap between the very best programmer and an ordinary programmer becomes
Past Becomes Important in the Era of AI Read Post »
AI performance is improving day by day, and its evolution may appear unstoppable. However, in reality, that is not what will happen. In this article, I explain why. The key terms in this article are epistemic uncertainty and aleatoric uncertainty. Epistemic uncertainty is uncertainty that can be resolved through observation, whereas aleatoric uncertainty arises from
Why AI Progress Will Stall Read Post »
Cognitive semantics is a linguistic approach that understands the meaning of words in relation to human cognition and ways of perceiving things. In this article, I would like to talk about the “cognition” of LLMs from the perspective of cognitive semantics. In the background of cognitive semantics are fields such as Gestalt psychology and cognitive
How LLMs Feel Language Read Post »
I think a lot of people feel a real sense of crisis about AI taking their jobs. I feel it too. In fact, I am in the almost absurd, joke-like situation where I was supposed to be working on building AI, and now even that very job looks as though it may itself be taken
How to Survive the AI Revolution Read Post »
We can flatten the parameters of a deep neural network into a single vector. For a large model, this vector can easily have tens of billions of dimensions. At first glance it may look like a meaningless list of numbers, but it has become clear that this vector has deep meaning as a vector. For
Model Merge Explained: Build Models without Training Read Post »
Over the past few years, LLMs have developed rapidly, significantly expanding our cognitive capabilities. Many people probably now turn to AI first when they encounter a problem. It may be that LLMs already know more than my brain does, and perhaps possess higher reasoning abilities than my brain. However, in this article, I will discuss
Why Your Brain Matters in the Era of AI: You Can’t Outsource Intuition Read Post »
In this article, I will argue that transformers are RNNs. I will not merely point out formal containment; I will also explain how similar transformers and RNNs are, and what practical implications follow from that viewpoint. Transformers are RNNs First, I will show that transformers are RNNs. To put it simply, a transformer is an
Transformers are RNNs: A Kernel Perspective Read Post »
Sorting is a classic task in computer science, but it has recently intersected with state-of-the-art LLMs and sparked a new research trend. Sorting can be executed as long as you define a comparison function. Traditional comparison functions assumed measurable numeric quantities such as height, price, or distance. However, if we call an LLM inside the
“Generating…”We are all familiar with the cursor blinking, slowly revealing text word by word. Whether it’s ChatGPT or a local LLM, the generation speed of autoregressive models based on Transformers is fundamentally bound by their sequential nature. It computes, emits one token, computes again, emits the next token, and repeats. This strict dependence chain introduces
Speculative Decoding Explained Read Post »
