What it is. Language models are usually described from the outside: you type, they answer. Neuron Explorer looks at the inside. It takes Qwen3-0.6B, a small open model, and shows every one of its 86,016 neurons as a dot on a map. Click a dot and you get that neuron’s own page: the sentences from two million words of web text where it fired hardest, highlighted word by word, the sentences that pushed it the other way, the words it nudges the model towards, and the other neurons it feeds. For about three thousand of them you can also read what Claude made of the evidence, move a slider and watch the model’s writing change as the neuron is pushed up or down. Nothing on the site is an illustration. Every highlight, number and sentence comes from running the real model.
Why I built it
A language model is a file of numbers. Qwen3-0.6B is about 600 million of them, and nobody can read that, not even the people who trained it. What you can do is open it up and watch it work: feed it text, record what every part does, and push on one part to see what changes. That is what I wanted to see with my own eyes instead of reading about it. The interpretability work coming out of the big labs is some of the most interesting research around, and most of it happens on models you cannot touch. A small open model on a home PC is enough to do a real version of the same thing: capture activations, map the neurons, steer them, and find out how much of the “the model has a neuron for X” story survives contact with the data.
There is also a work reflex in it. As an AML analyst I am used to not taking a system’s word for what it does, and going to the records instead. A model’s answers are what it says about itself. Its activations are the records.
The idea came from lyramakesmusic’s neuron-explorer. The code is written from scratch. Anthropic’s models are closed, so the subject had to be an open-weight model; Claude gets a different job: it wrote the plain-language explanation of each featured neuron, and picked the rounds for the game.
How you use it
The map shows the model as 28 translucent sheets, one per layer, each a grid of 3,072 dots. Neurons that write to the model in similar ways sit next to each other, so patterns show up as neighbourhoods. You orbit the stack, zoom in, scrub through layers, or search for a neuron by number. Click a dot and amber and blue lines run through the stack: amber for the neurons it excites, blue for the ones it suppresses. Open it and you are on its page. A Simple/Technical switch rewrites the jargon and hides the numbers if you want them gone, and every term has a hover definition.
Run a sentence through it. On the map a box on the left offers eight prepared sentences: one about gems, one from a clinic, a Moon landing with a year and a distance, two lines of Python, a Dutch sentence, one that stops halfway, a grammar pattern and an everyday one. Pick one and a bolt climbs the stack layer by layer, striking the neurons that woke hardest. A card on the right shows the sentence as tokens, what the model expects to come next and the named neurons among the loud ones, and clicking a token narrows it all to that word. The jeweller sentence wakes the gemstone neuron from the tour at about 60 percent of the hardest it ever fired in the sample, the loudest in its layer. You cannot type your own, and the box says so: nothing here runs a model.
If the map is too much at once, the tour walks you from tokens to steering in seven scenes, the learn page explains the four ideas you need and holds the glossary, and the game lets you test whether you can spot a neuron from its steered text. The landing page also shows a neuron of the day.
What the model looks like from the inside
- A neuron is simpler than the word suggests. It is one number per token: a dial that reads the running summary the model carries between layers, and adds its own direction back in. The page for a neuron is nothing more than that number, tracked over two million tokens, sorted.
- Some neurons are clean and most are not. Layer 21, neuron 2688 fires on the word right before a number. Layer 5, neuron 74 fires on legal citations. Those are the easy ones, and the site’s score is biased towards them because it rewards neurons that peak on the same token every time. Open a random dot and the top sentences often share no visible theme at all. The tidy “one neuron, one concept” picture is a minority of the model. Claude’s explanations say the same thing in numbers: of 2,996 featured neurons it was confident about 325, and the tour deliberately shows a messy one.
- Neurons fire both ways. This model’s neurons can go strongly negative as well as positive, and the negative side often means something different, not just “less of the same”. The site keeps both lists for that reason.
- The layers have different jobs. Early layers react to pieces of words and formatting. In the middle, the logit lens (what a neuron pushes the output towards, read straight off its weights) is mostly noise, because the text is still being transformed. Towards the end it becomes readable: concrete words the neuron is voting for. The running summary also grows about tenfold between layer 13 and the last layer, so “the same push” means something different at each depth.
- Steering has a narrow useful band. A small push shifts the topic and the text stays fluent. A large one breaks the model into loops. And a random direction of the same size also does something, which is why every slider has a “compare with a random direction” toggle: without the control you cannot tell a neuron’s effect from the effect of shoving the model at all.
- The wiring is partly readable from the weights alone. Which neurons excite or suppress which later neurons can be computed without running any text, ignoring attention. It is a partial diagram, but it is enough to see neighbourhoods: neurons that write in similar directions sit together on the map, and the lines through the stack follow real structure.
- What a neuron “does” depends on what you showed it. My first corpus used the model’s chat marker as a separator between documents, and a handful of neurons seemed to be about nothing but that marker. Change the data and their story changes. Every explanation on the site is relative to two million tokens of educational web text, and says so.
- Tokens are not words, and the model is not English. The vocabulary is multilingual, and a neuron that promotes “eighty” and “seventy” promotes the Chinese counting words next to them. Numbers across languages live in one place.
- The first layers do not care what you say. In layers 0 to 2 the loudest neurons are the same for every sentence: detectors for position and common tokens that saturate whatever goes in. That is why the trace card lists the named neurons separately from the loudest ones. Early layers reacting to surface features is itself part of the lesson.
- Small is already too big. A 0.6B model runs on a CPU, and it is still far beyond reading neuron by neuron. The featured set is 2,996 of 86,016, about three and a half percent. Understanding a model is a tooling problem before it is a reading problem, which is the whole reason the site exists.
Three neurons to try
Layer 21, neuron 2468 is the tour’s example: it fires on gemstones and jewellery words, and a small push turns the model’s musings towards carats and gold rings. Layer 5, neuron 74 fires on legal citations: “42 U.S.C. § 1395”, amendments, statutes. Push it a little and the model’s news prompt turns into government and policy talk. Push it to the maximum and the text collapses into repetition, which is its own lesson: steering is a blunt tool, and the slider shows you where it breaks. Layer 21, neuron 2688 fires on the word right before a number (“buried under”, “for”, “every”). Pushed negative the model writes strings of digits; pushed positive it avoids numbers and gets wordy.
Where it stands
Done, for the first model. The whole of Qwen3-0.6B has been processed: two million tokens of educational web text run through it on the CPU of my gaming PC (the AMD card has no PyTorch support on Windows), one and a half hours for the pass and another two and a half to write 86,016 JSON files. The featured neurons were steered overnight: four prompts, seven strengths, 48 tokens each, with random-direction controls so you can check that a neuron’s effect is not just noise. Claude then explained all 2,996 of them in under an hour, and rated each one for the game, which left 341 rounds. The tour, the glossary and the game came next, with a neuron of the day on the landing page and a flag link under every explanation. Last came Trace and the short opening animation of the layers growing. The data behind the site is about 650 MB compressed, served as static files from a small VPS; no model runs there.
What is left is a second model, which means running the pipeline again rather than touching the site. Typing your own sentence comes only if people use the prepared ones: it would need the model in the visitor’s browser or a small server. Any open model can be run through it.


