The man on the show Brainman did not memorize PI.
His brain had a way of calculating the number. He could seemingly do it accurately to an unlimited number of digits. One of the most important messages on the show was that this man's ability with numbers came not from memorization, but in the organic system that somehow developed in his brain that allowed him to calculate without using normal methods.
Jean Houston wrote about this one kid who could compute answers to mathematical formulae by sort of moving around and humming.
What's so interesting about this stuff is that it isn't straightforward memorization and recall, but some sort of organic process in which the brains of these individuals create or recreate results.
Richard Feynman said that the way math is taught in schools does not teach mathematics, but a brainless, mechanical way in which to produce predictable results without having to think.
Brainman, with his synasthesias and rich way of representing numbers and their relations, is absolutely amazing to me. Multi-sensory representation and processing seems to be one of the keys to using really amazing brainpower. The guy in the Mind of Mnemonist had synasethestic experiences, too.