You are spot-on about the Amusing Randi. One of his favorite tricks is to keep moving the bar, but he will resort to almost anything to "prove" someone or something is a hoax. We all know that the observer does influence the experiment--but this particular "observer" does it deliberately. I have known people who tried for the Randi money, and it was more brutal than most would imagine.
One field that has produced some real evidence is astrology. Michel Gauquelin did what for that time (when computers took up whole rooms to do what a pc will do today) was a tremendous research feat: to see if astrology had any validity. Actually, his goal was to disprove it, because he felt that it was pure quackery. However, the data proved him wrong. There was definite evidence of its validity--evidence which is ignored to this day by both the scientific community and many astrologers.
Another good researcher all but ruined his career: as an astronomer, Percy Seymour was reluctant to give any credit to something like astrology, but as an open-minded scientist (something which is distinctly rare in this society), he could not ignore the evidence, and so wrote "Astrology: The Evidence of Science."
Any time a scientist begins to change the landscape of a deeply entrenched belief, even with a lot of evidence, he or she will face incredible resistance from those who are invested in the current belief sets. We're seeing lots of breakthrough discoveries in our times, many of which are agreeing almost entirely with what spiritual writings and teachings have said for many thousands of years. However, we're also seeing the same ol' same ol' from the hard-to-move "scientific" community--but mind you, the scientists making these breakthroughs did arise from that same community, so I think there is hope.
