From www.astrology-and-science.com       12m 59kb       Home       Fast-Find Index

What tests are easy?
A guide for students doing projects

Geoffrey Dean

Abstract -- It is a myth to say astrology cannot be tested. How could astrologers know it works if they could not test it? Some tests are difficult. Other tests are easy and can be made by anyone. The key is to ask the right questions. The most overworked areas are identified (don't bother with these since one more test will make no difference) along with three easy tests of sun sign beliefs, two easy tests of sun sign columns, one easy test of astrologers, and one easy test of Barnum effects.

Can astrology be tested?
As soon as astrologers hear the word "test" they tend to raise objections such as the following:

Astrology (soul stuff) cannot be tested by science (materialism).
Psychology is too confused for its tests to be meaningful.
Proper testing is not possible in the absence of funding.
Testers are closed-minded and cannot be taken seriously.

Ignore such objections. First, if they were true then astrologers could never know anything about astrology. How could they know it works if they could not test it? Second, the objections are irrelevant to the questions that matter, such as are Leos more generous than non-Leos, or do astrologers agree on what a chart means. Third, yes, some tests are difficult but others are easy and can be made by anyone. See Research results under Doing Scientific Research for abstracts of 91 studies that illustrate the diversity of tests that have been made.

Helpful questions
Well-considered questions, even if not part of your eventual project, may help in clarifying your approach to astrology. For example, to evaluate any idea such as "astrology works", try asking the following questions. The aim is not to win but to learn:

(1) Why do you believe in this idea? This puts the burden of proof on the claimant where it belongs.

(2) What evidence would you accept as proving your idea wrong? This is a potent question because it opposes the tendency to consider only confirming cases.

(3) Are there other explanations that would produce the same outcome? This too is a potent question because it looks at how informed the claimant is.

Avoid open ended questions. Ask not "how does astrology work?" but "what is meant by astrology?" so you know what the issues are (astrologers seldom agree on this or anything else, so the answer will depend on who you talk to). Not "is astrology true?" but "to what extent is it true?" or "does it need to be true?" The key question is "does astrology deliver benefits beyond those due to non-astrological factors?", but this cannot be answered until the other questions have been answered. Do not expect to find this question answered in astrology books.

Projects about sun signs

Signs are the most researched topic in astrology with well over one hnndred empirical studies. Most studies are simply counts of people born under various signs, but such counts are too contaminated by ordinary influences (astronomy, sampling, demography, age incidence) to mean anything. We can try correcting for such influences but in practice the uncertainty is too great. So forget counting sun signs. If you need convincing, read this website's critiques of Sachs's Astrology File under Sun Signs.

The remaining studies, if adequately controlled against non-astrological influences, have invariably been negative. Signs are not only the most researched topic in astrology but are also the most disconfirmed. Signs are simply not valid, not even slightly. So forget validation, including validation of sun sign compatibility. Turn instead to looking at people's reactions towards an extremely popular belief now known (but not widely known) to be untrue.

Sun signs seem valid because they cannot fail to fit (everyone, not just Leos, will admit to being generous). They are also a good topic of conversation and are nice to have around. People tend to like them, but exactly why is not well understood despite many speculations. So one easy and useful project might be to ask people why they like or dislike sun signs, and to find reasons for any differences. For example men may view sun signs differently from women.

Reactions to disconfirmation
Sun signs let you study people's reactions to disconfirmation. Assemble a roomful of people, sit them in a circle according to sun sign (this adds intrigue and keeps the group focussed), and record each person's level of belief in sun signs (put a scale on display for them to choose from, such as strong, moderately strong, and so on). Ask for opinions why anyone should believe or disbelieve in sun signs, and get a debate going. Observe the arguments and record how each person reacts to views opposing their own. After a suitable period stop the debate and again record each person's level of belief. Ask why they changed or why they didn't change. Next, raise the question that nobody will have raised, namely what evidence would each person accept as disproving sun signs. Record their answers. Then summarise the evidence from this website about sun signs, and restart the debate. After a suitable period stop the debate and again record each person's level of belief. Rationally, everyone should end up as disbelievers, but most likely the believers will carry on believing. Ask them why. You now have the makings of an absorbing essay for your social science teacher. You may also be facing slings and arrows as in True Confessions under Sun Signs.

Sun sign junkies
Belief in sun signs is known to slightly affect a person's self-image and in some cases even their choice of job. The daily horoscope may well have an effect on daily mood. But measuring these things requires large samples, careful techniques, and lots of time, which puts them well beyond the resources of most projects. But there is a much easier way of getting useful insight into these things, namely by talking to people who take sun sign columns very seriously and cannot exist without them. These sun sign junkies comprise about 1% of the general population and may be hard to find, but your newsagent may know a few. This is an unexplored area that deserves attention.

Mix and match
Finally two easy projects that have been done before but are still informative and fun. (1) See if people can pick their own newspaper horoscope after you have mounted each one separately and removed all cues (they may need retyping). If picking is difficult, have people rank their top three or four choices. (2) Count agreements and disagreements between horoscopes from different newspapers or women's magazines. If you want more mileage, do a repeat but mixing the days so the horoscopes being compared are not for the same day. Then get others to repeat the exercise to see if their counts agree with yours. In each case it is a good idea to have a trial run to make sure there are no snags.

Projects about serious astrology

The bad news is that many tests are difficult, time consuming, and have already been done. For example testing whether an astrologer can identify people from their charts has already been done in 54 studies, some of which took several years to complete. Overall 742 astrologers and more than 1400 charts were tested but the results showed no support for astrology. (The support shown by some early tests was later found to be an artifact of sampling.) Another 20 studies involving nearly 500 subjects tested whether people can pick their own chart reading but again the results showed no support for astrology. See Meta-analyses under Doing SAcientific Research for overviews of all the studies in areas like these.

In general the better the study the more negative the outcome. The problem is that the sheer weight of studies will not be overturned by adding one more study. The good news is that other studies are much easier, just as useful, and more fun. And anyone can do them.

Tests of agreement
For example if you change the topic from seeing if astrologers are accurate to seeing if astrologers agree, the test becomes almost ridiculously easy. Just sit astrologers together in the same room, put a succession of charts on a screen, and ask questions that are a mixture of easy (is this person outgoing?) and difficult (does this person own a cat?). Any charts will do, even invented ones, because it is irrelevant what the person is actually like. What matters is whether the astrologers agree on what the chart is saying, and whether their agreement varies with difficulty.

To be feasible the above test requires access to a local astrology group. But it is worth the bother. Notice how all the usual excuses for error (birth time not accurate enough, person does not know themself, and so on) no longer apply, since accuracy is not a concern. The point is, if astrologers cannot agree on what a chart is saying, then what price astrology? So far 28 such tests have been made involving a total of 506 astrologers and 762 charts, but their agreement was scarcely better than no agreement. Do not expect to find this result mentioned in astrology books.

One size fits all
A popular but interesting test is pretending to obtain chart readings for a roomful of people, who rate each statement in the reading for accuracy and then indicate to what extent the reading has increased their belief in astrology. Each reading is preceded by the person's name and birth details, so it looks genuine, but in fact each person gets the same fake reading. Alternatively a different fake reading can be provided for every 5-10 people, which allows you to test more statements without lengthening the individual readings. The aim is to show people how easy it is to read specifics into generalities (this is called the Barnum effect), how this affects their belief level, and to compile a reading that will fit as many people as possible.

Here the sense seems to come from the reading but in fact it comes from our ability to make sense out of vague data, as when we see faces in clouds. The most readily-accepted Barnum statements are favourable (you are forceful and well-liked by others), or vague (you enjoy a certain amount of change and variety), or two-headed (you are generally cheerful but get depressed at times). Not unexpectedly, the result of accepting Barnum statements is an increase in belief. The Barnum effect has attracted around 70 studies but it still makes a telling point.

The fallacy of personal validation
Interestingly, the most famous Barnum study, made in 1949, and many studies since then, used 13 statements taken mostly from an astrology book. The statements were typical of what might be heard during a chart reading. The studies found that when the statements are presented as a personal reading, as would of course apply during a chart reading, most people rate their accuracy as good or excellent. In case you are interested, the statements are as follows:

To see how it works, look at volume 10 pages 25-27 of Noel Tyl's Principles and Practice of Astrology, Llewellyn 1975, where he gives a reading to show how astrological counselling should be conducted. Tyl, a famous American astrologer and a psychology graduate, is renowned among astrologers for the quality of his work, so we can take this as being representative of the best practice. Tyl is talking to Eric, a new client aged 24. I have added comments in [ ].

"Basically, we should begin with a general statement to differentiate you from ... all other people. The horoscope shows that you are gracious, friendly, that you express yourself softly [all this was already obvious, now comes the favourable Barnum], and that you're very sensitive. [Now the vague Barnum] But, Eric, there are tensions here ... linked to self-esteem, how you feel about yourself. [Now the double-headed Barnum] The horoscope suggests that, on the one hand, you're pretty particular about whom you relate to. [Now repeat vague Barnum] This might be a defense because you're not too pleased with yourself. [Now back to double-headed Barnum] And, then on the other hand, you're friendly to the extreme in order to be accepted. Do you think that is accurate?" To which Eric replied "Yeah (smiling). That's it, right on the button. [But how could it be otherwise?] Gee, I know a little about Astrology; how do you see all that?"

Sources of information

If your project requires a review of the literature, remember that books which examine the truth about astrology can be hard to find. For every book there are dozens of articles that can be even harder to find. Be aware that works earlier than 1990 tend to be dated. A comprehensive guide to finding information, plus an annotated book list, is on this website in Finding info under Adroit Utilities. Unfortunately books that suggest tests are almost nonexistent. Two recent titles are:

Blackmore S & Hart-Davis A (1995). Test Your Psychic Powers: Find Out the Truth for Yourself. Thorsons, London. Sets out simple experiments for testing sun sign columns and nine other paranormal areas (telepathy, crystals, dreams, dowsing, pendulums, premonitions, psychokinesis, ouija boards, and palmisty) that you might find more appealing! Blackmore was then a leading skeptic and investigator of the paranormal.

Wilson A (1998). What's the big idea? The paranormal. Hodder Headline, London, with illustrations by N Dewar. A humorous but critical look at the paranormal for young people, with occasional simple tests including one of sun sign columns. "If you want to know the truth about the paranormal, don't read the stars."

From www.astrology-and-science.com       12m 59kb       Home       Fast-Find Index