From www.astrology-and-science.com       12m 2g 63kb       Home       Fast-Find Index

Artifacts in data
Often wrongly seen as evidence for astrology

Geoffrey Dean

Abstract -- The controversy over astrology is largely about artifacts, or whether the claimed results could have ordinary explanations. Artifacts are fake effects due to hiccups in data (this article) or in reasoning (see the article Artifacts in reasoning). They are never mentioned in astrology books yet they occur throughout astrology, leading to wrong conclusions that astrology works. You will be led seriously astray if you learn about astrology without first learning about artifacts. This article looks at artifacts in the data, as when sun sign counts are not corrected for the earth's elliptical orbit (the sun spends two days longer in Cancer than in Capricorn). Some artifacts in data became famous in their day as the best claimed evidence for astrology. They include Nelson's correlation between planets and radio quality, Brown's lunar effects on oysters, Bradley's Jupiter Pluvius rainfall effect, Jung's astrological experiment with married couples, Vernon Clark's matching tests, John Addey's harmonics, the Mayo-Eysenck zodiac zig-zag, and Gauquelin's planetary effects. All have taken considerable time (decades) for their artifacts to be uncovered. So we should be cautious about any new astrological "proof". Of course the presence of artifacts does not deny the existence of genuine effects. But unless research can confirm such effects when artifacts are controlled (which so far is not the case), we have good reason to suspend belief. Artifacts of data and reasoning have always raged out of control in astrology. 27 references.

When astrologers claim that astrology works, they imply that all non-astrological influences leading to the same result have been ruled out. Such influences are called artifacts, not to be confused with artefacts as in man-made objects of archaeological interest. Artifacts are fake effects due to hiccups in data or reasoning that mimic astrology and persuade us (wrongly) that astrology works. Artifacts in data are discussed in the present article. Artifacts in reasoning are discussed in the companion article Hidden Persuaders on this website under Problem Areas.

An artifact is something other than astrology that would result in apparent astrological effects. Astrological effects have tiny effect sizes (see Index), and it does not need much of an artifact to produce a tiny effect size. But you will not find artifacts mentioned in astrology books. In fact astrologers have shown no interest in preventing artifacts, and every interest in using procedures known to encourage them. You will be led seriously astray if you learn about astrology without first learning about artifacts. Don't believe that astrology works unless you can be sure that artifacts have been ruled out. This and other articles on this website will get you started.

Some artifacts in data should be obvious, as when sun sign counts in the astrologers' tropical zodiac are not corrected for the earth's elliptical orbit (the sun spends two days longer in tropical Cancer than in tropical Capricorn), or when astronomical constraints are ignored (thus Venus cannot be square the Sun), or when the claimed frequency of astrological features is to be expected by chance anyway (for examples see the review of Astrology Really Works! on this website under Book Reviews). Other artifacts in data are more subtle and resistant to detection, as in the following examples, some of which became famous in their day as the best claimed evidence for astrology.

Nelson's correlation with radio quality
A claimed correlation between planetary positions and radio propagation quality (Nelson 1951) was due to the close but unequal spacing of planet days, which meant that small differences from radio days were more likely than large differences (Ianna and Margolin 1981, Meeus 1982). For 5507 of Nelson's published predictions of radio quality, the correlation between predicted quality and observed quality was 0.01, or effectively zero (Dean 1983ab), where 1 = perfect correlation and 0 = no correlation. A correlation of 0.01 is equivalent to a hit rate of 50.5% when 50% is expected by chance. These findings led to the gradual disappearance from astrology books of Nelson's work being cited as hard evidence for astrology, for example the 1970 edition of West and Toonder's The Case for Astrology said it "unequivocally supports" astrology, whereas the later 1991 edition makes no mention of it. But the disappearance was generally slow, and nearly twenty years later it was still not complete, eg Slevin (1999).

Brown's lunar effects on oysters
Alleged lunar effects on oysters when the oysters were moved a thousand miles inland from their original sea location (Brown 1954) disappeared when the data were were given a proper time-series analysis, and were evidently due to inadequate data, failure to apply controls, and wishful thinking (Enright 1965). The oysters showed no consistent rhythms, so the issue of lunar-induced changes is irrelevant.

Bradley's Jupiter Pluvius rainfall effect
A claimed tendency for heavy rainfalls to peak when Jupiter was in a multiple-of-90-degrees aspect to the local meridian when the Moon entered sidereal Capricorn, the Jupiter Pluvius effect (Bradley 1957), would, if true, rank among the most remarkable in astrology. This is because the connection does not involve Jupiter's position at the time of the rainfall but its position at successive moments averaging 27.32/4 days apart (27.32 days is the Moon's sidereal period), which would seem to deny any physical influence and to support only an astrological one. But the effect was later found to be an artifact. In 27.32 days Jupiter moves about 2 degrees (with considerable variation due to retrograde motion) and the earth rotates 27 full circles plus 89 degrees (with a variation due to small fluctuations in individual lunar periods). So the difference overlaps 90 degrees, which then tends to produce the observed multiple-of-90-degrees effect (James 1981). The tendency need only be small to match the small difference between peak and average rainfalls (peak rainfalls average only 5% more than the mean).

Jung's astrological experiment
Apparent support for astrology arose when the birth charts of married couples showed a small tendency to favour aspects between them indicative of marriage (Jung 1960). But the charts had come from the files of an astrologer, whose advice to the couples had nudged the sample into detectable conformity. Also Jung chose the aspects after examining the results, which inflated his statistics (and even the inflated effect size was only 0.09, equivalent to a hit rate of 54.5% when 50% is expected by chance). The effect did not replicate when artifact-free data were used (Dean 1996).

Vernon Clark's matching tests
Claimed success in matching charts to case histories (Vernon Clark 1961) was consistent with the use of small samples, typically 10 birth charts, whose huge sampling variations were mistaken for genuine effects (Eysenck and Nias 1982:86-87). That is, the apparent success was due to a small sample that (thanks to huge sampling variations) happened to contain a disproportionate number of hits, which then gave the appearance of success due to astrology when in reality the success was due to sampling variations. This was confirmed by (1) meta-analysis of the 54 existing matching tests, which among other things compares the expected variations due to sample size with the observed variations in the results. If the former exceeds the latter, as is the case here, then the results including the best and worst are entirely explained by sampling variations, leaving nothing to be explained by astrology. And by (2) sub-dividing a sample of 240 birth charts that showed almost no effect into 24 sets of 10 charts each, which then showed the predicted artifact. See figure below.

Observed effect sizes in 54 matching tests where a total of 742 astrologers matched a total of 1407 birth charts with their owners. Clark's three results are shown as grey dots. By meta-analysis the mean effect size weighted by sample size is 0.035 sd 0.117, equivalent to a hit rate of 51.7% sd 5.8%, no different (p=0.77) from the 50% expected for guessing. The expected sd due to sampling variations alone is 0.199, considerably higher than the observed sd of 0.117, which leaves nothing in the results (including Clark's results) for astrology to explain. So the scatter in results is entirely due to using too few birth charts (typically 10) per study. When two studies using 120 charts each (black dots) averaging 50.3% hits are sub-divided into 24 studies of 10 charts each, the same scatter emerges, showing how scatter is an unavoidable artifact of small sample sizes. If the samples are further sub-divided, the scatter increases, exactly as predicted.

Thus the scatter of results in the above figure can be entirely explained by sampling variations, a point further considered in the next section, which means that the results (including Clark's results) provide no support for astrology. Further analyses suggest that the slight preponderance of positive results (ie hits over 50%) is due to publication bias, the reluctance of editors to publish negative results. Unfortunately most astrologers seem unaware of the many tests that now exist, and how the current verdict is no longer biassed by the large sampling artifacts of the earlier tests. For example Geoffrey Cornelius (1994), after citing 9 early tests, says "I have not heard of other experiments of this type" (p.84), while Ken McRitchie (2004) says "few tests have been conducted" (p.28).

Illusory results due to sampling variations
The sample size needed to reliably detect effect size r, expressed as a correlation, is roughly 10/r². In technical terms to reliably detect r means detecting r in 4 out of 5 tests at a two-sided significance level of p = 0.05. So if we want to reliably detect an effect size of say 0.10, which is already more than the average of 0.04 typically observed in both Gauquelin and Vernon Clark studies, we will need a sample size of around 10/0.10² = 1000, or very considerably more than what astrologers traditionally use. The fatal consequences of using small samples are shown in the figure below:

Observed effect size in 1000 sub-samples taken at random from a very large sample in which the effect size is known to be 0.000. The sample sizes range from 3 to 250, mean 80, and the mean observed effect size is -0.004 sd 0.198, equivalent to a mean hit rate of 49.8% sd 9.9%, which as expected is not significantly different from the expected 0.000. In small samples such as those commonly used by astrologers, the observed effect size (thanks to sampling variations) can be wildly different from what is actually there, leading to wildly wrong conclusions.

As the above figure shows, just by using their usual small sample of ten birth charts, or even a hundred birth charts, astrologers are virtually guaranteed of finding something that is interesting, exciting, full of promise, and totally spurious. As next.

John Addey's harmonics
A revolution in astrology was promised by harmonics (Addey 1976). It was claimed that astrological influences were not boxlike, as in signs and houses, but varied continuously in harmonic waves around the circle with an influence related to the number symbolism of the harmonic. Thus the 3rd harmonic (repeating every 120 degrees) had a meaning different from the 4th harmonic (repeating every 90 degrees). Unlike most ideas in astrology, harmonics was based on decades of painstaking research by Addey and his many helpers, so it seemed sounder than most. However, later computer simulations showed that the results were due to small sample sizes, sampling errors (the sampling requirements in harmonics are unusually stringent), the use of moving averages (which can generate spurious waves), and incorrect statistics (Dean 1997).

Mayo-Eysenck zodiac zig-zag
An apparent correlation between sun signs and extraversion (Mayo, White and Eysenck 1978; Smithers and Cooper 1978) disappeared when the subjects had no prior knowledge of astrology, which showed that prior knowledge can nudge a person's self-image in the direction of astrology (Eysenck and Nias 1982:50-60, Dean 1983c, van Rooij 1999). Ask Sagittarians (who are supposedly sociable and outgoing) whether they like going to parties, and their answer might be tipped by astrology in favour of yes rather than no. The bias may be unconscious and very slight but in large samples it can become surprisingly consistent. Of course it has nothing to do with sun signs as normally conceived.

Gauquelin's planetary effects
Finally Gauquelin's planetary effects, which almost every astrology book cites as the best-known support for astrology while simultaneously ignoring Gauquelin's negative findings for signs, aspects, and transits (Gauquelin 1960, 1983), see elsewhere on this website under Gauquelin. The effects were consistent with parents faking birth data (easily done without detection in those days) to suit the prevailing pop astrology. Unless planetary effects emerge when artifacts are absent, as when parents are excluded from the birth reporting process, they can no longer be seen as supporting astrology. Similarly, if planetary effects do not emerge, there is no need for theories to explain them such as those based on the pineal gland (Frank McGillion), reincarnation (Peter Roberts), and magnetospheric resonance (Percy Seymour).

Conclusions
First, it can take a long time for artifacts to be found. To quote examples from other areas, the Cottingley fairies stood for over 60 years before they crumbled. It took 40 years for the Fox sisters to admit their spirit communications were a schoolgirl prank that got out of hand. It took 40 years for Soal's "watertight evidence" for psi to be revealed as fraudulent. It took nearly 60 years to learn that the Loch Ness Monster in the famous 1934 photo was really a fake. It has taken more than 60 years to discover that the sheep-goat and decline effects usually seen as support for ESP in card guessing are due to the effect of personality variables on sequential guessing behaviour that either tends to match (sheep) or not match (goats) the limited probabilstic runs possible in a deck of 25 Zener cards (Brugger & Taylor 2003). Even more orthodox areas such as direction finding in pigeons, or the link between smoking and cancer, have taken decades to study and unravel. So we need to be cautious about any new astrological "proof".

Second, the existence of artifacts does not deny genuine effects. But unless research can confirm such effects when artifacts are controlled (which in astrology is so far not the case), we have good reason to suspend belief. Of course we should not confuse levels. We can never be sure whether effect sizes of 0.01 (equivalent to a hit rate of 50.5%) are genuine, just as we can never be sure whether surfing in Hawaii affects the waves in Australia, but we can be sure about failing to find effect sizes commensurate with astrological claims, say 0.5, just as we can be sure about failing to find a cat in a shoebox.

Third, the above artifacts in data have effect sizes around 0.04 to 0.10, equivalent to hit rates of around 52% to 55%, so even if they were genuine astrology and not artifacts, they would still be too small to support the claims of astrologers.

References

From www.astrology-and-science.com       12m 2g 63kb       Home       Fast-Find Index