Artifacts in data
Abstract -- The controversy over astrology is largely about artifacts, or whether the claimed results could have ordinary explanations. Artifacts are fake effects due to hiccups in data (this article) or in reasoning (see the article Artifacts in reasoning). They are never mentioned in astrology books yet they occur throughout astrology, leading to wrong conclusions that astrology works. You will be led seriously astray if you learn about astrology without first learning about artifacts. This article looks at artifacts in the data, as when sun sign counts are not corrected for the earth's elliptical orbit (the sun spends two days longer in Cancer than in Capricorn). Some artifacts in data became famous in their day as the best claimed evidence for astrology. They include Nelson's correlation between planets and radio quality, Brown's lunar effects on oysters, Bradley's Jupiter Pluvius rainfall effect, Jung's astrological experiment with married couples, Vernon Clark's matching tests, John Addey's harmonics, the Mayo-Eysenck zodiac zig-zag, and Gauquelin's planetary effects. All have taken considerable time (decades) for their artifacts to be uncovered. So we should be cautious about any new astrological "proof". Of course the presence of artifacts does not deny the existence of genuine effects. But unless research can confirm such effects when artifacts are controlled (which so far is not the case), we have good reason to suspend belief. Artifacts of data and reasoning have always raged out of control in astrology. 27 references.
When astrologers claim that astrology works, they imply that all non-astrological influences leading to the same result have been ruled out. Such influences are called artifacts, not to be confused with artefacts as in man-made objects of archaeological interest. Artifacts are fake effects due to hiccups in data or reasoning that mimic astrology and persuade us (wrongly) that astrology works. Artifacts in data are discussed in the present article. Artifacts in reasoning are discussed in the companion article Hidden Persuaders on this website under Problem Areas.
An artifact is something other than astrology that would result in apparent astrological effects. Astrological effects have tiny effect sizes (see Index), and it does not need much of an artifact to produce a tiny effect size. But you will not find artifacts mentioned in astrology books. In fact astrologers have shown no interest in preventing artifacts, and every interest in using procedures known to encourage them. You will be led seriously astray if you learn about astrology without first learning about artifacts. Don't believe that astrology works unless you can be sure that artifacts have been ruled out. This and other articles on this website will get you started.
Some artifacts in data should be obvious, as when sun sign counts in the astrologers' tropical zodiac are not corrected for the earth's elliptical orbit (the sun spends two days longer in tropical Cancer than in tropical Capricorn), or when astronomical constraints are ignored (thus Venus cannot be square the Sun), or when the claimed frequency of astrological features is to be expected by chance anyway (for examples see the review of Astrology Really Works! on this website under Book Reviews). Other artifacts in data are more subtle and resistant to detection, as in the following examples, some of which became famous in their day as the best claimed evidence for astrology.
Nelson's correlation with radio quality
Brown's lunar effects on oysters
Bradley's Jupiter Pluvius rainfall effect
Jung's astrological experiment
Vernon Clark's matching tests
Observed effect sizes in 54 matching tests where a total of 742 astrologers matched a total of 1407 birth charts with their owners. Clark's three results are shown as grey dots. By meta-analysis the mean effect size weighted by sample size is 0.035 sd 0.117, equivalent to a hit rate of 51.7% sd 5.8%, no different (p=0.77) from the 50% expected for guessing. The expected sd due to sampling variations alone is 0.199, considerably higher than the observed sd of 0.117, which leaves nothing in the results (including Clark's results) for astrology to explain. So the scatter in results is entirely due to using too few birth charts (typically 10) per study. When two studies using 120 charts each (black dots) averaging 50.3% hits are sub-divided into 24 studies of 10 charts each, the same scatter emerges, showing how scatter is an unavoidable artifact of small sample sizes. If the samples are further sub-divided, the scatter increases, exactly as predicted.
Thus the scatter of results in the above figure can be entirely explained by sampling variations, a point further considered in the next section, which means that the results (including Clark's results) provide no support for astrology. Further analyses suggest that the slight preponderance of positive results (ie hits over 50%) is due to publication bias, the reluctance of editors to publish negative results. Unfortunately most astrologers seem unaware of the many tests that now exist, and how the current verdict is no longer biassed by the large sampling artifacts of the earlier tests. For example Geoffrey Cornelius (1994), after citing 9 early tests, says "I have not heard of other experiments of this type" (p.84), while Ken McRitchie (2004) says "few tests have been conducted" (p.28).
Illusory results due to sampling variations
Observed effect size in 1000 sub-samples taken at random from a very large sample in which the effect size is known to be 0.000. The sample sizes range from 3 to 250, mean 80, and the mean observed effect size is -0.004 sd 0.198, equivalent to a mean hit rate of 49.8% sd 9.9%, which as expected is not significantly different from the expected 0.000. In small samples such as those commonly used by astrologers, the observed effect size (thanks to sampling variations) can be wildly different from what is actually there, leading to wildly wrong conclusions.
As the above figure shows, just by using their usual small sample of ten birth charts, or even a hundred birth charts, astrologers are virtually guaranteed of finding something that is interesting, exciting, full of promise, and totally spurious. As next.
John Addey's harmonics
Mayo-Eysenck zodiac zig-zag
Gauquelin's planetary effects
Second, the existence of artifacts does not deny genuine effects. But unless research can confirm such effects when artifacts are controlled (which in astrology is so far not the case), we have good reason to suspend belief. Of course we should not confuse levels. We can never be sure whether effect sizes of 0.01 (equivalent to a hit rate of 50.5%) are genuine, just as we can never be sure whether surfing in Hawaii affects the waves in Australia, but we can be sure about failing to find effect sizes commensurate with astrological claims, say 0.5, just as we can be sure about failing to find a cat in a shoebox.
Third, the above artifacts in data have effect sizes around 0.04 to 0.10, equivalent to hit rates of around 52% to 55%, so even if they were genuine astrology and not artifacts, they would still be too small to support the claims of astrologers.
ReferencesAddey JM (1976). Harmonics in Astrology. Cambridge Circle Press, Green Bay WI.
Bradley DA (1957). Your powwow corner. American Astrology September 1957, 42-44 and 69. Appears under the pen-name of Garth Allen
Brown FA (1954). Persistent activity rhythms in the oyster. American Journal of Physiology 178, 510-514.
Brugger P & Taylor KI (2003). ESP: Extra Sensory Perception or Effect of Subjective Probability? Journal of Consciousness Studies 10(6-7), 221-246.
Clark V (1961). Experimental astrology. In Search, Spring, 101-112.
Cornelius G (1994). The Moment of Astrology. Penguin Arkana.
Dean G (1983a). Shortwave radio propagation non-correlation with planetary positions. Correlation 3(1), 4-37.
Dean GA (1983b). Forecasting radio quality by the planets Skeptical Inquirer, 8(1), 48-56.
Dean G (1983c). Can Self-Attribution Explain Sun-Sign Guessing? Correlation, 3(2), 22-27.
Dean G (1986). Can Astrology Predict E and N? 3: Discussion and Further Research. Correlation 6(2), 7-52. With 110 references. Includes meta-analyses of astrological studies.
Dean G (1996). A Re-Assessment of Jung's Astrological Experiment. Correlation, 14(2), 12-22.
Dean G (1997). John Addey's dream: Planetary harmonics and the character trait hypothesis. Correlation 16(2), 10-39.
Dean G (2002). Is the Mars effect a Social Effect? Skeptical Inquirer, 26(3), 33-38.
Enright JT (1965). Journal of Theoretical Biology 8, 426-468.
Eysenck HJ and Nias DKB (1982). Astrology Science or Superstition? St Martin's Press, New York.
Gauquelin M (1960). Les hommes et les astres. Denoel, Paris.
Gauquelin M (1983). The Truth about Astrology Blackwell, Oxford.
Ianna PA and Margolin CJ (1981). Planetary positions, radio propagation and the work of J H Nelson. Skeptical Inquirer 6(1), 32-39.
James C (1981). Bradley's Jupiter-Rainfall study Correlation 1(2), 19-23.
Jung CG (1960). The Structure and Dynamics of the Psyche, in The Collected Works of C.G.Jung, ed H Read, M Fordham and G Adler. London: Routledge & Kegan Paul, London, volume 8, 459-484.
Mayo J, White O and Eysenck HJ (1978). An Empirical Study of the Relation Between Astrological Factors and Personality. Journal of Social Psychology, 105, 229-236.
McRitchie KD (2004). Environmental Cosmology. Cognizance Books, Toronto.
Meeus J (1982). On the "Correlation" Between Radio Disturbance and Planetary Positions. Skeptical Inquirer, 6(4), 30-33.
Nelson JH (1951). Shortwave Radio Propagation Correlation with Planetary Positions. RCA Review March, 26-34.
Slevin J (1999). Why astrology works. Rialta 5(1), February 1999. Also available at http://cura.free.fr/decem/09slevin.html. Concludes that magnetic disturbances are why astrology works, and "The pioneering work of John H.Nelson and the recent theory of Percy Seymour have modern scientists poised to alter their entire perspective on the celestial mechanics of the universe". Ironically Slevin (then NCGR Co-Director of Education) wonders why Seymour does not cite Nelson.
Smithers AG and Cooper HJ (1978). Personality and Season of Birth. Journal of Social Psychology, 105, 237-241. Findings support those of the immediately preceding study by Mayo et al. (1978).
van Rooij JJ (1999). Self-Concept in Terms of Astrological Sun-Sign Traits. Psychological Reports 84, 541-546.