|
From www.astrology-and-science.com Click here to return to home page Vested Interests 1, Scientific Integrity 0 Ivan W Kelly This article without the sub-title was the lead article in Skeptical Briefs 10(1), March 2000. At the end Geoffrey Dean provides a follow-up covering the subsequent years to 2006. Ivan W Kelly is Professor of Educational Psychology and Special Education at the University of Saskatchewan. Abstract -- In the author's opinion Correlation under Rudolf Smit (editor 1993-1998) had become one of the most impressive and open-minded journals in existence, the model example of a journal in any area. Smit was also editor and translator for one of the Netherlands' leading scientific institutes, and his open-minded stance conflicted with the AA's attitude that only positive results are welcome. The AA argued that negative articles should be subject to their veto, which Smit (like his predecessor Simon Best 1981-1992) saw as contrary to the spirit of science. In due course the AA replaced Smit with practising astrologer Pat Harris, whose first issue revealed such a disastrous loss of rigor that, if it continues, can only provide ammunition for critics. A follow up reveals that the loss of rigor did indeed continue, together with farcical refereeing and censorship of unwelcome material. When in 2000 three Correlation consulting editors raised concerns about the journal's poor intellectual credibility, the editor reacted by sacking them. The days when Correlation could be regarded as a serious journal are clearly over. In the fall of 1998, Rudolf Smit, then editor and translator for one of the Netherlands' leading scientific institutes, and an ex-astrologer, was sacked from his spare-time position as editor of the British astrological research journal Correlation. Even if astrology is not your pet subject, Smit's sacking provides a salutary look at paranormal politics and what happens when science is unable to deliver the support required by vested interests. The story begins in 1958, when research-minded British astrologers founded the Astrological Association (AA), specifically to encourage research. Its aims included "to enlarge the knowledge of Astrology in a scientific spirit", and were accompanied by a brief statement urging the use of the scientific approach for "distinguishing factual truth from error, for checking theories". The AA was not the first astrological research body but it was to become the most active, the most expert, and easily the most important. Twenty years later its findings were included in Recent Advances in Natal Astrology, a massive critical review (608 pages, 1020 references) prepared under the aegis of the AA by Geoffrey Dean with Arthur Mather and 52 others from 10 countries. Nothing like it had appeared before and it quickly became the bible for astrology researchers world-wide. Its conclusion, "The picture emerging suggests that astrology works, but seldom in the way or to the extent that it is said to work", has been repeatedly confirmed by subsequent research and repeatedly ignored by most astrologers. In 1979 the interest raised by Recent Advances led the AA to found an ongoing series of London research conferences, and in 1981 the research journal Correlation was established. Previously research findings had been published (albeit infrequently) in the AA's Astrological Journal, now generally regarded as the leading English-language journal devoted to serious astrology, with nearly 30 shelf-inches of issues to date. Correlation was the world's first refereed scientific journal devoted to astrological research. It accepted "articles reporting empirical research into astrology, review articles, and those discussing methodological, conceptual and philosophical issues relating to astrology" with the proviso that "all submissions are refereed for content and clarity of expression." Definitely not your average astrology journal. Its circulation eventually reached about 600, which compares favorably with the median circulation for scientific journals in parapsychology (about 1000) or in astronomy or psychology (about 2000). But it remained generally unavailable in academic libraries, so its contents (about 6 shelf-inches of issues to date, which include about 70 empirical studies) are little known. Correlation inspired other research journals, notably in France, the USA and Holland, where one non-refereed precursor had been founded in 1977 by Rudolf Smit. But it remained the only one with high standards. Under the editorship of Simon Best (1981-1992) and Rudolf Smit (1993-1998), its twice-yearly issues achieved commendable levels of scientific rigor and impartiality, which ultimately led to the political problems described later. One frequent scientist contributor reported "never did I feel that articles with negative results were returned for revision or rejected ... AA organivers never made me feel like an outsider ... whereas other [skeptic] journals provided many instances of prejudiced, dogmatic and restrictive behavior to people of the wrong faith". No mean achievement. Indeed Correlation under Rudolf Smit had, in my opinion, become one of the most impressive and open-minded journals in existence, the model example of a journal in any area (and I do not say this lightly). Thus few if any of the journals put out by a particular school of thought such as the psychoanalytic journals have the rigor or the integrity to publish the kind of searching articles that Smit did. To appreciate his achievement one need only look at supposedly serious astrology journals such as The Mountain Astrologer, where informed debate is conspicuously absent (and, according to insider sources, actively discouraged). Unfortunately Smit's open-minded stance conflicted with the underlying attitude of most astrologers, which is that research should support (not challenge) existing beliefs and practices, so only positive results are welcome. But the research results submitted to Correlation tended to be negative, and even if positive they were never commensurate with the claims of astrologers. Whenever promising areas were investigated, such as Nelson's radio propagation claims, Bradley's link between rainfall and Jupiter, Mayo's zodiac zig-zag, Clark's chart matching experiments, Addey's harmonics, Gauquelin's link with character traits, the classic time twins George III and Samuel Hemmings, and more recently the sun-sgn claims of Gunter Sach and a Manchester University marketing study, serious flaws were uncovered and the promise disappeared. When combined with studies published elsewhere the result was a compelling case against the theory and practice of astrology. But it got worse. To astrologers even more upsetting were the key astrological discourses begun under Rudolf Smit in 1994, in which astrologers and scientists systematically surveyed key topics in astrological research. The first topic was "Is the scientific approach relevant to astrology?" Over 100 potential debaters were contacted, of which eventually twenty-five astrologers and eight scientists collaborated on a core discourse and then provided commentaries. The general conclusion was unremarkable, namely that the scientific approach is relevant only where claims are testable, but the debate itself was ground-breaking. Never before had astrologers and scientists tried so hard to resolve their differences. Readers in favour of research were delighted ("excellent ... a huge job that was well done"), but not the AA, whose secretary claimed it was "of little interest ... if further issues ... continue in the same vein, the subscription to Correlation will be halved." Subsequent astrological overviews involved the same collaborative approach and more AA displeasure. The topics were conceptual problems (devastating), theories of astrology (none credible), and biases of human judgment (neglected by astrologers but readily explaining their beliefs). No doubt the AA was displeased because the surveys seemed to deny even the promise of future promises. Furthermore the AA had revised its aims, the links with research now being "to encourage ... members to undertake astrological research ... for the good of astrology" (no scientific qualifier here), and "to enhance ... the good name of astrology among ... the scientific community" (which seems to leave no room for negative results). The once pro-science views of the AA had become anti-science or at best lukewarm, so rigorous scientific surveys were now unwelcome. The final straw occurred in 1997 when in issue 16(2) pages 40-56, Geoffrey Dean made a rigorous critique of the twenty-nine entries to a "Truth of Astrology" essay competition sponsored by various bodies including the AA. He concluded that even the prize-winning essay "abandons scholarship and clear thinking in favour of muddle and misrepresentation", so until astrologers become better informed about research there was little hope of their ever gaining recognition in scientific circles. Astrologers were outraged. They responded not with letters in which each criticism was systematically refuted (only one letter was received and even that was hopelessly muddled) but with tearful complaints to the AA. Even though the focus on "truth" invited the scientific approach, they failed to adopt it. [You can read Dean's critique on this website under Doing Scientific Research] The AA immediately called editor Rudolf Smit to account. They argued that the scientific approach to astrology was simplistic and misleading because astrology operates on principles outside those of known science, so non-scientific approaches such as spiritual or postmodernist should be favored. Their argument ignored the contrary findings of the collaborative surveys, and was a clear sign that from now on only vested interests would matter. The AA also argued that Geoffrey Dean's input was too extensive and too critical, which among other things supposedly discouraged potential contributors, so he should be banned from at least two subsequent issues. But, worst of all, the AA demanded that for the final refereeing of Dean's articles they would have the last word. Hence, if they did not like them, Dean's articles could not be published, period, whatever their quality. So much for editorial independence. (This is the same Geoffrey Dean whose two-part critique of astrology in Skeptical Inquirer 1986-87 was reprinted in the Hundredth Monkey Phenomena anthology, where it was described as "classic and thorough ... hailed as perhaps the best ever done" and a main contributor to the entry on astrology in Prometheus Books' Encyclopedia of the Paranormal.) Smit accepted the first argument even though postmodern and spiritual approaches seemed irrelevant to the testable claims made in astrology books. But he rejected the second as censorship and an attack on scientific integrity. The AA disagreed. In due course they sacked him and appointed Pat Harris, an astrologer and PhD student at Southampton University studying the application of astrology to health psychology. The AA gave an assurance that Correlation under Harris would continue with the same intellectual rigor as before. This seemed at odds with their previous arguments so I asked for a clarification. In condensed form my questions and the AA's answers were as follows: Q: If little will change in content, why change editor? Q: If the problem is too many critical articles, how will the new editor
solve it? By censorship? When it was pointed out that their answers evaded the issue, they responded with further evasion, and when challenged still further evasion. Evidently the AA publicly welcome science for its prestige but secretly reject it for its rigor. In my view the AA should now abandon any pretense that Correlation (or any other AA publication) is genuinely scientific. In fact there seems no need even for pretense -- the first issue of Correlation under its new editor reveals such a disastrous loss of rigor that, if it continues, can only provide ammunition for critics. Quite the reverse of the AA's aim of enhancing "the good name of astrology among ... the scientific community". My conclusion is this: Half a century of research has made it clear that no astrological body could embrace science and stay in business. It would be like introducing devil worship into theology. That no national astrological body including the AA has officially rejected the (at best) extremely problematic notion of sun signs is evidence enough that politics and membership fees come before scientific integrity. I should add that this conclusion affords no pleasure whatever. The efforts of the early AA in promoting scientific research into astrology deserve the highest praise. It seemed that astrologers were at last putting their house in order, which view must now be abandoned, perhaps forever. That this should happen in British astrology, often regarded as the last outpost of astrological sobriety, should give skeptics cause for concern. Follow-up July 2006 When a journal is found to be lacking in the very integrity and rigour that it lays claim to, one expects a responsible publisher to take corrective action. But in the six years since Professor Kelly raised his concerns the AA has taken no corrective action. The disastrous loss of rigour and censorship of unwelcome material still continue. Correlation has disqualified itself as a journal worthy of serious consideration. Farcical refereeing In mid 1998 Correlation 17(1) ran an article by Suitbert Ertel in which eleven experienced astrologers failed to tell 20 painters from 20 politicians using birth charts. A critique was subsequently received from Christopher Bagley but the referee concluded (in 1500 words listing 12 major deficiencies) that it was uninformed and not acceptable for publication. For example it attacked the study's motives even though Ertel was merely testing the astrologers' own claims, it took no heed of previous studies, key arguments were misleading or invalid, and much of it was irrelevant. So it should be returned for improvement. But the new editor Pat Harris ignored the referee's report and published the critique unchanged together with her glowing endorsement. At the time Bagley was her supervisor at Southampton University. Without checking with the consulting editors, the editor also changed Correlation's sub-title from "Journal of Research INTO Astrology" to "Journal of Research IN Astrology". However, when Correlation first started in 1981, INTO was adopted only after much discussion, and the choice was not made lightly. The editor was asked if her next editorial could explain the change, but she brushed away the request by saying the reason for the change was "obvious". Concerns about the brush-off were dismissed by "it's just me". Concerns about refereeing arose again when 19(1) published Mike Harding's article on "Prejudice in Astrological Research". Of one referee's 65 recommendations (6000 words with 40 references!), only 1 in 6 were adequately responded to. The rest were ignored. The referee raised the matter with the editor, who replied that her policy on referee recommendations is that "the author is not bound to follow them". This farcical policy led three consulting editors (Dean, Smit, and Spencer, all professional editors with more than fifty years of professional editing experience between them) to submit a letter to Correlation about the lack of rigour, using Harding's article as an example of unacceptable standards. You can read this letter in the Appendix. It was January 2001. The editor's response was evasive and delaying (the letter was not published until 12 months later), which led the three authors to circularise their concerns to the other consulting editors. To which the editor responded by sacking Dean and Smit as consulting editors. Spencer had already resigned in protest. As for Harding's article, the editor's justification for rejecting the referee's recommendations was that she "considered it was ready for publication". The same cavalier attitude towards referees still applies today. For example none of the referee's comments on Kollerstrom's article in 23(1) was acted on, not even the correction of typos. Now for censorship. Censorship at Correlation is alive and well Example 1 Round 1 The editor felt that "a major flaw in your study design" was due to "asking for a range of opinions on a very mixed bag of data". In fact the invitation was to submit tests not opinions. The editor felt that philosopher Dr William Grey's position on sun signs was "foolish and uninformed". In fact Dr Grey specialised in the philosophy of science, had conducted surveys of astrology, and was one of the few philosophers who had gone to the trouble of meeting astrologers and of having his chart read. The editor felt that the AA's refusal to reveal their position on sun signs could be because the AA considered the authors' invitation "to be a waste of time because of the flaws I have pointed out in the study design". But how is asking the AA for its position on sun signs a waste of time? Round 2 The authors raised these and other points with the editor, who replied "Why were not sunsign columnists consulted on the way in which their columns were written?". But they were. "There is no citation of research into differences between perceived accuracy in various sunsign writers' columns". How is this relevant to devising tests? "Your final conclusions are on very shaky ground. Where is your own reader survey of sunsign columns in order to compare this with astrologers' comments?" But why should devising a test require the authors to survey readers of sunsign columns? Where is the connection? "Your methodology is not sufficiently carefully thought out to support your overall conclusions." But if the authors get five replies to ten invitations, and conclude that five people replied, how is their conclusion not supported? Round 3 The authors stressed that they were merely reporting the response to an invitation to devise tests of sun signs. They were not performing tests, yet the editor was implying the opposite. The editor replied "The essential problem with your research is that it is not properly designed towards a practical end." But how was the end (devising tests of sun sign columns) not practical? "I repeat that this is not scientific writing and analysis and is not, therefore, suitable for Correlation in the form in which it is presented." Round 4 The authors then wondered if their results might be more suitable for Correlation's Forum, which is described as a section "open for various communications such as letters and comments" which will be "considered as preliminary publication open for encouraging and also critical comment". So they asked the editor under what conditions might their material be considered for Forum, for example with limits on length or focus. The editor replied "There are no conditions under which I would accept your sunsign material for Forum. Your article is unscientific in its presentation and analysis." It was now January 2000. Outcome Dean and Mather had then been collaborating on scientific articles about astrology for more than twenty years since finishing Recent Advances, and had a solid reputation among scientists for quality work. Their rejected sun sign article was immediately snapped up by another journal, and appears on this website under Sun Signs. See if you agree with the Correlation editor that it is flawed, careless, poorly designed and unscientific, and let this website know. Example 2 Example 3 Example 4 Example 5 In summary Appendix. Letter to Correlation From the following Consulting Editors in alphabetical order: Refereeing policy reduces Correlation's credibility To start with, PIAR boils down to arguing that astrology is incompatible with science, therefore science is largely irrelevant. Clearly this will not do. The problem is not that astrology is incompatible with science but that it is unconvincingly so -- the arguments typically offered in favour of astrology fall well below the standards of critical thinking expected of any intellectual discipline, see Kelly's article in Psychological Reports 1997, 81, 1035-1066, and the totality of research outcomes is so negative that the case for astrology as a source of knowledge could be fairly said to have collapsed, see issues of Correlation since 1990. Indeed, the emerging picture would seem to deny any incompatibility between astrology and science, see Chapters 9 and 10 of Phillipson's Astrology in the Year Zero. Overall, PIAR ignores informed criticism in favour of uninformed criticism, and condemns existing methodology without saying what should be done instead, thus contributing nothing to the debate. But the criteria that scientists and others use to judge the merits of an idea such as astrology (for example its empirical track record, its ability to meet critical scrutiny, and its consistency with the evidence) are not arbitrary. For them, astrology's conspicuous failure to meet these criteria is the deciding factor. Until the proponents of astrology recognise this, PIAR's complaints of prejudice are beside the point. As Phillipson says (p.181) "criticisms of science (no matter how valid they may be) do nothing to prove astrology." However, what arouses our main concern is PIAR's response to refereeing. Of one referee's 65 recommendations (6000 words, 40 references, or about 15 Correlation pages), only 1 in 6 were adequately responded to. The rest were ignored, leading to severe assailability both of PIAR and of Correlation's credibility, all of it avoidable, as shown by the following examples: PIAR ignores KTs [Key Topics previously published in Correlation], where most of its arguments are contested, and its treatment of Ertel ignores Ertel's letter in 18(2), as if scholarship did not matter. PIAR often argues by innuendo -- because some scientists are hostile to astrology, they all are; because Carlson's article was flawed in some areas, it was flawed in all areas. But neither is true. PIAR says that Adorno's anti-astrology views reveal only psychobabble, as if current criticism of such psychobabble did not also apply to astrology and PIAR's own astrobabble (as in fear = Saturn). PIAR says that astrological practice is based on experience, as if experience was unproblematic (it is hugely problematic, see KT4). PIAR says that phrenology's whole approach came from science, which is incorrect, and ignores how phrenology highlights the problems of experience despite this point being stressed by the referee. PIAR says science's worldview "is often fatally flawed by its own assumptions about the nature of human beings", but without giving examples, as if any doubt was inconceivable to the author. Worse, the possibility that astrology might also be "fatally flawed by its own assumptions" seems equally inconceivable, as if only science (but never astrology or anything else that is nonscientific) could ever be wrong. Is PIAR as unprejudiced as it demands others to be? It seems not. It gives no hint that many of its topics, such as research into astrology, have a scholarly literature. It does not discuss relevant findings. It ignores informed criticism. It attacks the Skeptical Inquirer, which it repeatedly mispells as Enquirer, but inexplicably ignores SI's articles on astrology. These examples (many more could be cited) show how PIAR has failed to maintain the standards of argument, of scholarship, of clarity, of impartiality, and of concern for the reader, that attention to referee recommendations would have ensured. These deficiences are not a matter of opinion but a matter of standards; they apply whether or not we agree with PIAR's views. But PIAR is not an isolated case. The referee recommendations (12 major items, 1500 words) for Bagley's article in Correlation 18(1) were similarly ignored, as was a follow-up letter pointing out the resulting lack of standards, despite the statement in 18(1) that "letters and comments on these papers will be very welcome." However the AA's policy on referee recommendations is that "the author is not bound to follow them", which is precisely the process attacked by professional editor Kenneth Irving in 16(2), since although it might improve an author's writing "it cannot and should not replace the kind of independent review used by any journal adhering to the standards of academic publishing." The same view emerged from a poll of Correlation consulting editors in 1995, who unanimously opposed a lowering of standards. But PIAR shows that conspicuously lower standards are now a reality, which confirms our experience that consulting editors have not been consulted on policy since the previous editor stepped down at the end of 1998. Correlation makes the claim that all submissions are "refereed for content and clarity of expression", which may reasonably be understood to imply a compliance with imposed standards. But due to AA policy the claim is meaningless, as is the claim that consulting editors are responsible for those standards. Every professional refereed journal that we know of puts the referees in charge of standards, not the authors. No referee is perfect, therefore authors always have the right to question decisions, but they have no right to decide standards. We therefore call upon our fellow consulting editors to seek changes to AA policy, and to insist that consulting editors be actually consulted on this and other policy matters. If no changes are made, we fear that readers will see Correlation as having become, as Kelly (Skeptical Briefs March 2000) puts it, a case of "Vested interests 1, Scientific integrity 0." No academic journal can afford such a reputation. Editor's response As editor I welcome Geoff Dean's, Rudolf Smit's and Wayne Spencer's comments. In early December I took an editorial decision to move entirely to anonymous peer review to bring Correlation into line with other academic practice. As of now all papers will be sent anonymously to a minimum of two reviewers. While in a small community anonymity will sometimes be impossible, since some researchers will be aware of, and recognise, each others' work, the goal is to focus on content rather than personality, which latter has often been the case in the past. I hope that Correlation will also thus set an example to other research periodicals and web sites dealing with astrology. As far as the use of reviewers' criticisms are concerned, these obviously fall into different classes. Factual errors must obviously be corrected, but differences of opinion are a matter of the editor's judgement. I propose that in future, when reviewers have extensive criticisms of a paper, I will send them a second draft for comment. The purpose of the editorial board is to advise the editor, as well as take on specific reviewing tasks, and I welcome all input from members of the board. Mike Harding's article "Prejudice in Astrological Research" 19(1) was resubmitted, after peer review, and I considered that it was ready for publication. It is a necessarily provocative article and I am happy to see that it has succeeded in provoking a response from Correlation readers. It was also warmly received by a number of subscribers to the journal, both in the UK and abroad, and also members of the editorial board. I am happy to be able to advise readers that there will be further papers from Mike in future issues of the journal. These will be subject, like all future material, to anonymous peer review. [There was no response from Mike Harding nor have further papers from him appeared in Correlation] From www.astrology-and-science.com Click here to return to home page |