By Gregory Coles. Greg is a Senior Research Fellow at The Center and is the author of Single, Gay, Christian and No Longer Strangers.
In our last episode, I suggested that the Journal of Human Sexuality isn’t a highly credible venue for a groundbreaking study of sexual orientation change to be published. Still, a dubious venue doesn’t necessarily mean that the research published there is bad. In fact, if other publishing venues are hostile towards a scientific study’s legitimate findings, dubious fringe venues might be the only place where legitimate scientific work can gain an audience.
So let’s take the Pela and Sutton study on its merits, beginning with:
PARTICIPANTS & RETENTION RATE
According to the abstract, this two-year longitudinal study featured 75 adult male participants. In reading the study itself, we learn that this is true, but not the whole truth. In fact, 105 adult males began the study—and by the end of two years, only 22 of them remained.
Buy Now!
Embodied: The Latest from Preston Sprinkle
Of the original 105 participants who took the pretest (the baseline survey against which future results could be measured every six months), 30 participants left the study in less than six months, never completing the first posttest. This left 75 participants whose six-month posttest data could be analyzed. At the one-year posttest mark, only 53 participants provided data; at the 18-month-mark, only 28 participants; and at the two-year mark, only 22 participants. Of those who stopped participating in the study, the majority did so because they stopped using Reintegrative Therapy.
In other words, the study was truly a longitudinal two-year study.[1] And it did in fact have 75 participants. But only 22 of those participants were actually studied for the full two years. The other 53 discontinued therapy before then. This places the study’s retention rate at a striking low 29%. (By comparison, the average retention rate for longitudinal studies with mostly male participants is around 70%.)[2]
There’s nothing inherently shocking about a longitudinal study losing some people over time. And there’s also nothing inherently shocking about people choosing to discontinue therapy sessions, as these authors are quick to describe:
In routine clinical settings clients autonomously end treatment for a variety of reasons. Often treatment ends because either the client, the therapist, or both believe that the therapeutic goals were met, or have determined that the treatment has plateaued in its effects. Other reasons for ending treatment include geographic relocation, changes in insurance coverage, or the desire to pursue other treatment options. Since this study took place in such a real-life clinical setting, treatment length was individualized according to the needs of the participants and therefore varied for each participant.
This is all quite true. But in the case of a study like this one, its truth raises some important questions:
- Wouldn’t people who continue opting into (and paying for) an expensive niche therapy approach be more likely to be the ones who think it’s working?
- Conversely, wouldn’t this mean that anyone who finds the therapy not working for them is far more likely to drop out of therapy (and thus drop out of the study) before the next six-month check-in point?
- Accordingly, wouldn’t this skew the gathered data disproportionately in favor of those who feel they are “succeeding” at becoming straighter?
- Let’s suppose, though, that all these people dropping out of Reintegrative Therapy are doing so because they feel it’s been effective. Wouldn’t it be helpful to continue collecting data from those participants after they finish therapy, to confirm that their newfound heterosexuality persists beyond the conclusion of their therapy?
In previous decades, the pattern of sexual orientation change testimonies has been for people to assert their changed orientation for a period of time, then later admit that they never really became straight and were instead influenced by wishful thinking and social pressures.[3] In light of this pattern, the purportedly “longitudinal” nature of the Pela and Sutton study is especially important. Yet even this two-year time frame applies to only 22 of the 105 participants originally recruited.
But those 22 participants definitely became straighter, right?
To answer that question, we need to evaluate…
STUDY DESIGN & CATEGORY ERRORS
The fact that this study relies on participants’ self-reports of their attractions is by no means damning, but it’s certainly noteworthy. Human beings aren’t always the most accurate judges of how their own unbidden attractions are occurring, especially when they’re highly incentivized to want a certain answer to be true. I was roughly thirteen years old the first time I told someone that my attraction to the same sex was “definitely getting a little better”—by which I meant that I was becoming straighter. I believed at the time that I was telling the truth; but in retrospect, I only thought it was the truth because I so deeply wanted (even philosophically needed) it to be the truth.
All that to say, self-report is a wily creature.[4]
The more significant concern about study design lies in the way “attraction” is defined and measured. The study measures the degrees of participants’ same-sex attraction and opposite-sex attraction by asking them about their sexual behavior, kissing, thoughts, feelings, and sexual identity. While all these data points might be interesting, only thoughts and feelings have to do with sexual orientation per se. I, for instance, have never engaged in any same-sex sexual behavior or kissing; but these things don’t make me any less oriented toward the same sex. By a similar token, a person who engages in frequent sexual behavior before therapy and decides to live chastely after therapy is now more chaste, but they are not necessarily any less gay in their orientation.
What about sexual identity? Here we see the same alchemy that previous Christian sexual orientation change efforts became adept in once they began to realize people’s sexual orientation wasn’t really changing: they emphasized the importance of no longer self-describing as gay. A person who still “experienced same-sex attractions” but no longer called themselves gay could be counted as “not gay” and therefore as proof of the movement’s success.
While the Pela and Sutton study, to its credit, keeps sexual identity distinct from the other four metrics of “attraction,” it still uses data about people’s changing self-description of their sexual identity to make an argument about orientation change. Far more problematically, the authors lump sexual behavior, kissing, thoughts, and feelings together into two generalized metrics they call “same-sex attraction experiences” and “opposite-sex attraction experiences.” By using these lumped-together metrics to make claims about orientation change, they render their data useless to offer any meaningful understanding of mere orientation (especially since the majority of their clients are religiously motivated and would desire to pursue chastity even apart from a change in their sexual attractions).
To demonstrate how unhelpful these metrics are for measuring sexual orientation: Imagine that a man exclusively attracted to other men went into therapy, then decided to start kissing a lot of women. Thanks to the structure of this study, his “opposite-sex attraction experience” number would skyrocket, thereby demonstrating a miraculous change in orientation. That “orientation change” would have no impact on his continuing exclusive attraction to other men.[5]
So perhaps it would have been better for the researchers to parse out changes in kissing and sexual activity from changes in thoughts and feelings.[6] But do the data at least show some people seeming to move towards heterosexuality?
Not exactly. Let’s finally talk about…
THE FINDINGS
When we look at the average rating of 75 participants’ “same-sex attraction experiences” at the beginning of the study, and compare it to the average rating of 22 participants’ “same-sex attraction experiences” two years later at the end of the study, we see that the number does indeed go down, dropping from 2.63 (on a 5-point scale) to 2.39. Likewise, the rating for “opposite-sex attraction experiences” goes up, and the rating comparing sexual identity self-descriptions indicates a shift away from gay identity.
But when we study the numbers more closely, an odd trend emerges. Even though the participants’ final rating of their same-sex attraction experiences at the two-year mark (2.39) is lower than their initial report, it is actually higher than the same-sex attraction experiences they reported at the 18-month mark (2.12), the one-year mark (2.19), or even the 6-month mark (2.32). In other words, the final six months of the study seem to show people becoming more gay in their sexual orientation, not less so. They increase from 2.12 back up to 2.39, undoing more than half of their “progress” toward heterosexuality. If the study had continued for another six months, and the same trend had continued, the remaining participants would apparently have been even gayer than they were when they entered therapy in the first place. [7]
The same is true for the opposite-sex attraction numbers. These numbers steadily increase until the 18-month mark, then suddenly plummet back almost as low as their starting rate. Even sexual identity starts to make a swing back toward a gay self-identification.
The careful reader, then, can’t help but wonder what happened after the study ended and the data stopped being collected. Did participants remain in precisely the same state of sexual attraction as they reported at the end of the study, slightly “less gay” than when they started? Or did their experiences continue to evolve, perhaps moving in the inconvenient direction indicated by the last set of data? Did they wind up even gayer than they started?
The study doesn’t tell us, alas. The study can’t tell us.
So is Reintegrative Therapy making people straight (or, at least, straighter)? Their website will tell you that the answer is yes. But their website will also tell you that they know this because of large-scale, longitudinal, peer-reviewed evidence showing significant changes in sexual attraction.
Now that you’ve seen that evidence for yourself, I’ll let you decide whether it’s worth believing.
[1] It is worth noting, however, that two years is a relatively short period to be considered “longitudinal.”
[2] See Samantha Teague et al., “Retention Strategies in Longitudinal Cohort Studies: A Systematic Review and Meta-Analysis,” BMC Medical Research Methodology, vol. 18, no. 151 (2018), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6258319/.
[3] For a thorough analysis of this trend, see Greg Johnson’s book Still Time to Care (Zondervan, 2021).
[4] But short of penile plethysmography (the measure of blood flow in the penis, which is sometimes used as a way to measure sexual arousal in males), it’s tricky to imagine what better metric of sexual attraction we could use. (And it’s even trickier to imagine a bunch of males willingly signing up for a study involving penile plethysmography.)
[5] These metrics also fail to account for same-sex oriented people in mixed-orientation marriages, who would rate high in “opposite-sex attraction experiences” by virtue of their marriage to an opposite-sex partner, even if their same-sex orientation remained unchanged.
[6] Here, too, it would have been helpful to parse out moments of willful lustful thoughts from moments of sexual temptation, or even just passing awareness of attraction.
[7] Of course, given that the number of study participants keeps diminishing at each six-month increment, it's not possible to make a perfectly equal comparison between the 75 men who reported their attractions at the six-month mark and the 22 men who reported their attractions at the 24-month mark. Perhaps the 53 men who terminated therapy before the 24-month mark were having better success at becoming straight. The lingering 22 men might have been those whose same-sex attractions were strongest all along, thus creating an apparent upswing towards same-sex sexuality because of the disappearance of other men's (lower) numbers from the data set. Unfortunately, the study provides us only with average numbers from the entire (ever-changing) group at each six-month interval, rather than allowing us to trace the shifting experiences of the same 22 men over a longitudinal period. This, among other reasons, is why the study's failure to provide readers with comparable data sets from the same group of people across multiple points in time makes it difficult to draw conclusions based on the limited data provided.