Research with vulnerable populations: considering the Bucharest Early Intervention Project (part 1).

The Neurocritic alerted me, in a comment on an earlier post, to a pair of papers in the 21 December 2007 issue of Science that raise some difficult ethical questions about what sorts of research are permissible. Quoth the Neurocritic:

This may be a little off-topic, but I was wondering if you read this article in Science, beginning of abstract pasted below.

In a randomized controlled trial, we compared abandoned children reared in institutions to abandoned children placed in institutions but then moved to foster care. Young children living in institutions were randomly assigned to continued institutional care or to placement in foster care, and their cognitive development was tracked through 54 months of age.

Rather horrifying! Can you imagine this experiment being performed in a first- (or second-)world country in the 21st century? But the title of the paper is:

Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project

Is it now OK to perform this experimental intervention, since it's in Romania? ...

The authors of the study, Nelson et al., do have a lengthy discussion of ethical issues within the paper (e.g., the secretary of state for child protection in Romania invited them to do the study, the IRBs at Minnesota, Tulane, and Maryland [PI home institutions] approved the study, etc.). However, to me it seems to set off alarm bells in terms of ethics. I'm definitely not a developmental psychologist, but this statement seems odd:

Clinical equipoise is the notion that there must be uncertainty in the expert community about the relative merits of experimental and control interventions such that no subject should be randomized to an intervention known to be inferior to the standard of care (27). Because of the uncertainty in the results of prior research [??], it had not been established unequivocally that foster care was superior to institutionalized care across all domains of functioning... [Is the superiority of foster care really in doubt?]

In this post, I'll look at both Nelson et al., "Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project". [1] In a second post in the not-too-distant future, I'll look at the accompanying policy forum article, Millum and Emmanuel, "The Ethics of International Research with Abandoned Children" [2]. (I'm breaking it up into two posts because otherwise it may require you a full pot of coffee, rather than a mug, to get through it all.) My aim in these two posts will be to lay out the recognized ethical guidelines for research with human subjects as they apply to the Bucharest Early Intervention Project (BEIP), and to identify the worries we might raise about this kind of research -- and, by extension, with the prevailing standards.

Let's start with the research study itself. Nelson et al. were examining what are undeniably interesting and important questions about human cognitive development: the effects on cognitive development of institutional versus non-institutional environments, whether the effects of an institutional environment on cognitive development are robust or can be altered by shifting a child to a non-institutional environment (like foster care), and the extent to which the age of a child matters in terms of how big an effect the environment (or a shift from one environment to another) has on the child's cognitive development. Knowing how cognition develops, when cognition develops, and what kinds of things influence that development, is just the sort of thing developmental psychologists might want to figure out. And, because developmental psychologists are scientists, they'll want to figure it out with data from a well-designed study.

The difficulty arises when we notice that the most obvious way to get such data is by experimenting on little kids -- in this case, on little kids who will either get to live in an institution or get foster parents.

Indeed, reading the findings presented by Nelson et al., it's hard not to think about the kids involved in generating the data. They write:

Three main findings emerge from this study. First, as we have previously reported, children reared in institutions showed greatly diminished intellectual performance (borderline mental retardation) relative to children reared in their families of origin. Second, as a group, children randomly assigned to foster care experienced significant gains in cognitive function. Lastly, at first glance our findings suggest that there may be a sensitive period spanning the first 2 years of life within which the onset of foster care exerts a maximal effect on cognitive development. However, a closer reading of our analyses suggests a more parsimonious conclusion: That the younger a child is when placed in foster care, the better the outcome. Indeed, there was a continuing "cost" to children who remained in the institution over the course of our study. These results are compatible with the notion of a sensitive period, but discovering whether such a period truly exists or determining the borders that delineate it would likely require a larger sample size with a broader age range at intervention onset.

The results of this study have implications for child welfare because they suggest that placement in families is more advantageous for cognitive development in infants and young children than placement in institutional settings. For countries grappling with how best to care for abandoned, orphaned, and maltreated young children, these findings deserve consideration. The results also indicate that previously institutionalized children's cognitive development benefits most from foster care if placement occurs relatively early in a child's life. [3]

The study finds evidence to back the hunch that foster care provides a better environment for cognitive development than an institution -- but it does so by also finding evidence that "children reared in institutions showed greatly diminished intellectual performance (borderline mental retardation) relative to children reared in their families of origin".* The study establishes that a shift to foster care from an institutional setting can produce "significant gains in cognitive function" -- but this effect is established in part by the kids randomly left in the institution who didn't experience significant gains in cognitive function. And, the study suggests that earlier intervention (i.e., shift from institutional care to foster care) leads to greater improvement in cognitive development, and perhaps that there is a "sensitive period" for cognitive development (although more research would be required to establish and elucidate it) -- but this pattern is visible owing to the "continued 'cost' to children who remained in the institution".

This was not free knowledge. Not even close.

The challenge here is whether there is an ethical way to get important scientific knowledge -- in this case, about the cognitive development of young children who are orphaned or abandoned by their parents. Such research subjects are clearly a vulnerable population. They have a lot at stake in terms of the conditions they're exposed to in the research, and they are too young to render anything like informed consent, and in this case they don't even have parents who could properly consent for them by proxy.

Given this sort of pool of research subjects, there is a heavy burden on the researchers to ensure that the research they are undertaking is ethical, and that it does everything possible to safeguard the well-being of the subjects.

Nelson et al. address their efforts in this regard [4]:

At the outset of our study, we implemented procedures to ensure its ethical integrity. A detailed description of these procedures is included in (18), but they are outlined here. First, our study was initiated at the invitation of the then-secretary of state for child protection in Romania and was approved by the local commissions on child protection in Bucharest, the Romanian ministry of health, and, in 2002, by an ad hoc ethics committee comprising appointees from several government and Bucharest University academic departments. It was therefore done with the participation and approval of local authorities.

That the Romanian authorities (and academics) approved the protocol indicates that it fits reasonably well within their understanding of what constitutes ethical treatment of human research research subjects. That the secretary of state for child protection in Romania asked the researchers to do the research further suggests that these researchers weren't just shopping for a country with lax ethical standards in which to do a research project on which they had already set their hearts. In other words, this is less a situation of "outsourcing" research to than one where U.S. researchers are being asked to study a question in a developing country by people in that country who think such research can be of use locally (more on this point below).

Second, the institutional review boards (IRBs) of the home institutions of the three principal investigators (the University of Minnesota, Tulane University, and the University of Maryland) approved the project.

The idea here is that the IRBs would not approve the protocol if it didn't fit reasonably well with the standards of ethical research with human subjects that are recognized in the U.S. These IRBs could be expected not to approve the protocol if it seemed that the researchers were doing a study in Romania in order to avoid having to work within the ethical standards imposed on research in the U.S.

Third, we implemented a policy of noninterference with placement of children in both groups into alternative family care environments, leaving those decisions to Romanian child protection authorities (according to Romanian law). The only exception to the noninterference rule was that we ensured that no child placed in foster care as part of the randomization process would ever be returned to an institution.

Although children in the study were randomly assigned to stay in institutional care or to be moved to foster care, the researchers gave the local child protection authorities the last word as to whether and when to move any of the subjects in the control group (that is, the kids in the institutional setting) out of an institutional setting. Even though more kids in the control group would give "cleaner" data for a comparison, considerations of the welfare of the individual kids -- as made by the child protection authorities, not the researchers -- were prioritized. This is supposed to reduce conflicts of interest (e.g., a researcher who wants clearer results might be less quick to act on information suggesting that a kid needed to be moved out of the control group).

The decision that no kid would be moved from foster care back to institutional care is an interesting one. It underlines the researchers' hunch that foster care is superior (from the point of view of cognitive development and/or the welfare of the kids more generally). Also, it suggests that the researchers recognize a strong responsibility for the welfare of the study participants even in potential circumstances where the local authorities might be viewed as acting against it. (This may well complicate the opening assumption that staying within the local ethical standards marks this as an ethical research project.)

Fourth, after our preliminary results began to suggest positive benefits of foster care, we held a press conference to announce the results of our investigation. Key ministries in the Romanian government were invited to attend and sent representatives to this meeting. The then-U.S. ambassador to Romania (who was briefed in advance about our findings) gave the opening remarks at the conference.

Again, the idea here is that the researchers are prioritizing the welfare of the kids in the study, not the number of data points collected in the study. Presumably, in light of the preliminary results, Romanian child protection authorities could halt the study and put all the kids in foster care -- at least, assuming they had the resources to do so.

Fifth, although the usefulness of clinical equipoise is controversial among bioethicists, a reasonable interpretation of clinical equipoise supports the research design in this project. Clinical equipoise is the notion that there must be uncertainty in the expert community about the relative merits of experimental and control interventions such that no subject should be randomized to an intervention known to be inferior to the standard of care. Because of the uncertainty in the results of prior research, it had not been established unequivocally that foster care was superior to institutionalized care across all domains of functioning, especially with respect to how young children initially placed in institutional care function when placed in foster care as compared with children who remain in the institutional setting. Moreover, at the start of our study there was uncertainty about the relative merits of institutional and foster care in the Romanian child welfare community, with a historical bias in favor of institutional care. Additionally, given that the study was invited by Romanian authorities and conducted there, with the aim of guiding child welfare policy in Romania, it made sense to assess the study in view of the local standard of care, which was institutional care. The study also presented no more than minimal risk to the subjects; specifically, children assigned to the IG** continued to receive the same care as if the study had not been conducted, and the measures we used have all been used for many years in developmental science research.

"Clinical equipoise" is the tricky issue here. The reason to do a piece of research is to gain knowledge where there is currently uncertainty. Doing an experiment in conditions where you have a very clear reason to expect a particular result isn't "research," at least not research that can be expected to produce new knowledge. Rather, such an experiment is usually described as an unnecessary duplication (since it will almost certainly tell us something we already knew) -- or a student lab.

In research with human subjects, though, if there is any kind of risk to the subjects from participating in the research, there must be some expected benefit to the research in the form of new (and useful) information generated in the study. If the study promises to tell us nothing but what we already knew, you're exposing subjects to risk for no good reason -- and that, clearly, would be unethical.

Nelson et al. point out that:

• The previous research in this area hadn't yielded clear results. Indeed, they write:

  This literature on the effects of early institutional care suffers from methodological limitations, particularly selection bias: In nonrandomized studies, a biased sample (e.g., healthier children or more psychologically competent children) may be adopted into families while others remain in institutions. These nonrandom factors make it difficult to attribute differences in behavioral characteristics of children reared in or out of institutional settings to the different environments in which the children were reared. [5]

  In other words, while there may have been anecdotal data suggesting that foster care would be better and institutional care would be worse for a child's cognitive development, this didn't yet meet the burden of proof for scientific knowledge. A study like the one conducted here would be necessary to meet that burden of proof, and until that burden of proof was met, what the scientists had was a promising hunch.
  

• Foster care as done in the U.S. and foster care as done in Romania (where, essentially, it wasn't done prior to this research) might turn out to have really different effects. (See the last bullet with respect to the status of pre-existing hunches in the absence of good data.)
  
• Prior research didn't ask (or answer) the question of how the age of the child would influence the effect of foster care or institutional care on cognitive development. Again, the answer to this question couldn't be known without the research.
  
• Before the initiation of this research project, the presumption among child protection authorities in Romania was in favor of institutional care. This is what the abandoned children were already getting, and what they likely would continue to get if the research had not been conducted. In these circumstances, the "intervention" of being moved to foster care was expected to leave study participants no worse off than if they were not participants in the study, and being in the control group (in institutional care) amounted to staying in the institution where the kids already were plus having their cognitive functioning assessed with methods that (presumably) are just fine with American parents who sign their babies up for child development research.
  

The researchers emphasize here that this study is not an unnecessary duplication, and that the kids who were research subjects were not subject to anything more than "minimal risk" in order to obtain the new knowledge.

Lastly, we were aware from the outset of the policy implications of our work, and as the study progressed we made our results available to government officials and child protection professionals. Indeed, several years after our study began, the Romanian government passed a law that prohibits institutionalizing children less than 2 years old, unless the child is severely handicapped.

This part is underlining the value of the knowledge the researchers expected this study to produce -- not just to satisfy their scientific curiosity, but to help guide actual policy affecting the lives of actual children in Romania. The fact that this knowledge can be expected to help produce a societal good, and indeed that such a societal good might be difficult or impossible to secure without an accurate scientific understanding of what is harmful and what is beneficial to these kids, is supposed to make the "minimal risk" the research participants face more acceptable than it would be if the study were simply satisfying scientific curiosity.

Over the course of the study, there were instances of change in actual living arrangements and, in some cases, subject attrition. For example, of the 68 children who composed the IG, only 20 remained in institutions at the 54-month assessment. Seventeen children were lost to attrition. Of these, 9 were adopted or returned to their biological families, and their families decided not to continue participating in the study. Other children who remained in the study changed status: 2 children were adopted, 18 were placed in government foster care (which was not available at the onset of the study), 9 were reintegrated into their biological families, and 2 were placed in families with extended family members. Although some children changed their group assignment, an intent-to-treat approach was followed, whereby all analyses we report are based on children's original group assignment. Thus, our findings represent a conservative estimate of the response to intervention.

Here, I think the authors are trying to say that they are consciously striving not to draw conclusions that are stronger than the data warrant, despite the fact that the data depart from the "ideal" comparison they might have gotten by rigidly keeping all the kids who started in the control group in the control group and all the kids who started in the intervention group in the intervention group -- something, clearly, that could only be done by putting the data they were after above the welfare of the kids they were studying. Even prioritizing the welfare of the subjects, they are saying that they have data that lets them draw scientifically meaningful conclusions. To the extent that the kids they were studying took on extra risk of harm by being part of the study, then, this cost that they bore was not wasted because new (and useful) knowledge was produced.

To recap, Nelson et al. are saying that their research passes ethical muster because it conforms to Romanian ethical norms and to the ethical norms recognized by IRBs at American universities; because it seeks to answer an important scientific question that was not yet answered and whose answer is expected to produce real societal benefits, especially for the population of whom the research subjects are a part; because it exposes the research subjects to no significant additional harm relative to what their situation would be if they were not participants in the study; and because the involvement of the Romanian child protection authorities in monitoring the research and moving kids out of it if considerations of their welfare warrant it gives the kids something like an advocate without a bias in favor of continuing the research.

In my upcoming post on the Millum and Emanuel paper, we'll examine the bioethicists' explanation for why, given our prevailing guidelines for ethical research with human subjects, the Nelson et al. study doesn't amount to exploitation of the abandoned children they studied. We'll also note some lingering worries, and consider whether these are unavoidable when scientific research goes global.
________
*It's worth noting that "families of origin" are not foster families. What is known about families of origin may or may not apply to foster families, and scrupulous scientists may demand evidence to underwrite lumping these two environments together.

**IG = institutional group (the control group). In this study, the intervention group was the group of children randomly assigned to foster care. (Because both intervention and institution start with "I", it's easy to get mixed up by this abbreviation if you're reading quickly.)

-------
[1] Charles A. Nelson, III, Charles H. Zeanah, Nathan A. Fox, Peter J. Marshall, Anna T. Smyke, and Donald Guthrie, "Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project," Science 21 December 2007: Vol. 318. no. 5858, pp. 1937 - 1940.

[2] Joseph Millum and Ezekiel J. Emanuel, "The Ethics of International Research with Abandoned Children," Science 21 December 2007: Vol. 318. no. 5858, pp. 1874 - 1875.

[3] Nelson et al., p. 1940.

[4] Nelson et al., p. 1939.

[5] Nelson et al., p. 1938.

Nelson, C.A., Zeanah, C.H., Fox, N.A., Marshall, P.J., Smyke, A.T., Guthrie, D. (2007). Cognitive Recovery in Socially Deprived Young Children: The Bucharest Early Intervention Project. Science, 318(5858), 1937-1940. DOI: 10.1126/science.1143921