Psychiatry and Behavioral Sciences Scientific Culture & Accountability Plan

The Department of Psychiatry and Behavioral Sciences is committed to creating and maintaining a scientific environment that upholds the highest standards of scientific integrity.  The leadership of the department acknowledges that scientific integrity characterizes both individual researchers and the institutions in which they work. For scientists, integrity embodies the individual’s commitment to intellectual honesty and personal responsibility. It is these aspects of moral character and experiences that serve as the basic building blocks for scientific integrity. 

The School of Medicine has opted to allow individual departments to formulate their own policies and procedures related to scientific accountability and integrity. For this reason, the Department of Psychiatry and Behavioral Sciences has taken strides to put in place policies and procedures that reflect the highest professional conduct expected of all research faculty and staff.  The aim of the Department’s policies and procedures is to promote a culture in which scientific results are critically reviewed and to establish a process that clearly delineates the process for ensuring the accountability for data integrity. The Department’s policies also allow for concerns about data integrity to be raised without hesitation and provide mechanisms by which these concerns are addressed fairly and expeditiously.

In the Department of Psychiatry and Behavioral Sciences, we recognize that these objectives require the active participation of all parties from trainees to division chiefs, and ultimately, to the department chair. In recent years, events at Duke and elsewhere have underscored the ongoing need for explicit processes and mechanisms to manage scientific research data in a responsible manner. At the same time, it is acknowledged that most investigators have received little, if any, training or guidance in this area, and thus the following policies and procedures represent important guidelines for researchers regardless of prior training or research experience.

The Department of Psychiatry and Behavioral Sciences’ Scientific Culture & Accountability Plan (SCAP) sets expectations and provides recommendations for how department administrators, division leaders, and individual investigators can guarantee the responsible management and critical review of scientific data.

The Department of Psychiatry and Behavioral Sciences SCAP reflects these important principles:

  1. We foster an environment where scientific integrity is the highest priority.
  2. We emphasize high‑quality, reproducible data and results.
  3. We value constructive critiques of research.
  4. We encourage open discussion of any concerns regarding research conduct or integrity.

Each member of the Department of Psychiatry and Behavioral Sciences — faculty, trainees, staff and administrators — is expected to reflect and pursue these values. Ours is a shared commitment to the highest standards of scientific activity. Compliance with established policies that result in research excellence is our fundamental goal.

I. Recommended practices for Improving Scientific Accountability within  investigator’s laboratory

The principles described in this plan provide guidance for ensuring the integrity of research data and for maintaining compliance with federal and institutional regulations and policies as well as those areas outlined in the Department’s Scientific Culture and Accountability Plan. It is expected that all researchers will discuss these expectations with their research team, and develop explicit processes within the lab to monitor compliance with these policies.

It is expected that researchers will review the following steps in order to comply with these policies at the annual review meeting with division heads.

  • As a principle investigator, you set the example for your team through honest and open discussion of results and through an emphasis on scientific integrity and data quality over positive results. Do not, in any way, encourage or put pressure on lab personnel to obtain specific results. Make it clear at all times that the highest priority for researchers is to obtain the true result of studies, irrespective of the effect that such a result may have on the overall project, grant submission, or manuscript. Make it clear that as lead researcher, you have a zero-tolerance policy with respect to data manipulation, alteration, or falsification.
  • High-quality research begins with careful planning and study design. Engage appropriate collaborators, statisticians, and other relevant team members for constructive input before actual experiments or clinical studies begin. Having well-defined study goals protects against fraud and improves the quality of results. Frame research questions in ways that allow negative, neutral and positive results to be interesting and useful to the line of inquiry; that is, refrain from expectations that one type of result is more valuable than another. The analytic strategy should include the use of multiple statistical approaches and various techniques so as to fully examine the reproducibility and comparability of the results from your experiments.
  • Most of us rely on our trust of and judgment in others to ensure the integrity of our data. However, this alone is not sufficient. Examples exist where even the most seemingly trustworthy people have manipulated data. While investigators should continue to put faith in others, their research collaborators, and support staff, researchers should reinforce this with specific practices and institute processes to ensure that your data is managed responsibly. One recommended method is to cross-train lab personnel so that one person can independently verify the results of another, and no one person is alone in providing data or analysis.
  • Implement a policy of best practices with respect to research records, including basic laboratory notebooks or clinical research records. When possible, utilize electronic recording solutions that automatically record date and timestamps of entries and data changes (e.g. REDCap). Ensure that entries are being made in a way that conforms to lab policies. Make these policies clear to all lab personnel and enforce them. We recommend that lab notebooks are reviewed at meetings when the data is discussed. Additionally, consider performing periodic audits of laboratory notebooks to ensure that a third-party reviewer would be satisfied with the level of documentation provided for an experiment. With regards to clinical research, maintain appropriate documents in accordance with department policies, IRB requirements, and appropriate state and federal regulations. Consider the implementation of competency training specific to individual lab or clinical research tasks.
  • Rather than create your own individual clinical research or laboratory database, leverage institutional resources such as the Duke Biobank, REDCap databases, Pedigene or other similar centralized infrastructure that limits users, directly obtains input from source elements, and tracks all changes in the data elements or samples.
  • For all data that are generated by instruments (e.g., plate readers, flow cytometers) require personnel to include the specific instrument, its location, and the date and time the analysis was performed in their lab notebook. This will make it easier to match the lab notebook with instrument-generated raw data. In addition, conduct frequent monitoring, calibration, and validation of all laboratory equipment.
  • For any instrument-generated data, implement a procedure by which the raw instrument-generated data are permanently archived in a secure location that can only be accessed by the principle investigator and/or a specific delegate. This may include digital data transferred to a secure server, data burned to compact disks (CDs), or hard copy printouts archived in a place other than individual lab notebooks. Ideally, these data would be stored in a read-only format that could not be altered once it is deposited. It may be that researchers within the Department of Psychiatry and Behavioral Science have server space available. For this situation, it is expected that the server(s) is maintained in a manner that meets or exceeds institutional and departmental policies. The U.S. Department of Health and Human Services requires that all project data be retained for at least 3 years after the funding period ends.
  • Develop a plan with collaborators to ensure data integrity. For example, the Principle Investigator may request a copy of the raw data generated by project collaborators for archiving in your own lab. Similar to data analysis performed on the Principle Investigator’s laboratory-generated data, when possible, perform an independent analysis of data generated by collaborators to verify accuracy.
  • To the extent possible, independently replicate study results. For example, have different individuals perform experimental procedures (such as treating groups of mice with a drug) and experimental readouts (such as determining the phenotype of the treated mice). It is also good practice to require that research personnel assaying readouts be blinded to the experimental groups.
  • The Principle Investigator or a designate should re-analyze all critical data such as those included in grant or manuscript submissions. It is encouraged that these analyses be conducted on the archived raw data and not on secondary data sets derived from the source information. Principle Investigators need to consider including a person outside of the laboratory with the appropriate expertise to conduct the research procedures including manipulations. For clinical and translational studies, this can be accomplished by having study data independently analyzed by a statistician separate from the investigative team. For human research projects, a masked or blinded individual may be needed to perform certain study activities.
  • Develop a process (e.g. PI notebook) to document critical results, the date the principle investigator learned of them, principle investigator’s interpretations of those results, and conclusions or discoveries that the results imply. In addition, the research staff should document procedures taken to confirm the validity of the results, such as the Principle Investigator’s review of the raw data and re-analysis and conclusions. These records will allow the principle investigator to document the intellectual progress of specific projects, to develop future hypotheses or research plans, and if needed, to support intellectual property claims. Furthermore, if questions of data integrity were to arise, such a notebook would serve to document that you verified all critical studies to the full extent possible.
  • All individuals engaged in research are required to complete online training modules, similar to those required by Institutional Animal Care and Use Committee and the Institutional Review Board, that emphasizes these principles. The Department and SOM will collaborate to develop these new modules. In addition, individuals should take advantage of programs offered through the School of Medicine that are designed to address research integrity. Additional competency training for investigators and their research staff involved in human research can include the current core curriculum from Duke Office of Clinical Research, i.e. Informed Consent Process, Study Documentation, Data Integrity and Security, and annual Human Subject Research Overview for clinical investigators. At a faculty level, participation in institutional programs such as Leadership Education Advancement Development Endowment and Resources (LEADER) will provide an opportunity for further education in research management and oversight.
  • If anyone has concerns about the integrity of someone’s data, be it a member of the research team or someone else’s, this individual(s) should feel comfortable voicing their concerns. This is true whether the individual thinks a certain analytic method needs to be better validated or if scientific misconduct is suspected.
  • Raising concerns about data integrity is not the same thing as accusing someone of scientific misconduct. The Department of Psychiatry and Behavioral Science strives to uphold a culture in which all aspects of scientific findings are critically reviewed. This includes all steps in the scientific process, from study design to data acquisition to methods of analysis to the formulation of conclusions. Raising and responding to questions about data integrity should be a routine part of the critical review process – in other words, it need not be reserved solely for cases of suspected scientific misconduct. It is through this process that we all can work together to ensure the highest possible quality of science at Duke.
  • Have mechanisms in place within the research laboratory or research team to prevent misconduct and to address research-related complaints. The procedures need to emphasize a no-tolerance policy related to (1) falsifying data, (2) deceptive advertising, and (3) enrollment of subjects not qualified to be in studies.
  • If the laboratory engages in translational or 'omics research, it is strongly recommended that biannual discussions be held with the staff about the challenges inherent to the research. Review existing resources and discuss new resources available across Duke, especially services to make the research more reliable and transparent.

II. Departmental efforts to promote a Culture of Scientific Accountability.

The Department of Psychiatry and Behavioral Science leadership will take steps to support, guide, and ensure that a culture of scientific integrity is upheld, including the actions listed below.

  • Outline a chain of persons available to address research integrity concerns. These individuals will be available to assist faculty and staff within the divisions and will be ready to address potential concerns raised from outside the division. Concerns should be raised and addressed initially to a division chief, but in case of real or perceived conflict of interest, concerns may be raised to a departmental vice chairs or to the chair.
  • Work with the School of Medicine to develop institutional policies and modules in training of responsible conduct of research and ensuring data integrity.
  • Work with Centers (e.g., Brain Imaging and Analysis Center (BIAC)) or Institutes (e.g., Duke Global Health) in which Psychiatry faculty resides to ensure alignment and integration with their programs of scientific accountability.
  • Develop mentorship programs for new investigators/junior faculty that includes an emphasis on the outlined principles. Strive to attain a culture that values and understands that adhering to the Scientific Culture and Accountability principles is relevant to how the Department provides patient care and to the conduct of future studies.
  • Conducting quality assurance (QA) reviews to allow investigators to explain and demonstrate their processes and procedures for data integrity and analysis confirmation. Allow this process to be a learning experience for the investigator with the outcomes to be shared, as appropriate, with members of the division and department.
  • Confirm that every member of the clinical research staff completes their human subjects research and ethics training coursework and required recertification, currently mandated by the School of Medicine.
  • In cooperation with the Psychiatry Clinical Research Unit (CRU), have investigators with active IRB protocol(s) to complete role-based training in Informed Consent Process, Data Integrity and Security, and Study Documentation.
  • Develop and refine educational modules based on research staff requests and findings from institutional audits, departmental QA reviews, and IRB reviews, including a new series of “How To” vignettes as well as workshops and financial management training.
  • With the involvement of the Division Director, develop annual competency evaluation forms to evaluate specific skills required for research roles.
  • Promote the sharing of best practices regarding data integrity through a central resource of documents and materials available to Psychiatry faculty and trainees. Provide software solutions, analytical support and other resources as appropriate.
  • Validate the Psychiatry Clinical Research Unit’s research data security plan in the Institutional Review Board application.
  • Educate the faculty and staff of the Department about available resources and reporting mechanisms for scientific accountability, such as:
  • Online Learning Tool for Research Integrity and Image Processing (http://ori.hhs.gov/education/products/RIandImages/default.html)
  • The Compliance and Fraud Hotline: To anonymously report a suspected compliance violation or concern, the Compliance and Fraud Hotline at Duke is at 800-849-9793.

Recommended safeguards to ensure data integrity within your division

The Department of Psychiatry and Behavioral Science believes that the proper accountable unit for ensuring data integrity resides at the level of the division. For this reason, divisions within the Department of Psychiatry and Behavioral Science are required to follow the key principles and recommended steps as outlined in the Department’s Scientific Culture & Accountability Plan. Although this document includes the best practices with regards to scientific accountability, it is also recognized that within each division not all steps, or the same set of recommended activities, may be appropriate. It may be the case that each division establishes multiple accountable units to better interface with specific groups of investigators. Ultimately, the Department requires that the division directors maintain evidence of compliance with the Scientific Culture and Accountability Plan on behalf of their divisions, as suggested in the following:

  • Divisions must ensure that divisional investigators and personnel engaged in scientific research complete all required institutional training modules in responsible data management. These will be developed and monitored for compliance through central School of Medicine resources (e.g., Duke Office of Clinical Research), similar to the Institutional Review Board modules required for studies involving human subjects.
  • Divisions must ensure that divisional investigators implement and maintain policies for responsible data management within their labs or research groups. Divisions must also establish mechanisms by which the validity and integrity of critical data generated by every division investigator can be confirmed. These mechanisms should not place an undue burden on the investigators or the division staff but should ensure that investigators are adhering to policies of responsible data management. It is recommended that these policies are reviewed at least once a year by the division director, an appropriate representative of the division director, or the individual directing the accountability research unit. For example you may specifically review and assess compliance at an individual level with all the points in Step 1 as part of a yearly meeting with each faculty member in your division.
  • Divisions must ensure that investigators regularly present their research findings to other investigators outside their own lab or research group in a forum that allows open and critical discussion of the data and its analysis. Examples include participation in other lab meetings, regularly schedule multi-disciplinary group meetings thematically organized around common research interests, or a divisional work-in-progress series. In order to monitor compliance, we suggest maintaining log sheets of faculty participation in these conferences.
  • Review and follow best practices for data integrity and manuscript preparation as required by the top journals in your field(s).
  • Division leaders are expected to confirm that each investigator is aware of the various school, department and division resources that support researchers and their activities, including access to statistician input and review of study design, analysis and publication.
  • Divisions should require that faculty and staff involved in human research complete the Data Integrity and Security class presented by DOCR, as appropriate.
  • Explore ways for lab personnel to report to more than one investigator.

In addressing the element described in this Scientific Culture and Accountability Plan: Faculty development relies on the cohesiveness and structure of the division.  Divisional meetings provide investigators the opportunity for critical interactions between investigators with shared scientific interests. The division unit, with its smaller number of faculty members, is conducive for faculty to have a greater opportunity to express their opinions, something that is challenging in larger departmental meetings. It is also acknowledged that certain issues may have greater relevance to faculty at different stages of their research careers.  In an attempt to addresses such differences the following actions are proposed:

Junior Faculty Issues:

On a bi-yearly basis, division heads will hold a special division meeting for junior faculty so that concerns and issues faced during the early stages of a research career are discussed. In the case that the issue(s) involve concerns with the division head, the Director of the Clinical Research Unit (CRU) or the Vice Chair of Faculty will lead these discussions at the bi-yearly meeting. At this meeting, the emphasis is placed on maintaining research integrity and its various components as key elements with the over-all goal of fortifying the scientific culture of the department.  To facilitate discussion, division heads and junior faculty will be directed to the departmental webpage that will contain content useful in stimulating discussions on such topics as research misconduct, data-related misconduct, and publication-related misconduct. As research scientist and health providers, we acknowledge that such topic areas are rarely, if ever, discussed during periods of training or among the faculty. Thus, the goal of this forum is to introduce a mechanism that allows for honest and safe discussions on aspects of research integrity that are infrequently addressed. 

We also recognize that faculty members are subject to pressures that are unavoidable aspects of the process employed in promotion and advancement. It is also acknowledged that junior faculty may face additional pressures. Discussions of these aspects may best be done within a meeting of junior faculty in the absence of senior leadership.  It will be the responsibility of the division head to provide appropriate administrational support so that such meeting can occur. 

Senior Faculty Issues:

While the elements described in the Scientific and Culture Accountability Plan are applicable to all research faculty, the Department of Psychiatry and Behavioral Sciences is cognizant of issues specific to senior faculty that are critical in maintaining the highest standards of accountability and promoting the scientific culture in the department.  One specific issue faced by senior researchers is mentoring. Being a research mentor carries a huge responsibility. It is common knowledge that in today's publish-or-perish world principal investigators (PI) feel pressures to find positive results, publish papers and get new funding. This may leave little time for PIs to invest the time to provide the necessary mentoring junior faculty, trainees, and research staff, fellows particularly in laboratories that include of several post-docs, graduate students, and/or research assistants. The little time a PI may have will likely go towards providing the skills to perform experiments and publish.  

Moreover, a mentor's job does not stop at skill training. A mentor acts as a role model for engaging in responsible research practices and guiding his/her students. Graduate and medical students, residents, and post-doctoral fellows should know that they can go to their mentors with questions regarding research integrity. It will be the task of the division chiefs to provide guidance for senior faculty to ensure that the mentees receive the best training possible. With the support of departmental leadership, special meetings will be held to address such questions as:

  • How do research mentors and supervisors help with research integrity?
  • What are the potential consequences if research mentors and supervisors do not give sufficient priority to research integrity?
  • What steps can be taken to support students if they do not feel comfortable discussing or reporting research integrity concerns with those they work directly with?
  • What can you do if you feel your current mentor or supervisor is not giving you the support you need for questions related to integrity.

To facilitate further the development of effective mentoring skills, the Department is considering the development of an evaluation tool that could be used highlight the skill set needed for effective mentoring. As envisioned, the yet-to-be developed tool would be completed by both mentor and mentee(s) so that a complete picture could be generated identifying the strengths and weaknesses of the mentor. 

Process in Place:

If an issue arises, it will be the responsibility of the division chiefs and or his/her designee (e.g., CRU Director, Vice Chair of Faculty) to assess the nature of the problem and determine if it can be addressed at the division level.  Division heads are encouraged to seek guidance from existing institutional policies covering various areas of research integrity. If the allegations are determined to involve research misconduct, the division head must promptly report to the Chair and appropriate leadership of the department.

Limitations:

The department acknowledges that any plan must be amended as faculty gain a better grasp of the challenges of creating a scientific culture with the highest level of integrity.  We are, however, limited in certain areas.  

  • No established measures for assessing integrity in the research environment exist.
  • Promulgation of and adherence to policies and procedures are necessary, but they are not sufficient means to ensure the responsible conduct of research.
  • There is a lack of evidence to definitively support any one way to approach the problem of promoting and evaluating research integrity.

While we will not be able to fully address these limitations, the CRU and Psychiatry leadership are committed to evaluations that help identify areas of strengths and weaknesses useful in guiding future iterations of the hereto plan.