IRB Information/Documentation: NWDA

Protocol sections

 

1.         Brief Description:  A brief description (one paragraph) of the significance and objectives of this project in lay terms.  Include the intended use for this research (e.g., thesis, publication, presentation, program evaluation, etc.).  Include the aims and hypotheses to be tested, if any.

This is a usability study of the Northwest Digital Archives (NWDA) website (http://nwda.wsulibs.wsu.edu). This usability study involves learning about user’s perceptions and observing user’s reactions to the finding aids (guides to archival collections). The goal of the usability study is gather feedback from participants to inform improvements to the NWDA site and finding aids included. This research will be used by both the NWDA to improve its website, and by the researcher as part of a conference presentation.

2.         Background and Significance:  A brief description that should support the objectives of the research

The Northwest Digital Archives (NWDA) provides enhanced access to archival and manuscript materials in Alaska, Idaho, Montana, Oregon, and Washington through a union database of Encoded Archival Description (EAD) finding aids. The NWDA was established in June 2002 through grants from the National Endowment for the Humanities (NEH) and the National Historical Publications and Records Commission (NHPRC). With additional support from NEH and NHPRC, Phase II began in July 2005 and will continue through June 2007. This round of usability testing marks the third round of testing for this phase of the grant; I will look specifically at how users interact with and use the finding aids on the NWDA website.

The research findings will also be used as part of a presentation I am giving at the Society for American Archivists conference on how much detail users need about archival collections to locate relevant information. It is my hypothesis that the fewer details archivists provide, the harder it is to find relevant information. 

3.         Methods and Procedures:  A full description of the methods and procedures to be followed during this research project.  Be specific.  Outline the chronological sequence of events involving human participants, beginning with recruitment.  Include a step-by-step explanation of each procedure that will be followed.  Clearly identify which are the experimental activities/evaluation activities.  Include the estimated time commitment for a participant’s involvement.  Attach written copies of all recruitment materials (e.g., verbal statements, email messages, letters, posters or flyers). 

 

A flyer announcing the testing will be placed at the Archives Reference desk in The Valley Library. The flyer will include a brief description of the project and the principle researcher’s contact information. Interested participants will be given a recruitment form, which will ask for the participant’s name, email, and experience level with historical research. Participants will be provided with an envelope to seal the application form in, and the envelope will be secured in a box in the researcher’s locked office. Only the principle researcher will have access to the information on the form; however, Archives’ staff members will be asked to distribute forms to interested participants. While the Archives’ staff is professionally trained to maintain the confidentiality of records and information, they will not have access to the completed forms for this study.

 

Based on the experience level reported by the participant, they will be placed in one of two groups: experienced or not experienced. The principle researcher will contact the participant via email to schedule a testing time. The principle researcher’s OSU email will be used for communication. Each subject will be assigned an identification code (their initials), and then the recruitment form will be destroyed. Once the times for testing have been determined, the email communication will be permanently deleted. Any subsequent email that might be sent from participants will be answered, and then permanently deleted.

5-7 participants will be recruited for each groups; usability literature states that problem areas in the product being tested should reveal themselves within 5-10 user tests.

Participants will use a laptop computer in a conference room on the third floor of The Valley Library for this testing; this room is near the OSU Archives. This computer will have an internet connection, but will not have the capability of recording the participants’ responses. The principle researcher will record participants’ responses on an audio tape. The study will be conducted by providing the user with a set of questions about the NWDA site and documenting their responses as they navigate the NWDA site. While the questions and activities have been predetermined, the test will also involve questions and directions from the principle researcher to both solicit spontaneous feedback and guide the participant through the test.

 

Several questions involve asking the user to compare the NWDA to other union databases; finding aids from the Online Archive of California (http://www.oac.cdlib.org/) and the A2A database (http://www.a2a.org.uk/) will be used for comparison. Participants will be asked to find the same information in both databases; based on their search, they will be asked for feedback on the presentation of the information, ease of use, and any other reactions they would like to share. Predictably, the information provided by both databases will be presented in a different manner, and how the user responds to these differences is important for the future development of the NWDA website.

 

Each participant will do one session of testing. Each session will take approximately 1 hour.

The results will be compiled for each user, and then an overall summary of recommendations for improvements to the site will be prepared. Once this has been done, all recorded data will be destroyed.

 

4.         Risks/Benefit Assessment:

•           Risks - A description of the risks (if any) to the participants involved in this research and how those risks will be minimized.  Indicate if there are no foreseeable risks to participants.  Include a thorough description of how risks and discomforts will be minimized (per 45 CFR 46.111 (a) (1 and 2), 21 CFR 56.111(a) (1 and 2)).  If vulnerable populations are to be included (e.g., children, pregnant women, prisoners or individuals with cognitive impairment), detail the special precautions will be used to minimize risks to these subjects (45 CFR 46 Subparts B-D).  Also identify what available alternatives the person has if he/she chooses not to participate in the study. 

 

There are minimal risks in this study. There is a possibility that participants may feel uncomfortable while they are observed and their actions recorded for 60 minutes. Participants are free to end the test at any time, without any questions being asked.

 

•           Benefits - A description of the benefits (if any) to the participants involved in this research.  Indicate if there are no direct benefits to participants.

 

There are no direct benefits to the participants.

 

Usability testing will provide feedback to the administrators of the NWDA website about how researchers use the collection guides. The general community of users will benefit through subsequent redesign of and improvements to the NWDA website. The scholarly and archival community will benefit from more focused study of web-based search and retrieval, as well as a practical study of researcher needs, desires, and expectations for accessing archival and manuscript materials. The general undergraduate and university community would benefit from the results and improvements implemented from the results of this testing.

 

•           Conclusion - A summary statement of the risk/benefit.

Since there are minimal risks involved with this study, the benefits of this testing outweigh any risks.

5.         Participant Population: 

•           The approximate number of participants to be recruited over the life of the study (includes controls and screening failures).

 

There will be between 10-14 participants in this study. They will be divided into two separate groups of 5-7 based on their own reported experience with archival research. 

 

•           Include a description of participant characteristics.  Please indicate if the participant population is not restricted to any gender or ethnic group; a justification is required if the participant population is restricted to a unique population (e.g., specific age range, gender, ethnic group).

Participant population is not restricted to any gender, race, or ethnic group; however, the participation will be restricted to those aged 18 and older. This age restriction and the need for proof of identification will be included on both the promotional flyer and recruitment form.

 

10-14 participants will be recruited for the study.

 

This study will compare novice and experienced users; therefore, the participant group will be divided into two groups. Placement in each group will be determined by the participants themselves, based on their own reported experience and comfort level.

•           Describe method of selection.

The first 14 researchers who submit recruitment forms will be contacted to confirm their interest, level of experience, and availability.

6.         Subject Identification and Recruitment:  Describe recruitment procedures; including how you will ensure that subject selection is equitable and all relevant demographic groups have access to study participation (per 45 CFR 46.111(a)(3), 21 CFR 56.111(a)(3)). 

 

The participants will be recruited by posting a flyer with the researcher’s contact information at the Archives Reference desk in The Valley Library; please see the answer for question 3 (Methods and Procedures) for details about the initial recruitment contact. Participants will be selected based on a first-come, first-serve basis and will set a study day and time based on participants’ availability. All participants will be asked to bring identification cards to the session so I can ensure that they are over 18.

 

There will be two separate groups of participants; placement in each group will be based on participants’ reported experience and comfort level with archival research. 

Since the participant population for this test is quite small (10-14 subjects) and comprised of general users, sampling techniques or mass-communication recruitment tools will not be used.

7.         Compensation:  Include a description of any compensation that will be given to participants.  Include details concerning the conditions under which research participants would receive partial payment or no payment at all (e.g., withdrawing early from the study).

Participants will receive a $20 gift certificate to Amazon.com. This certificate will be presented even if the participant withdraws early from the study.

 

8.         Informed Consent Process:  A description of the methods and the process by which informed consent will be obtained and documented.  Informed consent documents are to be on OSU department letterhead and formatted using the OSU informed consent template (assent and consent document templates are available at http://oregonstate.edu/research/opsprc/rc/humansubjects.htm).

•           If relevant, please explain how the subject’s continued consent to participate in this research will be assessed.

•           If requesting a waiver of documentation of informed consent, justify.

•           If requesting a waiver of informed consent, justify.

•           If subjects who are not competent to give consent are included, describe how competency will be assessed.

Prospective participants will be given the opportunity, via email, to ask me questions and to have their questions answered before they are asked to make a decision about their role in the study. This correspondence will be deleted once the questions have been answered.

 

Before they arrive for the testing, participants will be emailed the consent form. The form is attached to this application. Participants will be asked to read the form, but they will not be required to print it and bring it to the test; a paper copy will be provided at the time of the test. They must agree to the consent form before any testing begins. When participants arrive at the Archives Reference Desk for the test, I will ask them to show me identification to verify that they are over 18. I will show them to the conference room where the testing will take place, explain the process for the testing, explain the consent process, ask for any questions, and then will ask them to sign the consent form. I will provide them with a copy the signed form for their records.

 

9.         Anonymity or Confidentiality:  A description of the procedures that will be followed to ensure the anonymity or confidentiality of the participants’ identity and how the information will be maintained (e.g., in a secure location).  Describe intended coding process, if applicable.

During this study, participants' identity will be kept strictly confidential. If the results of this study are written in a scientific journal or presented at a conference, names will not be used. Names will be excluded from the transcription; instead, I will use the participants’ first and last initials for identification. Following transcription, the tape will be erased.

10.       Audio-taping

One aspect of this study involves making audio recordings. I will be recording the testing so I can transcribe the session after it is finished. Tapes will be kept in my office, which is always locked in my absence; as added security, my office is in the OSU Archives Workroom, which is a locked area. I will be the only one to have access to the tapes; however, I will share the transcript with the Usability Working Group of the NWDA. The participants’ names will be excluded from the transcription; instead, I will use their first and last initials for identification. I will erase the tapes used for recording the testing session once transcription is complete. The transcripts will be retained for the duration of this project; they will be stored in my office.