Tiah’s notes:

1. Mission/purpose of the group: our charge is to test the usability of the site from a functional and aesthetic standpoint, and then make recommendations on how to improve the site for use and design.

--> What did you want when you signed up to be on this group?

--> What's going right?

--> How would you like to see the group change?

--> What is our role within the structure of the program? How much "power" do we have?

--> Does this model of combining the stylesheet & usability group work? Should these be 2 groups? Or 2 parts of one group.

--> How do we fit with the Digital Service group?

--> What are some pet projects that people would be interested in working on to keep them involved while Megan is doing UT5?

--> Laura and mission statement. Laura (lib field studies) & Tony (experience with NWDA): best practices? Ellie: study & present to us on higher level searching problems, create how-to pages? Tiah: online tutorials/ online instruction pages based on Ellie's research?

2. Who are our users?

--> On-site, through the main page: direct users: what do we know about them?

--> Off-site, through a search engine: 89% are drop in: indirect users: what do we know about them?

--> How do we know what both groups want? Are their needs different? Are their outcomes different (e.g. do they all end up at the repository?) How can this be a meaningful and realistic test?

--> Do we test internal functionality and external use/functionality?

--> What will we do with these answers? How much do we want to radically consider and reconsider how the NWDA site and finding aids are meeting (or not) user needs?

2. MEGAN: Talk about what you'd like UT5 to look like, things we discussed @ lunch.

3. Online tutorial: brainstorming session if there is time-- can be email/wiki brainstorm

LZ: Work with a group to do UT on NWDA as we determined it would be needed: it was her charge to find her own people to test, those who would use the site, do the testing? How can I be contributing and add value?

EA: Improve short comings on searching, MHS converted database that had great searching capabilities to NWDA which was not so slick. Different ways to search, different results based on search terms/strategy. Boolean terms and their function, where is the explanation of what kinds of searches work and how they work? Look at search pages how how they work: 3 search pages, which all function differently with different kinds of searching. If someone comes onto our site, how easily can they search? Instruction pages.

MF: Continue doing testing as a group as we determined it was needed, working off tests of past and thinking of new tests as well. Integrating testing design as well as finding aids themselves. Most important: most of the decisions we make should be based on testing, based on what users need and want, we are the voice of the users.

TK: Provide continuity as a resource for group members. Explore new directions, continue internal workings of site-- testing that is unresolved-- to ensure they are resolved.

Stylesheet & Usability Merger

LZ: Not practical to separate function and design, test all at once.

TK: Many times asked to be combined. Work of two groups were dependent on each other.

MF: Real question: what elements of design do we test with users and what elements can we fix just because we know there are problems. Survey of members, provide feedback, make recommendations.

LZ: Provide exec team with list of recommendations, where there is pushback, test those.

MF: Set up a Google docs/spreadsheet with problems, deadline date, progress, rank or categorized. Update the list ourselves, then discuss with the group before referring to Jodi/Al.

Project director requests:

MF: Can we gather data from PD's themselves and we can funnel them to Al/Jodi. We would know what people think needs to be changed, before they are changed without us knowing. Can be cc'd to us/TEM. Tiny details that fall through the cracks.

What does the Digital Program/Project group do?

TK: Larger issues, Orbis-Cascade wide, not just NWDA. Doing surveys, creating necessary info for IMLS planning grant to do a further study of what needs exist in terms of digital services and how NWDA can accomplish that? Assess what programs are out there and whether there is a need for assistance or Alliance-wide support.

UT5

MF: Internal search page test, external search from engine, search functionality from finding aid.

MF/EA: Make list of Ellie's problems, searching issues.

LF: Send out to everyone.

LF: Usability testing experience. Best practices for Utesting in general, more general info about user behavior/searching, look at it through the archives lens ultimately. Inform Best Practices project, inform user analysis-- e.g what are people searching for, are they getting what they want, are we providing useful research resource.

Online Tutorials: Karen Monroe: @ UC Berkeley, Best practices, how to create.

 

Laura’s notes:

1.     Mission/purpose of the group: our charge is to test the usability of the site from a functional and aesthetic standpoint, and then make recommendations on how to improve the site for use and design.

a.     Laura thought individuals in group were to recruit usability testers, but was OK not actually recruiting people to test if this was not part of individual responsibilities.

b.    Ellie wanted to improve what she sees as shortcomings on the searching. Wanted to make more obvious to people how they search and what they search. Finding divergent results depending on whether or not used certain Boolean terms and no explanation on web site of what the Boolean terms were. Hard to find what kinds of searches do what and they don’t work like Google searches. People may think there is nothing there, when something is. Interest in the search page and possibly eliminating one of three search pages and making Advanced Search the way to go. All three function differently in terms of search language. We should fix discrepencies b/w diff’t search screens. We’re not in control of the external/Google searching.

c.     Meagan wants to continue to do usability as a group as needed, working off tests from past. Excited by idea that we are integrating testing of design of web site and not just finding aids. Most important for Meagan is that all decisions we make should be based on the testing, not our personal opinions – we need to back them up with the testing. We’re the voice of what the users want and testing is how to figure that out.

d.    Tony in same boat as Tiah. Wants to continue w/ what all have mentioned already. Help fix areas of site, such as one’s Ellie mentions, that remain unresolved. Make sure those are moving along.

2.     Tiah said that one thing she and Jodi talked about what is their role/power w/I structure of program. Especially if we back it up with user feedback and experience our voice will be strong. How is model of combining usability and design group work? Pros and cons to this.

a.     Laura – makes sense to combine, do all at once.

b.    Tony – agrees w/ this. Two groups previously worked hand in hand. Asked to be combined b/c too hard to be in two separate working groups w/ work depending on the other. By second phase of grant were one group and shared report writing.

c.     Meagan – See the real question now, to what Ellie getting to, as what elements of design or usability do we test w/ users and what elements do we just say “this is a problem and we need to have fixed asap”? Laura = make a list! All – put problems, deadline date, etc.

d.    Tony = had a list we gave to Al Cornish. All reflected in testing to some degree. Things that were broken, etc., including search. List went away b/c project management site has lapsed. Yes, we need a centralized list.

e.     Tiah = even if not accessed by anyone else but this group, would be feeling of progress.

f.     Tiah = trying to figure out what happening w/ PBWiki site, but now cannot log in. Was a link that Karen sent out re: how to get to their section. Thought was more functional.

g.    Google emails = people will email Laura these and Laura will set up a Google Docs area for us. Tiah emailed Ellie the web site. If doesn’t work, Tiah said they had Wiki software at the library we could potentially use.

h.     Meagan = once we have document set up, we should not take action until we decide as a group which to take action on or save for larger testing. We should wait to take action until we collectively decide to get user feedback on.

i.      Tony = we could potentially rank them.

j.      Tiah = one thing Tony and I did was to chat over the spreadsheet for UT3. That is part of the Google Docs I think. http://docs.google.com

 

3.     Tracking on site issues. Meagan – people emailing Jodi and Al directly, not us. We should be cc’d on these requests so we know what is fixed and not, etc. Laura = yes, where is the discussion on this? Meagan = stylesheet change was major, but still a lot of tiny details that have fallen through the cracks, etc. It would be great to get everything in one place.

4.     Tiah = asking b/c next on list. How do we fit w/in Digital Service group? Who are they? Digital Projects group. They are looking at larger issues. Tony = exploring what kinds of digital service could be applied Alliance-wide, not just NWDA – allying w/ larger Alliance purpose. Creating surveys and other data for IMLS planning grants, etc. What needs exist for digital services and what role can NWDA and Alliance play to accomplish that, as a new program? Assessment of what programs out there and what services may need an Alliance-wide approach.

5.     Meagan = how do we fit w/ them? We don’t really until they are really doing work to put those projects together. Tiah = Jodi and she talked about that usability studies and user studies are not synonymous (Elizabeth Nielsen). Jodi really starting to look at users and who our users are, above and beyond usability testing. What can we gather (i.e. demographic) through Google Analytics, etc.? Usability testing very time-intensive.

6.     UT5 - Meagan will be doing UT5. UT5 will be testing (per phone call in November) that this would focus on searching – how users search; how main search page does or doesn’t work and how finding aid search does or does not work. Again, coming from Google or straight through search page on front. Solely search function stuff this time around. Ellie has done a lot of the searching on her own – could be a user and try these things we are testing. Meagan would like that information (about what is already broken) to make sure these things are tested.

a.     Ellie = some functions work well if you know which ones to search. Problem is knowing which to use in which search screens. Meagan = curious to know if users even use Boolean searching. What do users instinctually do? We’ll try to get some answers on that. Tiah = maybe Meagan could compile that. Ellie will resend list of search engine issues to Meagan. Meagan will look over list and pick Ellie’s brain on this. Ellie wants to retest some of these in a different time and place and see if things are functioning way she thinks. Ellie could also convey strategies she has seen users use, per Tony’s Firefox vs. IE analysis, etc. Note: They still have not made those corrections.

b.    We need to make priorities clearer to Jodi and Al. A centralized list will help all of us greatly.

c.     Laura = would like to test as well. Meagan = yes. Sometimes the questions are open ended and depends on research questions. Apologizing in advance if questions are vague.

d.    Tiah = something Jodi and I are talking about is best practices. Laura = interested in this in terms of best practices? Interest from w/I NWDA from Jodi and interest in larger profession for both usability/best practices for archival usability testing and what has already happened in library world. We don’t need to repeat tests already done, but stepping out would be good – we need to know generally about user behavior and expectations when use Web at all. What my (Tiah) theory is is that users are not distinguishing (80%+ coming from Google). To them, whether they get to OSU Library or some random page or a finding aid – it’s whatever is ranked up at the top. Knowing what generally our users are needing would be helpful. Based on Jodi’s last survey we know who project director’s have ranked as their user’s. I’m intrigued by the people we never talk to, who wouldn’t physically come – would never let us know they were there. Google Analytics will allow gathering some data. Knowing in general about search behavior. How do people process information? Fascinating to see how people improve through usability testing. Ties into what Ellie was talking about w/ instruction pages. What is this? How do I use it? Learning modules.

                                          i.    Best practices usability testing in general.

                                         ii.    Users’ expectations.

                                        iii.    Searching behavior in general now (those w/ research topic). Look at library literature.

                                        iv.    Tiah encouraged Laura to investigate these areas.

                                         v.    Meagan = bigger is better at this point. Having more information off the bat for all of us to take a look and then submit to Jodi at the end would be good. Tiah = desired outcome = for us to have a more general base about user searching behavior. What different groups are doing now. Being able to look at more specifically at archives users and best practices for archives usability testing and also looking at our users and whether we are meeting their needs.  Ellie = have wondered about off-site users if they find specific thing they are looking for in Google and say “Wow! This is cool” and start searching w/in NWDA. Also, how Google ranks the sites. When you get 7000+ hits and you’re not going to wade through them, how is this determined.

                                        vi.    Meagan = in terms of what happens when people drop in from Google, Tiah and I discussed at lunch a few weeks ago. We need to get some researchers doing their own research. Are they coming into Google looking for something else. Search for something they are really interested in and see reaction when they get into NWDA. Tiah = meaningful usability study, but also realistic. Behavior when it is not canned or constructed and us saying “we know what outcome is”. Laura =

contextual testing”. Tony = we did on first test. Very informative. Tiah = one thing changed in UT4 is having live link to repository encouraged people to explore more.

7.     Tiah = will complete executive summary after get student schedule done. Would like to start gathering information and doing own research on online tutorials and instruction pages. Will be informed by all these other projects. Get some text to put up. Trevor did a first stab at a tutorial that at least showed people how to search. Laura = check out Karen Munro at UC Berkeley for best practices on creating these kinds of things.

8.     Tony = question about Google metrics. Any possibility to share that info? Tiah = yes, they are still tinkering with it, but we could share it.

9.     How many going to Alaska in May (NWDA) = Meagan, Tiah and Tony will be there. Laura = please send us your notes afterwards. Tiah = we are presenting there. Meagan = questions like Google metrics we can pose during NWDA meeting. Next year will be in Portland in late April 2009. Doing with Oregon Heritage Commission. Tiah = great dream to involve historians and oral historians, etc. Extending invitations to NW historians, etc. Tribal archives and museums are having their conference in Portland as well in October. One person Tiah knows on both committees. Some very public accessible place. NW Archivists annual conference. The day before is usually for NWDA admin meeting or workshops offered. Tiah = we’re meeting in a week and a half on this.

10.  Should we meet on phone more? Laura = once a month? Tiah = phone not a perfect substitute, but better than not. Challenge of geography. Let’s make technology work for us (phone and otherwise).

11.  Tuesday afternoons generally good for most of us? Tiah = thought had desk duty next term. We could make phone conversations short, but having deadlines may help. Meagan = will do U5 in late March or early April. Schedule talk for week of April 7^th? Tentatively our next phone conference will be on Tuesday, April 8^th at 1 p.m. PST / 2 p.m. MST. Tiah will contact Jody.

 

The End