Design Assessment Methodologies

(Smaller scale and informal assessments of projects and designs)

Muddiest Point / Clearest Point

A very widely used form of assessment, this method involves giving the participants in a learning activity note-cards. Ask them to write the single point that was most unclear on one side and the single most clear point on the other.This technique is useful for getting feedback on presentation technique. Often, it is most useful to wait until you have collected cards from several sessions before tallying the responses since patterns and trends tend to be more useful than individual responses.

Strengths:

·       Easy

·       Low overhead

·       Not demanding on the audience

Concerns:

·       Need many, many cards to get useful data

·       Not in-depth or reflective

·       Can be difficult to interpret accurately

Peer Review Assessment

This is a classroom assessment technique that engages participants in the process of assessing each others' work. It is especially useful for information literacy sessions that present evaluation skills. In this technique participants create a product and then exchange them and evaluate each others' work based on a provided set of criteria. By creating a product, having it evaluated by a peer, and evaluating a peers work, learning outcomes are reinforced from several perspectives. Librarians are also able to see evaluate participants understand from several perspectives as well.

Strengths:

·       Engage participants in active evaluation process

·       Reinforces evaluation skills

Concerns:

·       Limited applicability outside the classroom

Quick Write / Journal Assessment

Free writing assessments are useful because they engage patrons in the reflective act of writing while simultaneously creating a document that librarians can read and gain insight on the patron experience. This is useful in classroom settings or any setting where an audience has a few moments to reflect on a particular aspect of library service.

Strengths:

·       Engages audience in reflective activity

·       Gives voice to thoughts surveys and forms may miss

Concerns:

·       Requires a captive audience

·       Qualitative data can be difficult to process accurately

Assessment 2.0

This technique takes advantage of the two-way communication of new web communication techniques. By giving patrons the opportunity to provide comments and feedback on our web based materials, we involve them in the design process and give them tangible evidence that the library is receptive (if not responsive) to their concerns.

Strengths:

·       Built in to blogs and wikis

·       Involves patrons in ownership of the library site

·       Builds conversation in the community

Concerns:

·       Not everyone is comfortable providing feedback or at home in the 2.0 world

·       Anonymous forums have the potential for abuse, spam, or harassing comments

"Anthropologist" Assessment

This technique involves the careful and intentional observation of patron activities as they interact with library services. This technique seeks to answer the question "what do patrons do when then use our services" and can provide a more accurate account of how patrons use the library than asking direct questions.

Strengths:

·       More reliable than self-reported claims of current or future behavior

·       Low overhead

·       Provides evidence and data to support other assessments

Concerns:

·       Subject to reviewer bias

·       Time consuming

"Snitch Assessment"

This technique involves asking an outside authority with a vested interest in the library to report on the effect of the library program or service being assessed. Parents and teachers are excellent sources of external insight while assessing our services.

Strengths:

·       May provide insight that the patron does not possess.

·       If the person queried has a personal relationship with the patron(s), they may have valuable insight into behavior and/or library use patterns.

Concerns:

·       While additional perspectives can clarify, they may also obfuscate.

·       Privacy issues may arise

·       Indirect

Program Assessment Methodologies

(Larger scale & more scientific assessment of programs and services)

Focus Groups

A small group selected from a wider population to discuss their opinions about a particular subject or area. Initially used in market research, they are currently used to gather information in many other arenas.

Strengths:

·       they can be done if there is a short time frame to gather information

·       they are good if you are unsure of what the issues are in the library

·       they are a good way to develop questions for surveys

·       To target a specific population

·       You can probe beyond the initial response to find out the “whys”

Concerns:

·       Plan ahead, it’s simple but requires some planning

·       The cost is nominal but participants need incentives to participate

·       Much rides on the skill of the facilitator

o      Time keeping

o      Group dynamics

·       Note taker and recorder

·       Transcription is tedious

Surveys

A gathering of a sample of data or opinions considered to be representative of a larger group.

Strengths:

·       Can gather information from a large number of people

·       Good for strategic planning

·       Can target specific questions

·       Can be quantified

Concerns:

·       Time consuming

·       Costs vary based on size of survey, method of administration, and type of analysis

·       Accurate questions are the key! Make it a group effort. Write and rewrite the questions, test them, and then rewrite again.

o      Paper surveys

§       Can focus on a specific area of the brick and mortar library

§       Take more staff time to administer and analyze

§       Can be entered into software for analysis

o      Electronic surveys

§       Faster to gather and analyze data

§       Can target website users, email large population

§       Take less staff time to administer and analyze.

Space study analysis

Assessing the way a particular space is used by a form of observation.

Strengths:

·       Use when you are considering rearranging the furniture, getting a new space, when there are questions you have about certain areas of the library

·       Use when you want to validate another methodology or general impressions

·       Use to see what people do, not what they think or say

Concerns:

·       Be creative

·       Use to create questions for a survey or to validate survey findings

Rubrics

A tool for assessing instruction and performance according to predetermined expectations and criteria. (Taggart, et al, 1998)

Strengths:

·       Adaptable to ACRL standards

·       Clear measures

·       Objective

·       Easy to use the data to make changes

·       Authentic assessment methodology

·       Good communication tool

·       Transferable to many learning environments and disciplines

Concerns:

·       Time consuming to create and use

·       Dependent on course assignment (for information literacy) and available resources