A Wetpaint Site

Sign in to Wetpaint

CUWL Wiki

Already a member?
Sign in

Welcome! Wikis are websites that everyone can build together. It's easy!

Location: Metalib Assessment Bibliography

Discussion: Possible metalib assessment protocols that we can use

Keyword tags: None
Watch

steve.frye	Possible metalib assessment protocols that we can use Mar 13 2008, 10:53 AM EDT Please post your protocols before we meet from 12-1:30pm on 3/24 Post reply Do you find this valuable?
rhoeper rhoeper	RE: Possible metalib assessment protocols that we can use Mar 20 2008, 2:21 PM EDT Hi, all: I've read--or at least skimmed--all of the bolded articles. Most of them seem to advocate some type of actual observed usability testing. I think an online, or even mailed, survey/questionnaire would be easiest for us, but passive; we'd be dependent upon users actually going to or responding to the survey. However, observed usability testing brings its own set of problems: 1) it takes a lot longer; 2) which campuses would conduct the testing? there is only one rep from several campuses on this working group, and usability testing is labor-intensive; 3) do we have the expertise on this committee that is needed to code and evaluate responses? 4) how would we record the studies? I have other questions related to observed usability studies, but can raise those if we decide to do one. I also wondered, do we plan to compensate our test participants somehow? Most of the studies in the articles seemed to include some type of incentive. My last question, I am wondering if we need to figure out just what it is we want to find out from our testing or surveying of MetaLib before we make a final decision on testing protocol? Which protocol would best serve the questions we are trying to answer? Those are my thoughts on this Thursday afternoon. Enjoy your weekends, everyone! Post reply Do you find this valuable?
steve.frye	RE: Possible metalib assessment protocols that we can use Mar 24 2008, 10:42 AM EDT I was hoping to be able to go into more detail then the following... but have run out of time... so here's my bullets to discuss during our meeting today: * In terms of what we assess, I would lean towards assessing quicksearch sets. I think we should assess quicksearch sets in a comparative context... i.e. compared to native interface and/or Google Scholar. * I don't think we need to assess a large number of people to obtain meaningful (if not statistically significant) results. * I do think part of our assessment should be done under the observation of a library staff member. * I don't think we should assess the customsearch or the 'My Research' part of metalib. * In addition to assessing quicksearch sets, we might want to assess metalib as a resource discovery tool. * I think the protocol should be set-up with a number of canned (but real) assignments (tasks) * I think we already have quite a nice literature review to write up and present to our colleagues Post reply Do you find this valuable?
tmuraski tmuraski	RE: Possible metalib assessment protocols that we can use Mar 24 2008, 11:41 AM EDT I like the idea of comparing quicksearch sets to the native interface - maybe Ebsco I'd also like to have some assessment of the effectiveness of the user interface - the tabs, results view, icons, etc. I agree that usability testing is probably the most effective way to get at this info and that a small group would work. I do think that there's quite a bit of information from our lit review. Post reply Do you find this valuable?
mollelajoie	RE: Possible metalib assessment protocols that we can use Mar 24 2008, 11:48 AM EDT Belliston's study at Brigham Young seemed like an interesting way to go if we were looking for an assessment of federated searching. However, like Ronna, I am wondering if we need to figure out what it is we want to find out about MetaLib and users before we decide on a protocol. I am wondering ultimately what our research question or problem statement is. I think getting a definitive problem statement will be of great help in choosing the most appropriate protocol. Are we doing an assessment of satisfaction levels of federated search in the UW-System or are we wondering about particular aspects of the MetaLib product itself? Post reply Do you find this valuable?
joiao	RE: Possible metalib assessment protocols that we can use Mar 26 2008, 6:16 PM EDT "I like the idea of comparing quicksearch sets to the native interface...I'd also like to have some assessment of the effectiveness of the user interface - the tabs, results view, icons, etc. " I am assuming when you say user interface--you mean assess the user interface within Metalib. Can you tell me any more about what other aspects of the user interface you might like assessed...you said the tabs, results view, any others? As I am coming up with my own ideas of what to assess (as well as how to go about it), I'd like to hear from group members as to what they think we should be assessing. Post reply Do you find this valuable?
tmuraski tmuraski	RE: Possible metalib assessment protocols that we can use Apr 3 2008, 1:01 PM EDT I would especially be interested in the Tabs and the words used to describe each function in Metalib via tabs, preferences between table & brief views. The advanced vs. simple search functionality might be another area - I find the search functionality of selecting 30 records problemmatic, but I don't know if most undergrads do. That might be interesting as well. Post reply Do you find this valuable?