Modified Delphi Card Sort

“A Modified Delphi Approach to a New Card Sorting Methodology” by Celeste Lyn Paul

In Celeste Lyn Paul’s research article, she describes a card sorting methodology called the “Modified Delphi Approach” which she argues is a superior card sorting method that improves the quality of results and lowers time and cost investment for researchers and participants. For example, Paul demonstrates that, compared to open card sorting, the modified Delphi approach allows participants to collaborate without potential collaboration bias from bandwagon or halo effects, requires fewer participants which lowers time and cost investment for researchers, lowers cognitive load or cognitive costs for participants while testing, and provides more effective results through its iterative process.

I believe that her study demonstrated some promising results and introduces a method that I would personally like to test in either my current project, or future projects as a UX researcher and designer. What I found to be the most interesting and compelling idea is how this Delphi method employs an iterative process that starts with a “seed participant” and then future participants continually make changes to the information architecture until a consensus is reached. I think this a really interesting and potentially useful approach to pre-design info arch testing because it reduces the overall cognitive load on participants making them more willing and able to complete an effective card sort and the results of each iteration may be more useful than having to analyze the statistical differences between groups in more typical open card sorting synthesis and analysis.

However, I believe that there still may be group or collaborative biases that are present with this collaborative method. Although, band wagon and halo effects may be reduced, participants may still feel less and less willing to make changes to an existing structure the farther along they are in the testing process and may not be as willing to make significant changes to a previous participants structure because they believe they may not be qualified or are just hesitant to make major changes to an already existing info structure.