Collaborators: Sricharan Reddy Varra,听Benjamin Vernon,听Irina Wagner, Emilie Young, Chris Zimmer
The feedback card in Buff Portal monitors the pulse of the student experience. Since Buff Portal's beta release in March 2019, the User Experience Research & Design team (UXRD) has received over 14,000 feedback card responses from students. In the feedback card, students are asked to rate their satisfaction with Buff Portal on an emoji scale and leave a comment. The emoji scale helps gauge the overall satisfaction of students, while the text comments provide the Buff Portal team with more specific information about the experiences of our users - what鈥檚 working and what needs to be improved. While the text comments are the most useful data on Buff Portal鈥檚 usability, they are also the most time-consuming to analyze.
To alleviate the pressures of tackling comment analysis during times of high-volume feedback, OIT鈥檚 UXRD team has been seeking out ways to automate comment analysis using natural language processing approaches. In the fall of 2020, OIT鈥檚 UXRD team partnered with 精品SM在线影片鈥檚 Laboratory for Interdisciplinary Statistical Analysis (LISA) for support with this task. The goal of our collaboration was to create a script that would automatically analyze the sentiments and key terms of the text comments at a given window of time. This would allow the Buff Portal team to quickly identify issues with the portal (or other CU tools) before a large number of students is affected.
We partnered with Sricharan Reddy Varra and Chris Zimmer, two statistical collaborators with LISA, to leverage AWS鈥檚 Comprehend tool and custom developed scripts in Python for natural language processing and sentiment analysis around key phrases. Using these approaches in the fall, the team was able to get to 70% precision in evaluating the feedback card comment sentiment on a positive to negative scale. We also identified a very promising path for categorizing and automatically assigning topics to the comments (e.g., 鈥渞egistration鈥� vs. 鈥済rades鈥�).
The Buff Portal team is thankful for this partnership with LISA, specifically, all of the hard work that Sricharan Reddy Varra and Chris Zimmer put into this project. We continue to iterate on the original Python script created by Sricharan Reddy Varra and we are excited to leverage this analytical approach to continue to improve how we support students and their technology experience at 精品SM在线影片.
听