Research initiative to measure the maturity of the “subgroup” feature of GitLab
You can watch a presentation (10 minutes) I made, covering the process and our findings. You can also view the research initiative, and the resulting improvement recomendations in the orignal GitLab issues.
For this research project, I was the DRI, but supported by my product manager, and with guidance from our UX Research team.
To begin, I wanted to evaluate how a user would go through the process of creating a small organization with groups and projects within our platform. I worked with our UX research team and created a test script and participant screener to find qualified testing candidates.
Next I thought about how I could create a test environment for research participants to enter into a fresh GitLab instance, and build a small organization in the instance.
I was immedeately presented with a problem; in that, creating groups and subgroups in our environment is generally public, and research participants would see the previous tests from other participants. To resolve this, I worked with support engineers to create test environment where test participants could log in to a private environment and complete the test with out viewing other data.
I created 6 test scenarios which were graded against the UMUX-Lite scoring system. The tests were live but unguided, so I was able to see what particiapnts were doing and listen to their problem solving process in order to complete the tasks. After the tests were completed, I synthesized the data to build a better understanding of where our product was failing in supporting users in completing the tasks successfully.
After going over the interviews and finding comonalities, I proposed a number of improvements that could help improve the maturity score in the future. I gave a presentation on this and found out a number of other teams were experiencing similar concerns and had related proposals to improve the platform. This validated my research in that it was just not isolated to our team, but rather, impacted various other sections of the platform, and it was reassuring that we were on a path to address the difficulties users were experiencing and had proposals in place to fix them.
You can watch the presentation I gave where I go into detail on this research effort in the video below.