IBM developerWorks

Website Usability Evaluation And Testing
project

THE CHALLENGE

Our group was tasked to develop recommendations for removing redundancies from the developerWorks site, with specific focus on card sorting Technical Topics, providing recommendations for improving ease and functionality of Developer Centers and dW Answers, improving Communities by testing specific tasks, and finding and repairing breaks.

PROJECT GOAL

The goals for this project were to meet the client's requests for improving three functions (Develop Centers, dW Answers, and Communities) and draw valuable findings and recommendations. In order to understand how users use IBM developerWorks, we made an interaction map, conducted interviews with 5 users, completed personas based on our interviews, conducted surveys, and performed comparative analysis and usability tests. Finally, a video was also made to present our findings and recommendations to IBM developerWorks.

INTERACTION MAP

Our team generated this static representation of the IBM developerWorks website system.

project

INTERVIEWS

We interviewed five potential users (programmers with varying degrees of experience). The intent of the interviews was to receive feedback on the habits of potential users, to get a sense of what draws them toward collaborative sites, along with what detracts them, how they code, and which platforms they currently use to solve their programming challenges. Additionally, we wanted to ensure that interviewees received a basic overview of IBM developerWorks and had an opportunity to look through the site briefly, in order to get their input about likes and dislikes from an initial look at the system. Feedback from the initial interviews informed our group’s priorities, future client interactions, surveys and task-based evaluation.

PERSONA & SCENARIO

We created two personas & scenarios using the information we collected from the five interviews with potential users. The "explorer developer" Evan and the "traditional developer" Todd. Evan had an affinity for motivation while Todd had an affinity for responsibility.

project
project
project
project

SURVEY

In determining how to proceed with surveys, our team first reviewed our personas, along with an IBM Survey Personas document provided by our client. In developing our Target Population, developers were divided into four categories: developer, senior developer, software tester, and project manager. Each team member developed survey questions based on research questions within the following four areas: Background / Development Experience; IBM developerWorks Technical Topics; IBM Community; and Question and Answer Forums.

The intent of conducting surveys was to better understand user experiences with IBM developerWorks. We sought to determine how developers solve coding challenges, and continue shaping recommendations for improving the user experience with Technical topics and Community within IBM developerWorks. Feedback from the surveys allows us to revise our personas, develop additional scenarios, inform our group’s priorities, drive future client interactions, and inform our preparation of task-based evaluation.

COMPARATIVE ANALYSIS

The competitors that were identified included the Microsoft Development Center, Google Developers, StackOverflow, Oracle Communities and Github. We compared these products to specific functionalities of developerWorks, assessing the features of developerWorks communities, dW Answers and Developer Centers against their competitors.

project
project
project
project

HEURISTIC EVALUATION

As part of the heuristic evaluation, each member in our team analyzed parts of the IBM developerWorks website using the Nielsen Heuristics and identified the key usability problems pertaining to those parts. These individual findings were then combined together to create a combined usability evaluation based on findings generated and associated recommendations.

USABILITY TESTS

Our team’s goals for Usability Testing included both meeting our client’s requests for completion of tasks :Card Sorting, Independent Agent (IA) Survey, A/B Testing of IBM developerWorks landing site, along with our own efforts to validate findings and recommendations as determined from interviews, surveys, comparative analysis, and heuristic evaluations conducted earlier. Specifically, we focused on determining how users navigated from the main site to Developer Centers, dW Answers, and developerWorks Communities, and how they interacted with those sites upon arriving at them.

project

Card Sorting

Asked users to sort developerWorks categories into a hierarchy and name them as they deemed appropriate

Independent Agent (IA) Survey

Involved labels / terminology pertaining to developerWorks platform

project

A/B Testing

Required users to click on locations of the existing A and proposed B developerWorks landing page in order to show where they’d expect to find specific information

User Interface A


project

User Interface B


project

FINAL VIDEO REPORT

Here is the video of our project, which includes key findings and recommendations we've made for IBM developerWorks.

CLIENT

IBM developerWorks

TYPE

Group Project
User Experience Research
Usability Evaluation

TIME

Jan 2015 - May 2015

BACKGROUND

IBM developerWorks encompasses a wide array of IBM-based user tools, and serves as an interactive platform in order to drive collaboration, education, and sales of IBM products. Full use of IBM developerWorks requires registration, and provides access to tutorials, developer centers run by actual IBM product teams, access to the question and answer feature dW Answers, access to more than 3800 public communities, and other features.

TARGET USERS

The target users of developerWorks include any individuals or teams who utilize IBM products, ranging to individuals at small startups to large multi-national corporations.

METHODS

Interaction Map
Competitive analysis
User Interviews
Survey
Persona & Scenarios
Heuristic Evaluation
Usability testing

MY ROLE

Designing interview and survey questions, recruiting and managing interviewees from our university, and analyzing the findings and recommendations before we submitted them to IBM

TOOLS