← Home

Streamlining cumbersome data reviews

Team: NASA Ames HCI Group

Role: Design lead

Completed: Feb 2017

Summary

NASA organizations often store their authoritative data in Word documents and rely on built-in commenting features to facilitate their baseline reviews. As more and more orgs move to centralized, structured data systems, they need new tools to support those review processes. We designed a mechanism for users to have review discussions directly on the data they are reviewing.

Keep reading for more details on our process

The challenge

Reviewing and discussing data is a common use case for NASA organizations. Typically, these organizations use emails, spreadsheets, or Word's commenting feature for their reviews. That approach has several issues:

Many organizations compile and disposition all comments in a separate spreadsheet. Image redacted.

A centralized data system, in itself, helps to resolve a few of the above issues, but our framework only allows users to comment on a record as a whole, which still separates the discussion from its context.

We sought to improve our commenting component by allowing users to comment directly on the data they are discussing.

Our process

Even though there were already several known issues, we wanted to better understand the way our users actually handle their commenting processes. To do this, we conducted several retrospective interviews where users would walk us through their process, making sure to collect the associated artifacts (and then some) for further analysis. Additionally, our competitive analysis gave us insight into how other applications are handling commenting.

Our research pointed to a few key areas we wanted to focus on:

Overall, we wanted to stick to a design that would be familiar to our users who typically use Word's commenting features1, yet expanding on that functionality to address the issues found during our research.

Our solution

The data being reviewed in these processes is highly structured and, because it's important for comments to be closely coupled with the content being discussed, users may comment on each form field individually. However, comments aren't considered part of the actual form data, so our design aims to make that separation clear by moving the discussions to the right-hand gutter, outside of the form's borders.

Review discussions take place directly in context.

We know that reviews are often a back-and-forth, so each comment may be replied to arbitrarily by any user in the system. Additionally, each field may have an arbitrary number of discussion threads going on at any given time.

Users can reply to high-level comments to start a thread.

I like this a lot better! It's all in one place, I don't have to float a document back and forth

- Usability participant

After a discussion has taken place, the owner of the data needs to make changes where appropriate. To help keep track of which comments have already been addressed, users may mark comments as 'Resolved', hiding them from the comment list.

Resolved comments can still be viewed and re-opened if necessary.

Results

We usability tested the new design with 10 participants, moving through a real-life scenario that required the use of each feature. The tests resulted in the following findings:

The core commenting functions tested very well amongst our participants, with the majority of feedback regarding the way resolved comments are handled. Contrary to our hypothesis, users were surprisingly cavalier in their desire to delete comments once addressed.

It's a lot easier to navigate here than it is in a Word document.

- Usability participant

After using the new design for several months, one user community performed some analysis to quantify their time savings. They looked at documents which have had revisions go through both their old review process as well as the new one aided by our design. On average, they determined that the new review process saves 32 days2 per document review.

1. Google Docs is a popular option too.

2. There are lots of variables here and they are still gathering data, so we expect the number to drop a bit.