New Evaluation Framework Aims To Make Remote Collaboration Tools More Inclusive

As remote work cements itself in modern workplaces, digital collaboration platforms such as Zoom and Google Docs have become indispensable. Yet, researchers argue that these tools are still built around a flawed assumption—that all users collaborate in similar ways.

A team of researchers has now introduced a new human-computer interaction (HCI) framework called RemoteCollabEval (RCE), designed to uncover hidden barriers in digital teamwork and help developers create more inclusive collaboration environments.

The research falls within the broader field of Human-Computer Interaction, which focuses on improving usability and user experience in digital systems.

According to Sandeep Kuttal, an associate professor at North Carolina State University, existing evaluation methods rely heavily on simplified assumptions. One widely used technique, known as a groupware walkthrough, involves designers simulating how a small group of users might interact on a platform. However, these simulations often overlook the diversity in communication and collaboration styles.

Kuttal notes that individuals from different backgrounds approach teamwork differently, but current inspection methods fail to capture this variation—limiting how effective and inclusive collaboration tools can be.

Six factors shaping collaboration

To address this gap, researchers identified six core personality traits that influence how people work together:

  • Leadership approach—ranging from democratic to authoritative
  • Interruption behaviour—whether someone speaks over others or waits
  • Use of non-verbal cues—expressive versus reserved communication
  • Relationship focus—prioritising rapport versus task completion
  • Social awareness—attention to team dynamics
  • Collaborative confidence—belief in the group’s ability to succeed

Using these dimensions, the team created detailed user “personas” to represent different collaboration styles. These personas allow developers to simulate real-world friction and identify what the researchers call “inclusivity bugs”—issues that standard testing methods often miss.

Rethinking how platforms are tested

The RCE framework builds on traditional groupware walkthroughs but requires designers to actively consider all six personality facets during evaluation. By combining structured personas with a revised walkthrough process, the method provides a more nuanced assessment of how platforms perform across diverse user behaviours.

To test the approach, researchers conducted a study involving 29 students divided into 10 teams. Half the teams used conventional evaluation methods, while the others applied the RCE framework to assess the same collaboration platform.

The results were striking. Teams using RCE identified six times more inclusivity-related issues compared to those using traditional methods.

Toward better digital teamwork

The findings suggest that incorporating behavioural diversity into design testing can significantly improve how collaboration tools function in real-world settings. By identifying friction points early, developers can refine features and interfaces to better support varied teamwork styles.

Importantly, researchers emphasise that RCE is both practical and scalable. It does not require extensive resources or specialised infrastructure, making it accessible for design teams across organisations.

As remote and hybrid work environments continue to evolve, such approaches could play a critical role in shaping collaboration tools that are not just functional, but genuinely inclusive.

Also Read:

IPL 2026: You feel different as Google’s AI is factored in

Emotional Blindness May Raise Risk of Short Video Addiction, TikTok: Study

Teamwork makes the dream work?

Numbered jerseys effectively increase overall teamwork performance during cardiac arrest.

In new research from CHEST 2017, a team from Montefiore Medical Center in New York aimed to create a team-driven atmosphere in the hospital and hypothesized that the use of personalized numbered jerseys for each member of the code team would help to improve teamwork and overall time to perform critical clinical actions.

The study included ten critical care medicine fellows who were randomized into two groups. One group received personalized number jerseys for each code team member, while the control team wore their regular clothes. Each of the groups were asked to participate in six cardiac arrest scenarios using high-fidelity simulation, and participants alternated between team leader and team member roles for each case. Trained faculty using the validated16-item Mayo High Performance Teamwork scale graded teamwork.

Results found that the Mayo Teamwork score was significantly higher in the group donning team jerseys as compared with the control group. (23.5 vs 17.5, P <.01). In addition, there was a statistically significant difference in the percent of directed commands in the experimental vs control groups (35.63% vs 19.58%, P<.01).

“This study demonstrated that the use of numbered jerseys for individual code team members is an effective way to significantly increase the overall teamwork performance during a CA event,” says Dr. Yekaterina Kim, lead searcher, “in addition, the use of such jerseys significantly increased the number of direct commands by team leaders during such scenarios, thereby reducing the percent of ineffective commands.”

Further results from this study will be shared at CHEST Annual Meeting 2017 in Toronto on Wednesday, November 1, 1:30 PM-2:30 PM at the Metro Toronto Convention Centre, Exhibit Hall, Poster Number 130. The study abstract can be viewed on the journal CHEST® website.

 

Group project? Taking turns, working with friends may improve grades

It has become an almost essential element of academic life, from college lecture halls to elementary classrooms: the group assignment.

Dreaded by some, loved by others, group projects typically aim to build teamwork and accountability while students learn about a topic. But depending on the assignment and the structure of the groups, a project can turn out to be a source of great frustration — for instructor and students alike — or the highlight of the school year.

Now a University of Washington-led study of college students has found that the social dynamics of a group, such as whether one person dominates the conversation or whether students work with a friend, affect academic performance. Put simply, the more comfortable students are, the better they do, which yields benefits beyond the classroom.

“They learn more,” explained Elli Theobald, a postdoctoral researcher in the Department of Biology and the lead author on the study, published July 20 in PLOS ONE. “Employers are rating group work as the most important attribute in new recruits and new hires. If students are able to demonstrate that they have worked successfully in groups, it would seem that they should be more likely to land the job.”

Theobald is part of the UW’s Biology Education Research Group lab, formed by several faculty members in the Department of Biology about a decade ago to research how to most effectively teach biology to undergraduates.

A separate study by the BERG lab on group work, published in the July issue of Active Learning in Higher Education, finds that college students, when given a choice of whom to sit and work with in a large classroom setting, gravitate toward those who appear most like them — whether by gender, race and ethnicity, or academic skills.

Over the years, research spanning K-12 through post-secondary education has pointed to the value of group work in fostering collaborative skills and in cementing learning through interaction. In the sciences, labs are a common, though not the only, form of group work, Theobald said. As with many disciplines, STEM fields lend themselves to readings, worksheets and other activities that can be completed by multiple people working together.

For this study, researchers compared survey responses and test scores stemming from two different project styles — single-group and “jigsaw” — with three assignments each during two sections of an introductory biology class at the UW. Each of the 770 students enrolled in one of the two sections of the course experienced each project style at least once. In a single-group activity, student groups completed a worksheet together, relying on their notes and textbooks. In a jigsaw, student groups were assigned specific sections of the worksheet; students then were shuffled to new groups in which each person in the group had completed a different section of the worksheet and could teach their new groupmates what they had learned. Students took an eight-question test after each assignment.

The study found that students who reported a “dominator” in the group fared worse on the tests than those who didn’t express that concern. It also found that students who said they were comfortable in their group performed better than those who said they were less comfortable.

The jigsaw activity appeared to result in more collaboration: Students were 67 percent less likely to report a dominator in jigsaws than in single-group activities. “This suggests that jigsaw activities with intentional structure more effectively promote equity than group activities with less intentional structure,” researchers wrote.

The nearly 770 students who completed all the assignments, tests and surveys had formed two- and three-person groups with those who sat near them in class. (Jigsaw assignments later shuffled initial groups.) Two-thirds of participants were female; people of color, including students who identify as Asian, Under-Represented Minority, and International, made up more than half of respondents.

While the gender and racial and ethnic makeup of the participants informed the study, Theobald said, researchers don’t have details on who worked with whom so as to extrapolate from the composition of groups. For instance, were the experiences of women who worked with men different from those of women who worked in all-female groups? If a group contained only one person of color, what was that person’s experience compared to the rest of the group? That kind of information is ripe for further research, Theobald said.

However, one noticeable data point emerged: International and Asian American students were six times as likely to report a dominator than white American students. “Not all students experience group work the same way,” researchers wrote in the study. “If one student dominates a conversation, it can be particularly jarring to students from cultural backgrounds that place more emphasis on introspection and thinking on one’s own as opposed to a direct relationship between talking as a way to work through ideas.”

Though the data was collected from college students, the findings translate to other settings, Theobald said. She pointed to a study Google conducted to determine what made groups successful — establishing group routines and expectations (“norms”) and adding a brief window at the beginning of work time for casual talk. Such findings, along with those of the UW study, can inform employers as well as K-12 teachers about productive group work, she said.

The younger the students, the more structure a teacher is likely to have to establish, Theobald added. But when teachers make an assignment sufficiently interesting and complex, and manage student behavior, there is a potential for students to work together happily and productively.

“If we can get our groups to be more comfortable, students should learn better and work better,” Theobald said.