New Evaluation Framework Aims To Make Remote Collaboration Tools More Inclusive

As remote work cements itself in modern workplaces, digital collaboration platforms such as Zoom and Google Docs have become indispensable. Yet, researchers argue that these tools are still built around a flawed assumption—that all users collaborate in similar ways.

A team of researchers has now introduced a new human-computer interaction (HCI) framework called RemoteCollabEval (RCE), designed to uncover hidden barriers in digital teamwork and help developers create more inclusive collaboration environments.

The research falls within the broader field of Human-Computer Interaction, which focuses on improving usability and user experience in digital systems.

According to Sandeep Kuttal, an associate professor at North Carolina State University, existing evaluation methods rely heavily on simplified assumptions. One widely used technique, known as a groupware walkthrough, involves designers simulating how a small group of users might interact on a platform. However, these simulations often overlook the diversity in communication and collaboration styles.

Kuttal notes that individuals from different backgrounds approach teamwork differently, but current inspection methods fail to capture this variation—limiting how effective and inclusive collaboration tools can be.

Six factors shaping collaboration

To address this gap, researchers identified six core personality traits that influence how people work together:

• Leadership approach—ranging from democratic to authoritative
• Interruption behaviour—whether someone speaks over others or waits
• Use of non-verbal cues—expressive versus reserved communication
• Relationship focus—prioritising rapport versus task completion
• Social awareness—attention to team dynamics
• Collaborative confidence—belief in the group’s ability to succeed

Using these dimensions, the team created detailed user “personas” to represent different collaboration styles. These personas allow developers to simulate real-world friction and identify what the researchers call “inclusivity bugs”—issues that standard testing methods often miss.

Rethinking how platforms are tested

The RCE framework builds on traditional groupware walkthroughs but requires designers to actively consider all six personality facets during evaluation. By combining structured personas with a revised walkthrough process, the method provides a more nuanced assessment of how platforms perform across diverse user behaviours.

To test the approach, researchers conducted a study involving 29 students divided into 10 teams. Half the teams used conventional evaluation methods, while the others applied the RCE framework to assess the same collaboration platform.

The results were striking. Teams using RCE identified six times more inclusivity-related issues compared to those using traditional methods.

Toward better digital teamwork

The findings suggest that incorporating behavioural diversity into design testing can significantly improve how collaboration tools function in real-world settings. By identifying friction points early, developers can refine features and interfaces to better support varied teamwork styles.

Importantly, researchers emphasise that RCE is both practical and scalable. It does not require extensive resources or specialised infrastructure, making it accessible for design teams across organisations.

As remote and hybrid work environments continue to evolve, such approaches could play a critical role in shaping collaboration tools that are not just functional, but genuinely inclusive.