UI UX

Activate

Home exercises with nobody there to correct them

Context

Activate is my final year specialisation project, an application built to assist RSI Type 1 patients with their exercises in real time. It started from watching friends and family go through physiotherapy rather than from a brief.

The project aligns with Goal 3 of the United Nations Sustainable Development Goals, Health and Well-Being.

RSI Type 1 covers the specific named conditions rather than diffuse strain, which is why the screens carry carpal tunnel syndrome, tennis elbow and bursitis rather than one general diagnosis.

Findings

I interviewed people currently in physiotherapy, people who had completed it, and physiotherapists, and read up on the state of physiotherapy in India, the existing apps and available technology, and RSI literature. Here is what I found.

The WHO recommends one physiotherapist per 10,000 citizens. India has 5,000 qualified ones.
Patients may be told to follow varied steps for the same exercise.
RSI Type 1 is increasingly prevalent among people with desk jobs, with a rise of 30 to 40% observed in Bengaluru.
Exercising at home lacks guidance, leading to potential errors and an increase in pain.
Patients forget steps, perform with the wrong posture, and most of them fail to complete their treatment.

Explorations and decisions

Few options were compared before anything was designed.

Video calls Rejected

A physiotherapist watches remotely and corrects in the moment.

Dropped because it does not remove the constraint. A person still has to be present, so availability stays limited and the cost rises.

A patient on a laptop video call with a clinician.

Recorded exercises Rejected

Film the correct movement once and let patients follow along.

Dropped because a recording cannot see the patient. The steps are modified to how severe each case is, so a generic video is wrong for most people watching it, and it gives no signal about whether they are doing it right.

A tablet on a table playing a follow along exercise video.

Computer vision

Detect the patient’s own hand and body points, and judge the movement against a reference.

This was the option taken. Open source pose libraries already handle the detection, so the work was the judgement: turning it into a decision about whether the movement is correct, and saying so while the patient is still going.

A person mid pose with a skeleton overlay tracking their joints.

Idea

Using open source libraries like MediaPipe to detect hand points and body points, and extending the code further to guide patients with their exercises.

Four steps. One, the exercise is recorded at the clinic with the physiotherapist present and stored as data set D1. Two, the same exercise is performed at home and collected as D2. Three, the two are compared to decide correct or incorrect. Four, when incorrect the patient is given feedback in audio and visual form while they are still exercising.

Applications for both patient and orthopedist

The patient needs to be told what to do and corrected while doing it. The orthopedist needs to walk into the next appointment already knowing what happened in between. Those are different products, so Activate is two of them, each on desktop and mobile.

The final design

Patient side

Exercise preview

A gif previews the exercise, so the patient knows the shape of the movement before being judged on it.

The exercise screen, with three exercises as tabs and a preview clip beside the timing and instructions.

Guided exercise

Bad form is called out in audio and visual at once, with a snippet of the clinic recording in the corner to correct against.

Get feedback

Afterwards the patient gets a report on their duration and a feedback form, both sent to the orthopedist.

The feedback screen, showing time taken beside six pain rating sliders for different daily contexts.

Home

Feedback form

Feedback

Schedule

Orthopedist side

More but less

The home screen shows the working day rather than a set of metrics: appointments in order, who is being consulted now, and how the day’s diagnoses break down. Details open deeper information, so the list stays readable.

Orthopedist home screen, with the day's appointments beside a pie chart of diagnoses.

Easy monitoring

Patients falling behind on their exercises are highlighted.

Patient list screen, with patients who are not doing their exercises highlighted.

Clear insights

The detail is visualised rather than listed: pain on a body map, progress as a chart across months, a calendar of the days actually done, and observations as a dated log. Patterns show up at a glance.

Patient information screen, with a body map, progress chart, exercise calendar and observations log.

Home

Feedback

Schedule

Proof of concept

Used MediaPipe and Python to write the code and executed it using Unity and C#. Blue spheres indicate the exercise is performed in right form and red spheres indicate wrong form.

A webcam window with hand points detected and labelled left.

Detecting hand points

A hand rendered in Unity with blue spheres at the joints, indicating correct form.

Right form

The same hand rendered with red spheres at the joints, indicating incorrect form.

Wrong form

Watch the proof of concept →

Reflection

Healthcare was new to me, and so were Python and MediaPipe. I learned enough of all three to build the thing rather than describe it.

What I wanted next, written at the time: better accuracy in the posture correction, more areas of physiotherapy than RSI, and using the data to suggest exercises rather than only report.

It was never tested with patients. Until that happens the design is a proposal.

← Previous project Video Assessment Next project → Edwisely Brand Refresh