Swipe right for health
How hard you press while swiping could reveal something about your mental health
LFG: Learning from Games is a series dedicated to making the complex world of video game research just a little easier to understand. I post one about 2-3 times a month in between my Science Shorts and Coming Soon series. LFG #33 is about how we swipe on the phone and what that says about our mental health.
Hello JOMT Reader!
Any sufficiently advanced technology is indistinguishable from magic.
—Arthur C. Clarke
50 years ago, the idea of holding the world’s knowledge in your pocket, video calling anyone in the world, or detecting and treating fatal health conditions would have seemed like magic.
Today, it’s reality thanks to advances in technology.
But we may be starting the next magical transformation of technology — an always-on, passive monitor of health.
Until very recently, our approach to health care has been mostly reactive. We only go to the hospital or see a doctor after something has happened to our health. And it relies heavily on our ability to accurately recognize a health condition. Even though we’ve had major advances in technology, some technologies still fall short when trying to recognize health conditions.
The good news is those technologies are starting their magical transformation. Everyday things like talking, walking, and WiFi signals can be analyzed and translated for early detection of various health conditions. For example, changes in voice have been used to detect type 2 diabetes, especially when combined with other health metrics1. Changes in how we walk can signal shifts in brain function2. Even disturbances in ever-present WiFi signals can detect when someone has fallen3.
You smartphone is not just a doom-scrolling communication device. It can say something about brain function4. Taking it one step further, researchers in Australia are exploring how mobile games can detect mental health conditions like depression and anxiety.
So in this post, I want to dig into some of the magic behind the study.
📱Everyone loves mobile games
The researchers started with a simple assumption: that almost everyone has access to and enjoys playing mobile games. With close to 50% of revenue in the gaming industry generated through mobile games, it’s a pretty safe bet.
Next, instead of looking at game scores, levels cleared, or time taken to clear levels, the researchers looked at how players interacted with the game. They measured everything from timing of when a swipe action was started to the position on the screen a swipe started, as well as the pressure, speed, duration, distance, and direction of the swipe (we’ll call them swipe metrics for short). Basically, anything you might do while playing the mobile game, they measured.
The swipe metrics were measured for three custom-made mobile games, covering the puzzle, infinite runner, and object slicing genres.
In the puzzle game, players had to swipe in one of four directions to guide the transparent square out of a maze of square blocks through a designated exit (break in maze wall).
In the infinite runner game, players swiped in one of four directions to guide the cat over or under obstacles while collecting fishbones. It’s an infinite runner game so there is no end — but the difficulty gradually increases by speeding up the action.
In the object slicing game, players swiped in any direction to slice as many objects as possible within 90 seconds, without missing objects and avoiding bombs.
The researchers recorded swipe metrics as players played these three games and mapped these metrics to questionnaire results about depression (Patient Health Questionnaire, PHQ-8) and anxiety (Generalized Anxiety Disorder, GAD-7). The thinking was that there would be a difference in swipe metrics between those who have depression and anxiety versus those who don’t.
😟Swipe metrics for depression
Researchers analyzed swipe metrics across the three game types and found moderate to strong correlations with depression. Players with depression didn’t press as hard on the screen during the swiping motion in both the puzzle and infinite runner games, but not in the object slicing game.
They also swiped more slowly in all three game types compared to those without depression. For the puzzle and object slicing games, players with depression also swiped for longer.
Interestingly, in the object slicing game, those with depression pressed harder on the screen during the swiping motion, which contrasted the lighter touch seen in the other games.
😰Swipe metrics for anxiety
Swipe metrics for anxiety were similar to those for depression. Players with anxiety didn’t press as hard on the screen, and swiped more slowly and for longer. Despite the lighter touch in other games, players with anxiety pressed harder on the screen during the object slicing game.
In the infinite runner game, players with anxiety had fewer total swipes compared to those without anxiety.
🧠Translating swipe metrics into psychology
What do slower swipes, longer swipes, and pressing harder on the screen mean in real life? In terms of depression, there’s a phenomenon called psychomotor slowing, which is a noticeable slowing of mental and physical activities. This slowdown could explain the slow swipes.
That same metric, in terms of anxiety, could be related to a person’s ability to take in information and stay focused. Anxiety can disrupt these abilities making them less able to process information, leading to slower swipes. It would also explain the fewer total swipes as they were unable to react fast enough to whatever was happening on screen.
The contradicting observations on screen pressure are harder to translate. On the one hand, more pressure on screen could be interpreted as increased frustration, which is often associated with anxiety. On the other hand, a lighter touch could mean a lack of energy or tiredness, which is associated with both depression and anxiety.
🔮What this could mean for the future
Imagine playing a game on your phone while it silently monitors how long and fast you swipe or how hard you press on the screen. If the phone detects any trends or changes in these swipe metrics, it could alert you to do something about it.
But tracking that much data could be taxing, especially for older phones. Since we know what the types of interactions that are associated with depression and anxiety, games specifically designed to test those interactions could be used to diagnose these conditions. We can call them diagnostic games (like tests, but more fun), where your high scores are converted to health condition scores that can identify health conditions and track changes over time.
This kind of metric tracking isn’t limited to games. Any interaction with our phone that involves swiping (or other gestures) could potentially say something about our health. We’d need the studies for it to be sure but in theory, this isn’t limited to games, which means we can still dial back the gaming push without excluding anybody from the benefits of a technology like this.
Personally, I’d love a future where my annual checkup was a bunch of high scores sent to my doctor from which they would be able to recommend an appropriate health plan!
If you want to read the article that inspired this post, you can read it here for free: https://mental.jmir.org/2025/1/e70577/
If you liked what you read, please consider giving this post a like and sharing it with your community!
https://www.mcpdigitalhealth.org/article/S2949-7612(23)00073-1/fulltext
Who could have imagined that changes in our voices could carry information about our health?
https://www.minervamedica.it/en/journals/europa-medicophysica/article.php?cod=R33Y2016N04A0560
Walking is a bit more intuitive but until recently, only very obvious limps and gaits were noticeable indicators of our health.
https://www.sciencedirect.com/science/article/pii/S2949715924000283
It’s amazing how our magical technology can be coerced into saying something about our health without additional gadgets and gizmos attached to us.
https://www.mdpi.com/2076-3425/13/6/959
It was only a matter of time before someone looked at all the smartphone zombies and thought “there’s a hidden pattern I can probably analyze.”
My concern is who has access to this data, how is it used, and who's making all of those decisions? We still don't have even decent laws covering AI use here in the US, and now we'd need them for this, too? Seems a receipt for more dystopia. But I love the concept and, if limited to the doctor's office, ok.