Video games have long since moved on from their origin as novelty electronic distraction. They are open worlds, creative studios, and classrooms, community centres and marketplaces. Video games are now intrinsic to the digital wallpaper surrounding us all, whatever devices we may be using – from phone to virtual reality.
As much as this is increasingly true, it’s also true that a significant segment of society is being left behind. According to Novis Games co-founder and CEO Arianna Ortelli, more than 250 million blind or visually impaired people are largely excluded. And this is why she set off on a mission to try to change that.
In 2019, as an economics student in Turin, Italy, Arianna became involved in developing a video game concept that relied on sound and vibration to allow players to come together to play in open or outdoor environments. The experience provided her with something of an epiphany.
“We just looked at each other and said: ‘If I can play this with just sounds and vibrations, so can a blind person,” she recalls. That led to a further question: How might all games be improved to allow better access to the visually impaired.
Arianna attended a meeting of her local blind association shortly afterwards, and it was there she met Marco Andriano. Thrilled by the idea of opening up games to more people like himself, Marco would go on become her co-founder and Chief Marketing Officer.
Through Marco, Arianna says she was able to better understand the set of challenges that visually impaired people face when engaging with video games or interactive entertainment
“There are some games we started testing together at the time. For example, football games on different consoles. In simple games, where there is time to think and maybe other people to help him play, then it’s quite possible,” she says. However, she adds that most games are festivals of detailed visual complexity, often featuring competition with other humans, and that leave very little time for decision making
“And this is for people with low vision. Imagine how it feels with no vision. It’s also very difficult to enter the game at the first step, the first menu. It’s not possible to understand where I have to go to start my game.
— Arianna Ortelli
With a deeper understanding of the nuanced areas of challenge, Novis Games began developing ideas to address them. Initially, that involved creating their own original games, but they soon began to realise that what the community really needed were solutions for games developers across the whole industry. It is, she notes, a very difficult and expensive additional thing for developers to do on their own.
They began by considering how games’ user interfaces and menus could be enhanced in the simplest way to make them more accessible. Using existing technologies, they worked on ways to supplement games with automated audio descriptions of menu items and relevant on-screen imagery. That was followed by looking at in-game object recognition and mapping.
She says: “We started developing 3D sounds that help you understand the position of elements in the game, and also their meaning. For example, while you’re moving around in a beautiful environment, you can understand that there’s something in front of you, and whether it is dangerous, or good, or something to collect.
With these features developed, Novis Games then began work on packaging them with a tool that developers could use to begin integrating them with a wide variety of games. Over the last 18 months, Novis Games has created demos for game genres including platformers, first-person shooters, and point-and-click adventures.
Microsoft’s positive impact
While Novis Games has so far worked largely manually to add its accessibility features to simpler games, its goal is to begin transforming the experience of far more complex – and popular – genres for those with visual disabilities.
To achieve that, Novis Games is working with generative AI technology from Microsoft. By training AI using its work on simpler games, Arianna says the team believes they will be able to scale their tools to function helpfully in more challenging contexts.
“Action games – in which you have many, many elements, and many decisions to make and not much time to do it – are a challenge. The most difficult part is to do it automatically. We think we can train the AI and make it prepared.”
Arianna adds that even if it’s not ultimately possible to make fast-action games fully accessible for people with visual impairments, the effort to develop easily scalable, AI-driven tools to make them more so will open up new opportunities. She imagines new types of games, where different character roles can be taken up more easily by those with sight problems.
“If I’m not able to give all the information to certain types of player, maybe they can still be useful in other ways. Maybe they can use sounds, and their voice, and that can create collaboration between players,” she adds.
— Arianna Ortelli
Novis Games’ use of AI has been levelled-up by its involvement in Microsoft’s Entrepreneurship for Positive Impact programme. The scheme provides support to early-stage start-ups with missions to deliver social impact. That support comes in the form of mentoring and technology expertise, increased visibility, and access to Microsoft technology.
Arianna says her path to membership of the programme was an interesting – and transformative – one. Any budding social entrepreneur can apply to the programme via its website, and in 2021 the Novis Games team were invited to deliver a three-minute pitch in person to Microsoft Executive Vice President Jean-Philippe Courtois at an event in Milan. I few months later, they were onboarding into the programme.
“It was quite interesting!” she says. “It’s not just a mentoring programme, but more a custom programme for each of the start-ups involved. We had a single mentor who provides us with contacts, technologies, and also [connects us to] other people that want to join us and support us to scale our idea from our market to others.
Novis Games were paired with the “amazing” Juliana Tioanda, Chief of Staff for Microsoft’s Xbox Engineering division, and as such gained access to remarkable industry-specific mentoring.
“We came to understand, month after month, what were the main things we needed to focus on. We achieved goals time after time.”
In September 2023, Arianna joined Juliana and Wyatt Jackson, Principal Software Engineering Manager on Microsoft’s Xbox Cloud Gaming service, to lead a team in the company’s annual Global Hackathon. It was an opportunity to collect a large, diverse group of Microsoft experts together, from a dozen countries, to further explore how generative AI could help break down barriers to adopting accessibility features for game developers. The team achieved a second-place finish in the overall event.
“I involved of course all my Novis Games team. We had the opportunity to study what we had done with our point-and-click game and recreate that, in just one week, using generative AI, and to create this package we can use in diverse types of games,” says Arianna.
Novis Games has built its technology on Microsoft Azure services and with SeeingAI, a Microsoft-built iOS app that uses an iPhone’s camera to identify people and objects and then audibly describe them for the visually impaired.
“To give you an idea,” she says of both the programme and the Global Hackathon’s contribution to their use of AI, “what it previously cost us thousands of hours in developing, we can obtain the same result in just a matter of minutes.
“We can generate the same basic features – for example, audio descriptions or understanding in-game elements – directly from the game files and runtime, which is quite impressive.”
The fast advances Novis Games has been able to achieve with generative AI means they’re now pursuing the aim of being able to deliver templates to developers to fundamentally transform how they make their games more accessible. What used to happen (expensively) at the end of the development process, can now happen as the software is being built.
Ultimately, says Arianna, the work of Novis Games – aided by an impressive range of support from Microsoft experts – could and should lead to advances far beyond just gaming.
More accessible games are more accessible for everyone, after all, not just those with visual impairments. The new AI-powered features themselves could lead to whole new game dynamics. And video games are just one type of software that could benefit – applications are easy to imagine in GPS navigation, other location-based services, operating vehicles, or interacting with smart speakers, to name just a few.
For now, Novis Games is working with Microsoft to identify 10 games with which to test the technology, and via which it could ultimately be released to users via Xbox channels.
“It will be possible for the first time to play something like that – everyone will be able to play on their PC or Xbox. It’s quite amazing!”
