PROFILE — These 2 teenagers built a sign language translator during the pandemic | Item

ConchShell wristband turns sign language into spoken words


We portray cool kids doing cool things.
Do you know someone you think should be on our site?
Email us at [email protected] and tell us what makes them so great.


name

Jin Schofield
Sarvnaz Ale Mohammad

Hometown

Richmond Hill, Ontario

Age

17

Claim to fame:

While we were napping during lockdown, two 17-year-olds from Richmond Hill, Ontario, were hard at work inventing a new device to help deaf people.

The ConchShell bracelet designed by Jin Schofield and Sarvnaz Ale Mohammad uses a camera and a small computer to translate American Sign Language (ASL) into spoken English.

American Sign Language allows people to use hand movements and facial expressions to communicate without speaking.

Although the device is still under development, their pandemic project has already won at least $15,000 in prizes.

“It really gave us a lot of purpose and motivation for all the time we were indoors and in lockdown.” – Sarvnaz Ale Mohammad, 17 years old

Jin Schofield, left, shows the ConchShell bracelet codes on a laptop computer. Sarvnaz Ale Mohammad, right, shows how the device is worn. (Image submitted by Jin Schofield)

The light bulb moment

Jin told CBC Kids News she was in grade 9 when she volunteered at a lab and met a deaf student.

He had to work with a human translator to help him communicate with those who didn’t know sign language, and it stuck with him.

When Jin and Sarvnaz were thinking of ways to stay busy during the pandemic, the idea of ​​automating this translator came up.

“What if we put a camera on someone’s chest and it could translate ASL? Jin remembers asking Sarvnaz, half-jokingly.

To Jin’s surprise, Sarvnaz took her seriously and said, “That’s a great idea!”

The inventors used 30,000 images of ASL hand gestures to create the ConchShell wristband software. (Image submitted by Jin Schofield)

Build on the idea

The two had basic programming knowledge from their robotics team, but didn’t know where to start.

So they signed up for an eight-week online machine learning course designed by Stanford University in California.

They also arranged one-time phone calls with local college professors who directed them to other resources.

None of the girls are members of the deaf community, so part of their skill development was researching inventions that already existed online.

Teens also contacted people they knew who used ASL to ask about their preferences.

The finished product

After doing their research, Jin and Sarvnaz decided to build a bracelet instead of a chest camera.

The wristband has a front camera that recognizes ASL.

Right-handers wear it on their left hand and use their right to do sign language in front of the camera.

The device then uses specially designed computer software to translate aloud into English.

Lefties do the opposite.

The hardware of the ConchShell bracelet consists of a microcomputer called Raspberry Pi Zero W. It is connected by a wire to another microcomputer which acts as a camera. (Image submitted by Jin Schofield)

The future of ConchShell

There’s still work to be done to improve their invention, but Jin and Sarvnaz get a lot of encouragement.

In August 2020, the girls won the pitch competition for The DMZ, a tech startup program at Ryerson University in Toronto, and won $5,000.

They won $10,000 a year later when they won the Youth Impact Challenge, a national competition run by two Canadian companies that funds companies with positive environmental or social impacts.

The teens say they hope to use the money to continue experimenting with the device.

The biggest problem they face is how to make it fully functional, Sarvnaz said.

“Increasing accuracy,” she said, “is definitely something that takes time.”

What’s next for the girls

Sarvnaz said she hopes to attend college majoring in engineering or computer science, while Jin plans to major in life sciences and computer science.

The girls say they complement each other and hope to continue working as a team.

This includes a day of putting ConchShell wristbands on the arms of those who need them most.


TOP IMAGE CREDIT: Image submitted by Jin Schofield, graphic design by Philip Street/CBC