Portfolio

Jan 17th, 2017

Seeing the growth of Internet Of Things and Artificial Intelligence, I begin doing a Masters degree that covers both these areas. One of the products I built was a Smart Mirror.

This project achieved the transformation of an ordinary household mirror into a smart object; thereby increasing productivity by providing a wide range of functionality and interaction for various users. Typically, this would entail enabling a user to receive relevant information pertaining to everyday tasks performed before, after and during the mirror is typically used.

Infrared & Facial Recognition

PIR Sensor

The mirror has a built-in infrared sensor at the front. The sensor enables it to detect when a person is near. Once someone is detected, it comes out of its idle state and begins facial recognition to determine if the user has a profile.

If the current user does not have a profile, they are taken through the registration process.

Facial Recognition

It also boasts facial recognition features. By taking several images of the user’s face it determines its structure and returns a map based on this structure. I then generate a token and store this token in a database which is then consumed by a classifier which associates this map with a specific user.
As someone’s face is being recognised, it goes through this same process except it checks the classifier and determines if this user’s face as a token exists.

Speech, Text & Intent Recognition

Speech to Text

A speech to text API interprets spoken language in an understandable format. This text then goes into an interpretation API which is populated by myself to return an intent for each query. E.g. What time is it? – the intent of this question is to tell me the current time.

Text to Speech & Intent Recognition

Once the intent is determined it’s then able to display or retrieve information to be displayed using the temperature sensor or other APis. This information is then built into a template and shown and spoken to the user.

The mirror is able to speak using pre built speech algorithms built directly into the server

UI Demo

https://www.jabari-holder.com/wp/wp-content/uploads/2017/02/smartmirror.m4v

 

A very basic demonstration showing a few views with no audio.

Technology & Tools

This project branched into many varying aspects of modern web development technologies and a few tools to aid in greater productivity.

  • HTML5
  • CSS3 (SASS / LESS)
  • JS (Vanilla / OO)
  • jQuery
  • NodeJS
  • AJAX / JSON
  • Mobile
  • UX / UI
  • MySQL
  • SVN / GIT
  • Python
  • Usability & Accessibility
  • Logo Design
  • Wireframes
  • RaspberryPi
  • BBCMicrobit
  • AI
  • Adobe Photoshop
  • Adobe Illustrator
  • SublimeText 3
  • Coda
  • Microsoft Office Suite