Who is the fairest of them all….

Inspired by a number of Raspberry Pi projects already out there in the wild (oh and a very persistent eight year old), the concept was to build a magic mirror which you could ask

Who is the fairest of them all

… but the catch was that it should recognise when you are standing in front of it and interact with you to give it a more intuitive feel.

This eventually broke down into two tasks:

How do you recognise when someone is standing in front of the mirror.
How can you talk to the mirror.

Is there anybody there ?

To recognise if anyone was standing in front of the camera we used an old PS3 camera (as this could both see and later will be able to hear what questions we ask)

To recognise someone standing in front of the camera is a bit harder. For this we used OpenCV

First task was to produce a Raspberry Pi image which contained the latest version of Raspbian and OpenCV.

Then add some python to recognise if a face is standing in front of the camera.

Are we all listening…. then i shall begin

When you have a conversation you listen and respond based on what the other person is saying. This is important as you base you next response on what the person has said previously. This provides the context of the conversation. If you have context your response can become a reply. This feels intuitive which is what we are aiming for.

In this instance to hold a conversation with context we can use Dialogflow

This means we can ask questions, gain replies and maintain a conversational context. Once complete we have a conversation where you will be asked your name and then you can ask the all important question “Who is the fairest of them all”

As an aside DialogFlow works closely with Google Home so with some additional configuration we have the Princess Mirror which is available through Google Assistant.

Connecting the dots

So now we have a PI that can recognise a face and contextual conversation hosted by Google Assistant. Next we have to join these together. We want google to respond when a face is recognised in front of the mirror. (This is the MVP, later we can train the face recognition with some machine learning to distinguish between certain individuals).