Mark Zuckerberg announced earlier this year that he wanted to build a simple AI that could control his home, including his lights, temperature, appliances, music and security. He also wanted it to "learn his tastes and patterns, learn new words and concepts, and even entertain Max" (his daughter.) He ended 2016 completing his personal challenge to build a Jarvis-style AI to run his home.
He also published a blog post explaining how he did it.
Zuckerberg's Jarvis uses a lot of artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C.
"Before I could build any AI, I first needed to write code to connect these systems, which all speak different languages and protocols," Zuckerberg explained. "I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.
"Further, most appliances aren't even connected to the internet yet. It's possible to control some of these using internet-connected power switches that let you turn the power on and off remotely. But often that isn't enough."
For example, he said it's hard to find a toaster that will let you push the bread down while it's switched off so he had to find a 1950s toaster and rig it up with a connected switch. Elsewhere, he had to modify a food dispenser to feed his dog Beast and a t-shirt cannon to deliver his iconic grey shirts.
"For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other," Zuckerberg continued.
"I programmed Jarvis on my computer, but in order to be useful I wanted to be able to communicate with it from anywhere I happened to be. That meant the communication had to happen through my phone, not a device placed in my home," he said.
He began by building a messenger bot to communicate with Jarvis "because it was so much easier than building a separate app". He now texts the Jarvis bot and it can translate audio clips into commands. In the middle of the day, if someone arrives at his home, Jarvis also texts him an image to tell him who's there, or it can text him when he needs to go do something.
"I have always been optimistic about AI bots, but my experience with Jarvis has made me even more optimistic that we'll all communicate with bots like Jarvis in the future."
Zuckerberg has developed the first version of the Jarvis app for iOS and plans to build an Android version soon too.
"In the longer term, I'd like to explore teaching Jarvis how to learn new skills itself rather than me having to teach it how to perform specific tasks. If I spent another year on this challenge, I'd focus more on learning how learning works," Zuckerberg continued.
"Finally, over time it would be interesting to find ways to make this available to the world. I considered open sourcing my code, but it's currently too tightly tied to my own home, appliances and network configuration. If I ever build a layer that abstracts more home automation functionality, I may release that. Or, of course, that could be a great foundation to build a new product."
With this a virtual assistant from Facebook is definitely coming soon .
Need Tech Help? Click Here To Join Our Live Online Radio Show
Wanna Leave Without Sharing This? It's not fair