Maker Pro
Maker Pro

Project Ideas: Smart Home Hub

BlueObsecurit

Mar 24, 2021
36
Joined
Mar 24, 2021
Messages
36
Been toying with the idea of making my own voice responsive smart home hub based on arduino.
I have a couple of ideas of what I want to do, but I will keep an example simple:

I have a strip of WS2812 LEDs connected to a Pimoroni Plasma: https://shop.pimoroni.com/products/plasma-2040

So I want something to sit in my living room that I can program to do something with the light if I say something to it. Normally I'd think that out of reach for me, however I came across this product from Expressif: https://www.adafruit.com/product/5290

I can't make head or tail of wether or not this will be able to do what I want it to within my capacity to code it. It seems to have a simple setup for making voice commands, but it also seems this isn't user configurable and that there is a hard coded list of commands if I plan to use it this way, but there's no way to know from the documentation what limitations are there. (To the best of my ability to attempt to figure it out) Anyone have any experience with this software? It is just so convenient for what I envision, bringing together microcontroller and screen, power etc

I was thinking of maybe using 433MHz modules to communicate wirelessly with nodes around my house, but this may be a bad idea? Anything better to use?

To sum up, I want to make something that will take a (somewhat configurable) voice command and translate it to doing something with GPIO pins that I can use to send commands to nodes around my house. Is the ESP32 S3 BOX suitable? Are there better ideas?
 

Harald Kapp

Moderator
Moderator
Nov 17, 2011
13,700
Joined
Nov 17, 2011
Messages
13,700
Imho an arduino has not enough computing power to do speech recognition. A quick internet search turned up some projects that use external speech recognition via dedicated speech recognition modules or connection to a server that does the speech recognition. For more info do a web search using "arduino speech recognition".
A raspberry Pi on the other is a completely different matter. It has enough power to do the speech recognition plus the control functions for home automation using its GPIO pins. It can also easily be integrated in your local network via WIFi or Ethernet.

The ESP32 box from your link looks promising. The ESP32 has more power than an arduino. Whether it is sufficient for your purposes I can't say. The website says it does, so why not give it a try? Less expensive than a Raspberry plus comes with a nice case.

I was thinking of maybe using 433MHz modules to communicate wirelessly with nodes around my house, but this may be a bad idea? Anything better to use?
Not necessarily a bad idea, but why not use your local WiFi? The ESP-box (or a Raspberry) can be integrated into your existing WIFi plus you can use all the existing network hardware like access points, range extenders etc. to access the smart home from anywhere in your home.
 

BlueObsecurit

Mar 24, 2021
36
Joined
Mar 24, 2021
Messages
36
A raspberry Pi on the other is a completely different matter.
Arduino was really the wrong word to use, though I'm not sure what is better. Programmable in C maybe? In any case I can handle Arduino IDE and language, and probably will be able to learn something that is new, but as with all of us, time and resources are limited.

Raspberry Pi is a good idea but there are pros and cons and on balance this isn't the best solution for me. (Big, more expensive, need a screen, more power draw, OS software updates etc) But like you say, more powerful and probably easier to set up. Possibly in the future.

Reading the ESP box example again, it does say you can customise voice commands with a proper example. (I looked up and down for this and still missed it, my bad)

but why not use your local WiFi?
This seems like a very good idea and eliminates some possible hurdles. I have not done anything with microcontrollers and wifi. The S3 box is another case and I will have to work it out by myself. But for something like Arduino it looks like maybe the best way to do this is sending/receiving a UDP string? What kind of WiFi module and library is something you might use yourself?
 

Harald Kapp

Moderator
Moderator
Nov 17, 2011
13,700
Joined
Nov 17, 2011
Messages
13,700
What kind of WiFi module and library is something you might use yourself?
The S3 bos has built-in Wifi:
The ESP32-S3-BOX is an AIoT development board that is based on Espressif’s ESP32-S3 Wi-Fi + Bluetooth 5 (LE) SoC.
No need for an additional module. As for the library: use what comes with the ESP32. I've tinkered with the ESP32-CAM module and other ERSP-based modules. It was child's play to get Wifi up and running on these using the examples you find online.
it looks like maybe the best way to do this is sending/receiving a UDP string?
UDP is rather insecure. Use HTTP or MQTT (MQTT being kind of a standard for home automation).
 

BlueObsecurit

Mar 24, 2021
36
Joined
Mar 24, 2021
Messages
36
The S3 bos has built-in Wifi:
Yes, I got that :)
I meant for the other end of this, the Pimoroni Plasma (that I am coding in Arduino IDE) RP2040 chip.

If you have a quick look at https://github.com/espressif/esp-bo...md#voice-assistance-control-and-customization

It seems to me that maybe the software included allows the user to create new commands, but as far as I can see there is no reading material on how/if that is the way it works (So I am not able to tell what limitations there are). IMO the getting started guide is not very good and I can't really understand how you are supposed to work with this thing. It both seems like it is supposed to be coded on a computer in the Expressif IDF and also seems like they have made a firmware that allows you to just handle the easy stuff. Actually I find there is both a user guide (linked above) and a developer guide that is way too much for me to wrap my head around atm. I guess it is not something anybody can really answer though without the device in their hands. Hopefully this makes some sense?
 

Harald Kapp

Moderator
Moderator
Nov 17, 2011
13,700
Joined
Nov 17, 2011
Messages
13,700
It seems to me that maybe the software included allows the user to create new commands, but as far as I can see there is no reading material on how/if that is the way it works
See here. It is stated that
Currently, Espressif MultiNet supports up to 200 Chinese or English speech commands, such as “打开空调” (Turn on the air conditioner) and “打开卧室灯” (Turn on the bedroom light).
Seems that you'll have to put in some work to define your own commands.
 

BlueObsecurit

Mar 24, 2021
36
Joined
Mar 24, 2021
Messages
36
Seems that you'll have to put in some work to define your own commands.
I have seen that. But how do you define what the commands does? Maybe you first define the command like that then by using a phone you can define what it does as with the getting started guide? But can you have it send commands over WiFi (without coding all that in yourself) or maybe sending something over serial to a slave?
I also belive this is refering to the offline version, not sure what they really mean by it. Maybe it isn't a difference in functionality of the voice command and they are refering to something else. Either way I believe to some respect, trial and error is probably a good idea...
 

Harald Kapp

Moderator
Moderator
Nov 17, 2011
13,700
Joined
Nov 17, 2011
Messages
13,700
As far as I understand it (from what little I've had time to read) this is a 2-step process:
  1. Have the software (the AI model) recognize the speech and translate it into a code. This training seems o be done on a PC.
  2. Download the new model to the ESP32 and its firmware, then assign a command to the recognized speech.
Basically it doesn't matter (at least not for the ESP, certainly it does for you) which command is assigned to which action. You could "ab"use the built-in commands to do very different things. E.g. instead of turning LEDs on or off, you could use the same commands to trigger e.g. a relay which in turn opens a garage door. But obviously for the human operator it would be much easier to have commands like "open garage" and "close garage" instead, which, if not available with the built-in model, will have to be trained to the software model.
 
Top