Maker Pro
NXP FRDM

Baby monitor and care AI Robot

SB
August 31, 2024 by Sindhu B
Share
banner

Baby care robot is a multi-functional device, which can not only take care of the baby, but also ensure the safety of the baby efficiently. It can be interactive and social robot specifically designed to assist parents in the hard task of taking care of their children. AI-powered robot help parents to watch what the baby is doing from anywhere.

A baby monitor can help to listen or look out baby from a distance. For example, one can hear baby cry if parents are at downstairs or in another room where they cannot easily hear them. I am going to design a BABY MONITOR AND CARE ROBOT with FRDM-MCXN947 board which has on-board camera header to interface camera to look over the child position and MikroBUS header to interface Mic Click which is a compact add-on board equipped with a microphone accompanied by a suitable amplifier, a WiFi and BLE Arduino module to transfer data via thread or matter. FRDM-MCXN947 has neural processing unit to process image, video, voice for AI.

Hardware: FRDM Development Board for MCX N94/N54 MCUs


FRDM-MCXN947 are compact and scalable development boards for rapid prototyping of MCX N94 and N54 MCUs. They offer industry standard headers for easy access to the MCU’s I/Os, integrated open-standard serial interfaces, external flash memory and an on-board MCU-Link debugger. Additional tools like our Expansion Board Hub for add-on boards and the Application Code Hub for software examples are available through the MCUXpresso Developer Experience.

CMOS OV7670 Camera Module

The OV7670/OV7171 provides full-frame, sub-sampled or windowed 8-bit images in a wide range of formats, controlled through the Serial Camera Control Bus (SCCB).



SOFTWARE: MCUXpresso Integrated Development Environment (IDE)

 The MCUXpresso IDE brings developers an easy-to-use Eclipse-based development environment for NXP MCUs based on Arm Cortex-M cores, including its general purpose crossover and wireless - enabled MCUs. The MCUXpresso IDE offers advanced editing, compiling, and debugging features with the addition of MCU-specific debugging views, code trace and profiling, multicore debugging, and integrated configuration tools. The MCUXpresso IDE debug connections support all general purpose Arm Cortex-M based EVKs and your custom development boards with optimized open-source and commercial debug probes from NXP, P&E Micro, and SEGGER.

MCUXpresso Integrated Development Environment (IDE) download

https://www.nxp.com/webapp/swlicensing/sso/downloadSoftware.sp?catid=MCUXPRESSO

MCUXpresso IDE 11.10.0 Installation Guide

 

eIQ ML Software Development Environment

The NXP eIQ machine learning (ML) software development environment enables the use of ML algorithms on NXP EdgeVerse microcontrollers and microprocessors, including i.MX RT crossover MCUs, and i.MX family application processors. eIQ ML software includes a ML workflow tool called eIQ Toolkit, along with inference engines, neural network compilers and optimized libraries. This software leverages open-source and proprietary technologies and is fully integrated into our MCUXpresso SDK and Yocto development environments, allowing you to develop complete system-level applications with ease.

The CIFAR-10 dataset

The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.

Here are the classes in the dataset, as well as 10 random images from each:

The CIFAR-100 dataset

This dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each. There are 500 training images and 100 testing images per class. The 100 classes in the CIFAR-100 are grouped into 20 superclasses. Each image comes with a "fine" label (the class to which it belongs) and a "coarse" label (the superclass to which it belongs).

Here is the list of classes in the CIFAR-100:

Superclass: Classes

aquatic mammals: beaver, dolphin, otter, seal, whale

fish: aquarium fish, flatfish, ray, shark, trout

flowers: orchids, poppies, roses, sunflowers, tulips

food containers: bottles, bowls, cans, cups, plates

fruit and vegetables: apples, mushrooms, oranges, pears, sweet peppers

household electrical devices: clock, computer keyboard, lamp, telephone, television

household furniture: bed, chair, couch, table, wardrobe

insects: bee, beetle, butterfly, caterpillar, cockroach

large carnivores: bear, leopard, lion, tiger, wolf

large man-made outdoor things: bridge, castle, house, road, skyscraper

large natural outdoor scenes: cloud, forest, mountain, plain, sea

large omnivores and herbivores: camel, cattle, chimpanzee, elephant, kangaroo

medium-sized mammals: fox, porcupine, possum, raccoon, skunk

non-insect invertebrates: crab, lobster, snail, spider, worm

people: baby, boy, girl, man, woman

reptiles: crocodile, dinosaur, lizard, snake, turtle

small mammals: hamster, mouse, rabbit, shrew, squirrel

trees: maple, oak, palm, pine, willow

vehicles 1: bicycle, bus, motorcycle, pickup truck, train

vehicles 2: lawn-mower, rocket, streetcar, tank, tractor


Example application with MCXN947 for ML:

Train and Deploy Customer ML model to NPU

Part 1: Introduction

The eIQ Neutron Neural Processing Unit (NPU) is a highly scalable accelerator core architecture that provides machine learning (ML) acceleration. The eIQ Neutron NPU offers up to 42 times faster machine learning inference performance compared to a standalone CPU core. Specifically, the MCX N94 can execute 4.8G (150MHz * 4 * 4 * 2) INT8 operations per second. The eIQ Portal, developed in exclusive partnership with Au-Zone Technologies, is an intuitive graphical user interface (GUI) that simplifies vision based ML solutions development. Developers can create, optimize, debug and export ML models, as well as import datasets and models, rapidly train and deploy neural network models and ML workloads for vision applications.

Hardware Environment:

Development Board: FRDM-MCXN947

Display: 3.5" TFT LCD (Part Number: PAR-LCD-S035)

Camera: OV7670

Software Environment:

eIQ Portal: eIQ® ML Software Development Environment | NXP Semiconductors

MCUXpresso IDE v11.9.0

Application Code Hub Demo: Label CIFAR10 image

Part 2: Basic Model Classification Training and Deployment

The main content is divided into three steps: model training, model converting, and model deployment. following step to follow

1. Dataset Preparation

2. Create Project and Import Dataset into eIQ

3. Select Base Model for Training

4. Model Evaluation "VALIDATE" 

5. Model Export to TensorFlow Lite

6. Convert to TensorFlow Lite for Neutron (.tflite)

7. Deploy the Model to the Label CIFAR10 Image Project

Part 3: Experimental Results

 

Part 4: Summary

By efficiently utilizing the powerful performance of the eIQ Neutron NPU and the convenient tools of the eIQ Portal, developers can significantly streamline the entire process from model training to deployment. This not only accelerates the development cycle of machine learning applications but also enhances their performance and reliability. Therefore, for developers looking to implement efficient machine learning applications on MCX N-series edge devices, mastering these technologies and tools is crucial.

for more details follow the URL https://community.nxp.com/t5/MCX-Microcontrollers-Knowledge/MCXN947-How-to-Train-and-Deploy-Customer-ML-model-to-NPU/ta-p/1899497

ML Project with FRDM-MCXN947 for Baby monitor and care:

The dataset CIFAR 10 and CIFAR 100 is modified and included the baby behavior like "Baby cry", "baby sleep", "baby walk", "baby crawl". this actions help the mater to known what the baby status.

Modify the "labels[baby cry, baby crawl, baby sleep, baby walk]" array to match the order of labels displayed in the dataset in eIQ, as shown below:

Once the data is tested and trained hen it is ready to find the baby activity and can remotely monitor by mother and the same results are shown in the following figures:


With this detection mother get alert on the baby status so that she can be take necessary further care.

 /*  * Copyright 2020-2022 NXP  * All rights reserved.  *
 * SPDX-License-Identifier: BSD-3-Clause  */

#include "board_init.h"
#include "demo_config.h"
#include "demo_info.h"
#include "fsl_debug_console.h"
#include "image.h"
#include "image_utils.h"
#include "model.h"
#include "output_postproc.h"
#include "timer.h"
#include "video.h"
#include "ov7670.h"

int main(void)
{
    BOARD_Init();
    TIMER_Init();


    DEMO_PrintInfo();


    Ov7670_Init();

    display_init();

    ezh_start();

    cifar10_recognize();
    
    while(1)
    {

    }
}

Conclusion:

FRDM-MCXN947 has neural processing unit to process image, video, voice for AI. This kind of devices are very much helpful to mothers when they are at work also it can be used for elder people monitoring, patient condition in hospital, etc.

Contributors

Related Content

Tags

Comments


You May Also Like