Building an AI Voice Assistant with M5Stack CoreS3 SE and LLM Module Kit: A Complete Tutorial

The world of edge AI is rapidly evolving, and M5Stack has made it accessible with their CoreS3 SE and LLM Module Kit combination. This tutorial walks through building a fully offline AI voice assistant — no internet connection, no cloud API, just local speech recognition, language model inference, and text-to-speech running on the hardware itself.

What You'll Need

Hardware

• M5Stack CoreS3 SE IoT Controller ($38.90) — ESP32-S3, 2.0" capacitive touchscreen, built-in speaker/mic. A lighter version of the CoreS3 with the camera, proximity sensor, IMU, and compass stripped out to hit a lower price point.
• M5Stack LLM Module Kit ($49.90) — the LLM module (AX630C processor) plus the LLM Mate module for connectivity.

Software

• Arduino IDE
• M5Stack LLM library (M5Module-LLM)
• USB-C cable

The Hardware, Briefly

CoreS3 SE: ESP32-S3 dual-core Xtensa LX7 @ 240MHz, 16MB Flash, 8MB PSRAM, 2.0" IPS touchscreen (320×240), built-in 1W speaker and dual mics, WiFi, USB-C with OTG, AXP2101 power management.

LLM Module Kit: AX630C SoC, dual Cortex-A53 @ 1.2GHz, 3.2 TOPS @ INT8, 4GB LPDDR4 (1GB system / 3GB AI acceleration), 32GB eMMC with Ubuntu preinstalled. Handles wake word (KWS), speech recognition (ASR), language model inference, and text-to-speech — all on-device, at roughly 1.5W.

Setup

1. Install the latest Arduino IDE and add the M5Stack board package.
2. Select "M5Stack CoreS3" as the target board.
3. Install the "M5Module LLM" library from the Library Manager.

The Code

This is the working example from M5Stack's own repo, unmodified:

/*
 * SPDX-FileCopyrightText: 2024 M5Stack Technology CO LTD
 * SPDX-License-Identifier: MIT
 */
#include <Arduino.h>
#include <M5Unified.h>
#include <M5ModuleLLM.h>

M5ModuleLLM module_llm;
M5ModuleLLM_VoiceAssistant voice_assistant(&module_llm);

void on_asr_data_input(String data, bool isFinish, int index)
{
    M5.Display.setTextColor(TFT_GREEN, TFT_BLACK);
    M5.Display.printf(">> %s\n", data.c_str());
    if (isFinish) {
        M5.Display.setTextColor(TFT_YELLOW, TFT_BLACK);
        M5.Display.print(">> ");
    }
};

void on_llm_data_input(String data, bool isFinish, int index)
{
    M5.Display.print(data);
    if (isFinish) {
        M5.Display.print("\n");
    }
};

void setup()
{
    M5.begin();
    M5.Display.setTextSize(2);
    M5.Display.setTextScroll(true);

    int rxd = M5.getPin(m5::pin_name_t::port_c_rxd);
    int txd = M5.getPin(m5::pin_name_t::port_c_txd);
    Serial2.begin(115200, SERIAL_8N1, rxd, txd);

    module_llm.begin(&Serial2);

    M5.Display.printf(">> Check ModuleLLM connection..\n");
    while (1) {
        if (module_llm.checkConnection()) {
            break;
        }
    }

    M5.Display.printf(">> Begin voice assistant..\n");
    int ret = voice_assistant.begin("HELLO");
    if (ret != MODULE_LLM_OK) {
        while (1) {
            M5.Display.setTextColor(TFT_RED);
            M5.Display.printf(">> Begin voice assistant failed\n");
        }
    }

    voice_assistant.onAsrDataInput(on_asr_data_input);
    voice_assistant.onLlmDataInput(on_llm_data_input);

    M5.Display.printf(">> Voice assistant ready\n");
}

void loop()
{
    voice_assistant.update();
}

What's actually happening

• Two callbacks do the work. on_asr_data_input fires as speech gets recognized (shown in green, then yellow once the utterance is done). on_llm_data_input fires as the model streams its response back token by token — that's the typewriter effect on screen.
• Serial2 talks to the module. M5.getPin() auto-resolves the correct RX/TX pins for whatever M5Stack board you're on, so you don't have to hardcode them.
• voice_assistant.begin("HELLO") sets the wake word and loads the model. Swap the string for Chinese wake word/ASR: begin("你好你好", "", "zh_CN").
• The main loop is just update(). Audio capture, ASR, LLM inference, and TTS all happen inside the library — your sketch just has to get out of the way.

Flashing and Assembly

• Compile time: roughly 1-2 minutes depending on your machine.
• Power both devices off, align the LLM module with the CoreS3 SE's M5Bus connector, press until it clicks. Power on — the LLM module's LED should go green.

Testing It

On boot you'll see three lines: "Check ModuleLLM connection.." → "Begin voice assistant.." → "Voice assistant ready." Then try:

• "Hello" → "Hi, how can I help you today?"
• "Hello, what is your name?" → "I'm a large language model created by Qwen"
• "Hello, translate 'How are you?' to Spanish" → "¿Cómo estás?"

Performance

• Standby: 104.64µA @ 4.2V (battery). Active: 166.27mA @ 5V (USB). AI inference: ~1.5W.
• Wake word: instant. ASR: 1-2 sec for short phrases. Simple LLM response: 2-4 sec. Detailed responses: 5-10 sec.
• Memory: 1GB reserved for system, 3GB for inference and caching, out of 4GB total.

Troubleshooting

• Stuck on "Check ModuleLLM connection.." — check the module is seated and aligned on the M5Bus connector, both devices are powered, and the LLM module's LED is lit. Reseat if unsure.
• Pin errors on compile — M5.getPin() should handle this automatically, but if not: CoreS3 is RXD 18 / TXD 17, Core2 is RXD 13 / TXD 14, Basic is RXD 16 / TXD 17.
• Slow or no response — give it 30-60 sec on first boot for full initialization, speak clearly, keep queries short, and use USB power rather than battery during development.
• No speech recognized — check the mics aren't blocked, say "HELLO" clearly as the wake word, confirm the speaker outputs audio, and test somewhere quiet first.

Taking It Further

A few concrete extensions once the base example is working:

Custom wake word

int ret = voice_assistant.begin("JARVIS");  // instead of "HELLO"

Connection timeout instead of an infinite wait

int connection_attempts = 0;
while (connection_attempts < 30) {
    if (module_llm.checkConnection()) break;
    delay(1000);
    connection_attempts++;
    M5.Display.printf("Attempt %d/30\n", connection_attempts);
}
if (connection_attempts >= 30) {
    M5.Display.setTextColor(TFT_RED);
    M5.Display.printf(">> Connection failed after 30 seconds\n");
    while(1) delay(1000);
}

Acting on the response, not just displaying it

void on_llm_data_input(String data, bool isFinish, int index) {
    static String complete_response = "";
    complete_response += data;
    M5.Display.print(data);
    if (isFinish) {
        M5.Display.print("\n");
        if (complete_response.indexOf("temperature") >= 0) {
            displayTemperature();  // your own function
        }
        complete_response = "";
    }
}

The LLM module also supports model swaps (Qwen2.5-0.5B/1.5B, Llama-3.2-1B, plus vision and speech models) via apt update && apt upgrade on its own Ubuntu system, and OTA firmware updates over WiFi — worth exploring once the base build is solid.

Why This Matters

Fully offline voice AI on a ~$90 stack, running at 1.5W, is a genuinely new capability for makers — no API keys, no latency to a cloud endpoint, no monthly bill, and it keeps working even if your WiFi doesn't. That's the difference between a demo you show people once and something you'd actually wire into a project that has to keep working reliably in a garage or a field.