基于小智ESP32S3的智能控制小车
本文介绍了一种基于小智AI语音助手的低成本智能小车控制方案。使用ESP32开发套件搭配L298N电机驱动模块,通过GPIO口实现语音控制小车的运动。文章详细说明了硬件连接方式,包括电机与ESP32的引脚映射关系,并解释了为何需要L298N模块来实现"低压控制高压"的驱动原理。重点介绍了如何在小智AI源代码中添加小车控制功能模块,包括电机初始化、基本运动控制(前进、后退、转向等)
不同于传统的语音控制小车,这次我采用虾哥开源的小智AI控制小车,实现真正的智能语音交互控制,定制你的专属小车控制助手!最重要的是成本极低。

1.材料准备
a.小智ESP32开发套件,最后需要将小智语音助手搭建好,详细教程可参考虾哥百科全书:78/xiaozhi-esp32: An MCP-based chatbot | 一个基于MCP的聊天机器人
b.亚克力小车底板(四个直流电机+四个轮子)

c.L298N四路电机驱动模块

d.杜邦线若干+电池2节

e.最后组装好的实物:

2.工作流程
ESP32S3通过GPIO口发送电机控制信号给L298N驱动模块,然后L298N驱动电机运转。为什么需要使用L298N电机驱动模块呢,而不是直接将电机与esp32s3的GPIO口连接?
因为esp32GPIO口输出电压很小无法驱动电机运转L298N 可以外接独立的高电压电源(如 7-12V 电池)给电机供电,同时接收 ESP32 的 3.3V 控制信号,实现 “低压控制高压”,既满足电机动力需求,又不影响 esp32s3 的正常工作。
3. 硬件电路连接
我画了一个简单的硬件线路连接图(●ˇ∀ˇ●)。你可以按照的我接线方式连接硬件,当然也可以连接自己选择的GPIO口,只不过你需要记住各个引脚的映射关系方便后续的程序修改。
需要注意的是:
a.左侧的ESP32S3不只有这一个开发板,至少还要有数字麦克风+功放喇叭才能实现语音控制小车。
b.L298N 需要外接独立的高电压电源(如 7-12V 电池),2节3.7v电压的电池串联即可。

连接图对应如下的映射关系:
| 电机编号 | 电机位置 | L298N IN引脚 | ESP32 GPIO |
| 电机1 | 左前轮 | IN1 | GPIO19 |
| 电机1 | 左前轮 | IN2 | GPIO20 |
| 电机2 | 右前轮 | IN3 | GPIO3 |
| 电机2 | 右前轮 | IN4 | GPIO46 |
| 电机3 | 左后轮 | IN5 | GPIO9 |
| 电机3 | 左后轮 | IN6 | GPIO10 |
| 电机4 | 右后轮 | IN7 | GPIO11 |
| 电机4 | 右后轮 | IN8 | GPIO12 |
4.小智AI程序修改
好的,想必你现在已经将小车的线路连接完成了!下面我们需要为小智烧录程序,具体的烧录步骤虾哥的小智百科全书中有详细介绍。
因为我们需要实现控制小车的功能所以需要在原有78/xiaozhi-esp32: An MCP-based chatbot | 一个基于MCP的聊天机器人项目的代码上做一些更改。我这里使用的是esp-idf+cursor进行代码开发+编译烧录。
在v2.0.3版本的源代码中我们需要在main/mcp_server.cc文件中进行修改,注册一个控制小车的MCP工具,你可以将下方代码直接粘贴至你的mcp_server.cc文件中。具体的代码如下:
/*
* MCP Server Implementation
* Reference: https://modelcontextprotocol.io/specification/2024-11-05
*/
#include "mcp_server.h"
#include <esp_log.h>
#include <esp_app_desc.h>
#include <algorithm>
#include <cstring>
#include <esp_pthread.h>
#include <driver/uart.h>
#include "application.h"
#include "display.h"
#include "oled_display.h"
#include "board.h"
#include "settings.h"
#include "lvgl_theme.h"
#include "lvgl_display.h"
#define TAG "MCP"
McpServer::McpServer() {
// 立即停止所有电机,防止上电时自转
// 特别处理GPIO19,20,3,46这些可能默认为高电平的引脚
// 先设置引脚为输出模式
gpio_set_direction(GPIO_NUM_19, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_20, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_3, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_46, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_9, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_10, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_11, GPIO_MODE_OUTPUT);
gpio_set_direction(GPIO_NUM_12, GPIO_MODE_OUTPUT);
// 立即设置所有引脚为LOW,多次设置确保稳定
for (int i = 0; i < 5; i++) {
gpio_set_level(GPIO_NUM_19, 0); // IN1
gpio_set_level(GPIO_NUM_20, 0); // IN2
gpio_set_level(GPIO_NUM_3, 0); // IN3
gpio_set_level(GPIO_NUM_46, 0); // IN4
gpio_set_level(GPIO_NUM_9, 0); // IN5
gpio_set_level(GPIO_NUM_10, 0); // IN6
gpio_set_level(GPIO_NUM_11, 0); // IN7
gpio_set_level(GPIO_NUM_12, 0); // IN8
vTaskDelay(pdMS_TO_TICKS(10)); // 短暂延时
}
// 使用GPIO配置确保引脚状态
gpio_config_t config = {
.pin_bit_mask = (1ULL << GPIO_NUM_19) | (1ULL << GPIO_NUM_20) |
(1ULL << GPIO_NUM_3) | (1ULL << GPIO_NUM_46) |
(1ULL << GPIO_NUM_9) | (1ULL << GPIO_NUM_10) |
(1ULL << GPIO_NUM_11) | (1ULL << GPIO_NUM_12),
.mode = GPIO_MODE_OUTPUT,
.pull_up_en = GPIO_PULLUP_DISABLE,
.pull_down_en = GPIO_PULLDOWN_DISABLE,
.intr_type = GPIO_INTR_DISABLE,
};
gpio_config(&config);
// 最后再次确保所有引脚为LOW
gpio_set_level(GPIO_NUM_19, 0); // IN1
gpio_set_level(GPIO_NUM_20, 0); // IN2
gpio_set_level(GPIO_NUM_3, 0); // IN3
gpio_set_level(GPIO_NUM_46, 0); // IN4
gpio_set_level(GPIO_NUM_9, 0); // IN5
gpio_set_level(GPIO_NUM_10, 0); // IN6
gpio_set_level(GPIO_NUM_11, 0); // IN7
gpio_set_level(GPIO_NUM_12, 0); // IN8
ESP_LOGI(TAG, "MCP Server: All motor GPIO pins set to LOW to prevent auto-rotation");
ESP_LOGI(TAG, "Motor1 pins: GPIO19=%d, GPIO20=%d", gpio_get_level(GPIO_NUM_19), gpio_get_level(GPIO_NUM_20));
ESP_LOGI(TAG, "Motor2 pins: GPIO3=%d, GPIO46=%d", gpio_get_level(GPIO_NUM_3), gpio_get_level(GPIO_NUM_46));
}
McpServer::~McpServer() {
for (auto tool : tools_) {
delete tool;
}
tools_.clear();
}
void McpServer::AddCommonTools() {
// *Important* To speed up the response time, we add the common tools to the beginning of
// the tools list to utilize the prompt cache.
// **重要** 为了提升响应速度,我们把常用的工具放在前面,利用 prompt cache 的特性。
// Backup the original tools list and restore it after adding the common tools.
auto original_tools = std::move(tools_);
auto& board = Board::GetInstance();
// Do not add custom tools here.
// Custom tools must be added in the board's InitializeTools function.
AddTool("self.get_device_status",
"Provides the real-time information of the device, including the current status of the audio speaker, screen, battery, network, etc.\n"
"Use this tool for: \n"
"1. Answering questions about current condition (e.g. what is the current volume of the audio speaker?)\n"
"2. As the first step to control the device (e.g. turn up / down the volume of the audio speaker, etc.)",
PropertyList(),
[&board](const PropertyList& properties) -> ReturnValue {
return board.GetDeviceStatusJson();
});
AddTool("self.audio_speaker.set_volume",
"Set the volume of the audio speaker. If the current volume is unknown, you must call `self.get_device_status` tool first and then call this tool.",
PropertyList({
Property("volume", kPropertyTypeInteger, 0, 100)
}),
[&board](const PropertyList& properties) -> ReturnValue {
auto codec = board.GetAudioCodec();
codec->SetOutputVolume(properties["volume"].value<int>());
return true;
});
auto backlight = board.GetBacklight();
if (backlight) {
AddTool("self.screen.set_brightness",
"Set the brightness of the screen.",
PropertyList({
Property("brightness", kPropertyTypeInteger, 0, 100)
}),
[backlight](const PropertyList& properties) -> ReturnValue {
uint8_t brightness = static_cast<uint8_t>(properties["brightness"].value<int>());
backlight->SetBrightness(brightness, true);
return true;
});
}
#ifdef HAVE_LVGL
auto display = board.GetDisplay();
if (display && display->GetTheme() != nullptr) {
AddTool("self.screen.set_theme",
"Set the theme of the screen. The theme can be `light` or `dark`.",
PropertyList({
Property("theme", kPropertyTypeString)
}),
[display](const PropertyList& properties) -> ReturnValue {
auto theme_name = properties["theme"].value<std::string>();
auto& theme_manager = LvglThemeManager::GetInstance();
auto theme = theme_manager.GetTheme(theme_name);
if (theme != nullptr) {
display->SetTheme(theme);
return true;
}
return false;
});
}
auto camera = board.GetCamera();
if (camera) {
AddTool("self.camera.take_photo",
"Take a photo and explain it. Use this tool after the user asks you to see something.\n"
"Args:\n"
" `question`: The question that you want to ask about the photo.\n"
"Return:\n"
" A JSON object that provides the photo information.",
PropertyList({
Property("question", kPropertyTypeString)
}),
[camera](const PropertyList& properties) -> ReturnValue {
// Lower the priority to do the camera capture
TaskPriorityReset priority_reset(1);
if (!camera->Capture()) {
throw std::runtime_error("Failed to capture photo");
}
auto question = properties["question"].value<std::string>();
return camera->Explain(question);
});
}
#endif
// 小车整体控制工具 - 基于L298N四电机模块
AddTool("self.car.control",
"控制小车的整体运动。支持前进、后退、左转、右转、停止等基本动作。\n"
"Use this tool to control the car movement when user asks to move the car.\n"
"Args:\n"
" `action`: 运动动作,可选值: 'forward'(前进), 'backward'(后退), 'left'(左转), 'right'(右转), 'stop'(停止)\n"
" `duration`: 持续时间(毫秒),0表示持续运动,默认0\n"
"Return:\n"
" 返回控制结果和状态信息",
PropertyList({
Property("action", kPropertyTypeString),
Property("duration", kPropertyTypeInteger, 0, 0, 100000)
}),
[](const PropertyList& properties) -> ReturnValue {
auto action = properties["action"].value<std::string>();
int duration = properties["duration"].value<int>();
// GPIO引脚定义 - 四电机控制 (按照您的映射关系)
static bool gpio_initialized = false;
static const gpio_num_t motor_pins[4][2] = {
{GPIO_NUM_19, GPIO_NUM_20}, // 电机1: IN1, IN2 (左前轮)
{GPIO_NUM_3, GPIO_NUM_46}, // 电机2: IN3, IN4 (右前轮)
{GPIO_NUM_9, GPIO_NUM_10}, // 电机3: IN5, IN6 (左后轮)
{GPIO_NUM_11, GPIO_NUM_12} // 电机4: IN7, IN8 (右后轮)
};
// 初始化GPIO - 解决电机自转问题
if (!gpio_initialized) {
// 立即设置所有引脚为LOW,防止自转(在配置之前)
for (int i = 0; i < 4; i++) {
// 立即设置引脚为输出并设为LOW
gpio_set_direction(motor_pins[i][0], GPIO_MODE_OUTPUT);
gpio_set_direction(motor_pins[i][1], GPIO_MODE_OUTPUT);
gpio_set_level(motor_pins[i][0], 0);
gpio_set_level(motor_pins[i][1], 0);
}
// 特别处理电机1和电机2的GPIO(G19、G20、G3、G46)
// 这些GPIO可能默认为高电平
gpio_set_level(GPIO_NUM_19, 0); // IN1
gpio_set_level(GPIO_NUM_20, 0); // IN2
gpio_set_level(GPIO_NUM_3, 0); // IN3
gpio_set_level(GPIO_NUM_46, 0); // IN4
// 等待确保引脚稳定
vTaskDelay(pdMS_TO_TICKS(50));
// 然后进行完整的GPIO配置
for (int i = 0; i < 4; i++) {
gpio_config_t config = {
.pin_bit_mask = (1ULL << motor_pins[i][0]) | (1ULL << motor_pins[i][1]),
.mode = GPIO_MODE_OUTPUT,
.pull_up_en = GPIO_PULLUP_DISABLE,
.pull_down_en = GPIO_PULLDOWN_DISABLE,
.intr_type = GPIO_INTR_DISABLE,
};
ESP_ERROR_CHECK(gpio_config(&config));
// 再次确保引脚为LOW
gpio_set_level(motor_pins[i][0], 0);
gpio_set_level(motor_pins[i][1], 0);
}
// 最后再次确保所有引脚为LOW
for (int i = 0; i < 4; i++) {
gpio_set_level(motor_pins[i][0], 0);
gpio_set_level(motor_pins[i][1], 0);
}
gpio_initialized = true;
ESP_LOGI(TAG, "Car GPIO pins initialized and set to LOW to prevent auto-rotation");
ESP_LOGI(TAG, "Motor1(左前): IN1=GPIO%d, IN2=GPIO%d", motor_pins[0][0], motor_pins[0][1]);
ESP_LOGI(TAG, "Motor2(右前): IN3=GPIO%d, IN4=GPIO%d", motor_pins[1][0], motor_pins[1][1]);
ESP_LOGI(TAG, "Motor3(左后): IN5=GPIO%d, IN6=GPIO%d", motor_pins[2][0], motor_pins[2][1]);
ESP_LOGI(TAG, "Motor4(右后): IN7=GPIO%d, IN8=GPIO%d", motor_pins[3][0], motor_pins[3][1]);
ESP_LOGI(TAG, "All motors should be stopped now");
}
// 控制单个电机函数
auto control_motor = [&](int motor_index, int direction) {
if (direction > 0) {
// 正转:IN1=LOW, IN2=HIGH
gpio_set_level(motor_pins[motor_index][0], 0);
gpio_set_level(motor_pins[motor_index][1], 1);
} else if (direction < 0) {
// 反转:IN1=HIGH, IN2=LOW
gpio_set_level(motor_pins[motor_index][0], 1);
gpio_set_level(motor_pins[motor_index][1], 0);
} else {
// 停止:IN1=LOW, IN2=LOW
gpio_set_level(motor_pins[motor_index][0], 0);
gpio_set_level(motor_pins[motor_index][1], 0);
}
};
// 根据动作同时控制所有电机
if (action == "forward") {
// 前进:四个电机同时正转
control_motor(0, -1); // 电机1正转
control_motor(1, 1); // 电机2正转
control_motor(2, -1); // 电机3正转
control_motor(3, 1); // 电机4正转
ESP_LOGI(TAG, "Car moving forward - all motors rotating forward");
} else if (action == "backward") {
// 后退:四个电机同时反转
control_motor(0, 1); // 电机1反转
control_motor(1, -1); // 电机2反转
control_motor(2, 1); // 电机3反转
control_motor(3, -1); // 电机4反转
ESP_LOGI(TAG, "Car moving backward - all motors rotating backward");
} else if (action == "left") {
// 左转:1、2电机反转,3、4电机正转
control_motor(0, -1); // 电机1反转 (左前轮)
control_motor(1, 1); // 电机2反转 (右前轮)
control_motor(2, 1); // 电机3正转 (左后轮)
control_motor(3, -1); // 电机4正转 (右后轮)
ESP_LOGI(TAG, "Car turning left - motors 1,2 backward, motors 3,4 forward");
} else if (action == "right") {
// 右转:1、2电机正转,3、4电机反转
control_motor(0, 1); // 电机1正转 (左前轮)
control_motor(1, -1); // 电机2正转 (右前轮)
control_motor(2, -1); // 电机3反转 (左后轮)
control_motor(3, 1); // 电机4反转 (右后轮)
ESP_LOGI(TAG, "Car turning right - motors 1,2 forward, motors 3,4 backward");
} else if (action == "stop") {
// 停止:所有电机停止
control_motor(0, 0); // 电机1停止
control_motor(1, 0); // 电机2停止
control_motor(2, 0); // 电机3停止
control_motor(3, 0); // 电机4停止
ESP_LOGI(TAG, "Car stopped - all motors stopped");
} else {
throw std::runtime_error("Invalid action: " + action + ". Valid actions are: forward, backward, left, right, stop");
}
// 如果设置了持续时间,启动定时器停止
if (duration > 0) {
auto& app = Application::GetInstance();
app.Schedule([duration]() {
vTaskDelay(pdMS_TO_TICKS(duration));
// 停止所有电机
for (int i = 0; i < 4; i++) {
gpio_set_level(motor_pins[i][0], 0);
gpio_set_level(motor_pins[i][1], 0);
}
ESP_LOGI(TAG, "Car auto-stopped after %d ms", duration);
});
}
cJSON* result = cJSON_CreateObject();
cJSON_AddStringToObject(result, "action", action.c_str());
cJSON_AddNumberToObject(result, "duration", duration);
std::string description = "Car control: " + action + " - all motors controlled simultaneously";
cJSON_AddStringToObject(result, "description", description.c_str());
cJSON_AddStringToObject(result, "gpio_pins",
"IN1=GPIO19, IN2=GPIO20, IN3=GPIO3, IN4=GPIO46, IN5=GPIO9, IN6=GPIO10, IN7=GPIO11, IN8=GPIO12");
cJSON_AddBoolToObject(result, "success", true);
return result;
});
void McpServer::AddUserOnlyTools() {
// System tools
AddUserOnlyTool("self.get_system_info",
"Get the system information",
PropertyList(),
[this](const PropertyList& properties) -> ReturnValue {
auto& board = Board::GetInstance();
return board.GetSystemInfoJson();
});
AddUserOnlyTool("self.reboot", "Reboot the system",
PropertyList(),
[this](const PropertyList& properties) -> ReturnValue {
auto& app = Application::GetInstance();
app.Schedule([&app]() {
ESP_LOGW(TAG, "User requested reboot");
vTaskDelay(pdMS_TO_TICKS(1000));
app.Reboot();
});
return true;
});
// Firmware upgrade
AddUserOnlyTool("self.upgrade_firmware", "Upgrade firmware from a specific URL. This will download and install the firmware, then reboot the device.",
PropertyList({
Property("url", kPropertyTypeString, "The URL of the firmware binary file to download and install")
}),
[this](const PropertyList& properties) -> ReturnValue {
auto url = properties["url"].value<std::string>();
ESP_LOGI(TAG, "User requested firmware upgrade from URL: %s", url.c_str());
auto& app = Application::GetInstance();
app.Schedule([url, &app]() {
auto ota = std::make_unique<Ota>();
bool success = app.UpgradeFirmware(*ota, url);
if (!success) {
ESP_LOGE(TAG, "Firmware upgrade failed");
}
});
return true;
});
// Display control
#ifdef HAVE_LVGL
auto display = dynamic_cast<LvglDisplay*>(Board::GetInstance().GetDisplay());
if (display) {
AddUserOnlyTool("self.screen.get_info", "Information about the screen, including width, height, etc.",
PropertyList(),
[display](const PropertyList& properties) -> ReturnValue {
cJSON *json = cJSON_CreateObject();
cJSON_AddNumberToObject(json, "width", display->width());
cJSON_AddNumberToObject(json, "height", display->height());
if (dynamic_cast<OledDisplay*>(display)) {
cJSON_AddBoolToObject(json, "monochrome", true);
} else {
cJSON_AddBoolToObject(json, "monochrome", false);
}
return json;
});
#if CONFIG_LV_USE_SNAPSHOT
AddUserOnlyTool("self.screen.snapshot", "Snapshot the screen and upload it to a specific URL",
PropertyList({
Property("url", kPropertyTypeString),
Property("quality", kPropertyTypeInteger, 80, 1, 100)
}),
[display](const PropertyList& properties) -> ReturnValue {
auto url = properties["url"].value<std::string>();
auto quality = properties["quality"].value<int>();
std::string jpeg_data;
if (!display->SnapshotToJpeg(jpeg_data, quality)) {
throw std::runtime_error("Failed to snapshot screen");
}
ESP_LOGI(TAG, "Upload snapshot %u bytes to %s", jpeg_data.size(), url.c_str());
// 构造multipart/form-data请求体
std::string boundary = "----ESP32_SCREEN_SNAPSHOT_BOUNDARY";
auto http = Board::GetInstance().GetNetwork()->CreateHttp(3);
http->SetHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
if (!http->Open("POST", url)) {
throw std::runtime_error("Failed to open URL: " + url);
}
{
// 文件字段头部
std::string file_header;
file_header += "--" + boundary + "\r\n";
file_header += "Content-Disposition: form-data; name=\"file\"; filename=\"screenshot.jpg\"\r\n";
file_header += "Content-Type: image/jpeg\r\n";
file_header += "\r\n";
http->Write(file_header.c_str(), file_header.size());
}
// JPEG数据
http->Write((const char*)jpeg_data.data(), jpeg_data.size());
{
// multipart尾部
std::string multipart_footer;
multipart_footer += "\r\n--" + boundary + "--\r\n";
http->Write(multipart_footer.c_str(), multipart_footer.size());
}
http->Write("", 0);
if (http->GetStatusCode() != 200) {
throw std::runtime_error("Unexpected status code: " + std::to_string(http->GetStatusCode()));
}
std::string result = http->ReadAll();
http->Close();
ESP_LOGI(TAG, "Snapshot screen result: %s", result.c_str());
return true;
});
AddUserOnlyTool("self.screen.preview_image", "Preview an image on the screen",
PropertyList({
Property("url", kPropertyTypeString)
}),
[display](const PropertyList& properties) -> ReturnValue {
auto url = properties["url"].value<std::string>();
auto http = Board::GetInstance().GetNetwork()->CreateHttp(3);
if (!http->Open("GET", url)) {
throw std::runtime_error("Failed to open URL: " + url);
}
int status_code = http->GetStatusCode();
if (status_code != 200) {
throw std::runtime_error("Unexpected status code: " + std::to_string(status_code));
}
size_t content_length = http->GetBodyLength();
char* data = (char*)heap_caps_malloc(content_length, MALLOC_CAP_8BIT);
if (data == nullptr) {
throw std::runtime_error("Failed to allocate memory for image: " + url);
}
size_t total_read = 0;
while (total_read < content_length) {
int ret = http->Read(data + total_read, content_length - total_read);
if (ret < 0) {
heap_caps_free(data);
throw std::runtime_error("Failed to download image: " + url);
}
if (ret == 0) {
break;
}
total_read += ret;
}
http->Close();
auto image = std::make_unique<LvglAllocatedImage>(data, content_length);
display->SetPreviewImage(std::move(image));
return true;
});
#endif // CONFIG_LV_USE_SNAPSHOT
}
#endif // HAVE_LVGL
// Assets download url
auto& assets = Assets::GetInstance();
if (assets.partition_valid()) {
AddUserOnlyTool("self.assets.set_download_url", "Set the download url for the assets",
PropertyList({
Property("url", kPropertyTypeString)
}),
[](const PropertyList& properties) -> ReturnValue {
auto url = properties["url"].value<std::string>();
Settings settings("assets", true);
settings.SetString("download_url", url);
return true;
});
}
}
void McpServer::AddTool(McpTool* tool) {
// Prevent adding duplicate tools
if (std::find_if(tools_.begin(), tools_.end(), [tool](const McpTool* t) { return t->name() == tool->name(); }) != tools_.end()) {
ESP_LOGW(TAG, "Tool %s already added", tool->name().c_str());
return;
}
ESP_LOGI(TAG, "Add tool: %s%s", tool->name().c_str(), tool->user_only() ? " [user]" : "");
tools_.push_back(tool);
}
void McpServer::AddTool(const std::string& name, const std::string& description, const PropertyList& properties, std::function<ReturnValue(const PropertyList&)> callback) {
AddTool(new McpTool(name, description, properties, callback));
}
void McpServer::AddUserOnlyTool(const std::string& name, const std::string& description, const PropertyList& properties, std::function<ReturnValue(const PropertyList&)> callback) {
auto tool = new McpTool(name, description, properties, callback);
tool->set_user_only(true);
AddTool(tool);
}
void McpServer::ParseMessage(const std::string& message) {
cJSON* json = cJSON_Parse(message.c_str());
if (json == nullptr) {
ESP_LOGE(TAG, "Failed to parse MCP message: %s", message.c_str());
return;
}
ParseMessage(json);
cJSON_Delete(json);
}
void McpServer::ParseCapabilities(const cJSON* capabilities) {
auto vision = cJSON_GetObjectItem(capabilities, "vision");
if (cJSON_IsObject(vision)) {
auto url = cJSON_GetObjectItem(vision, "url");
auto token = cJSON_GetObjectItem(vision, "token");
if (cJSON_IsString(url)) {
auto camera = Board::GetInstance().GetCamera();
if (camera) {
std::string url_str = std::string(url->valuestring);
std::string token_str;
if (cJSON_IsString(token)) {
token_str = std::string(token->valuestring);
}
camera->SetExplainUrl(url_str, token_str);
}
}
}
}
void McpServer::ParseMessage(const cJSON* json) {
// Check JSONRPC version
auto version = cJSON_GetObjectItem(json, "jsonrpc");
if (version == nullptr || !cJSON_IsString(version) || strcmp(version->valuestring, "2.0") != 0) {
ESP_LOGE(TAG, "Invalid JSONRPC version: %s", version ? version->valuestring : "null");
return;
}
// Check method
auto method = cJSON_GetObjectItem(json, "method");
if (method == nullptr || !cJSON_IsString(method)) {
ESP_LOGE(TAG, "Missing method");
return;
}
auto method_str = std::string(method->valuestring);
if (method_str.find("notifications") == 0) {
return;
}
// Check params
auto params = cJSON_GetObjectItem(json, "params");
if (params != nullptr && !cJSON_IsObject(params)) {
ESP_LOGE(TAG, "Invalid params for method: %s", method_str.c_str());
return;
}
auto id = cJSON_GetObjectItem(json, "id");
if (id == nullptr || !cJSON_IsNumber(id)) {
ESP_LOGE(TAG, "Invalid id for method: %s", method_str.c_str());
return;
}
auto id_int = id->valueint;
if (method_str == "initialize") {
if (cJSON_IsObject(params)) {
auto capabilities = cJSON_GetObjectItem(params, "capabilities");
if (cJSON_IsObject(capabilities)) {
ParseCapabilities(capabilities);
}
}
auto app_desc = esp_app_get_description();
std::string message = "{\"protocolVersion\":\"2024-11-05\",\"capabilities\":{\"tools\":{}},\"serverInfo\":{\"name\":\"" BOARD_NAME "\",\"version\":\"";
message += app_desc->version;
message += "\"}}";
ReplyResult(id_int, message);
} else if (method_str == "tools/list") {
std::string cursor_str = "";
bool list_user_only_tools = false;
if (params != nullptr) {
auto cursor = cJSON_GetObjectItem(params, "cursor");
if (cJSON_IsString(cursor)) {
cursor_str = std::string(cursor->valuestring);
}
auto with_user_tools = cJSON_GetObjectItem(params, "withUserTools");
if (cJSON_IsBool(with_user_tools)) {
list_user_only_tools = with_user_tools->valueint == 1;
}
}
GetToolsList(id_int, cursor_str, list_user_only_tools);
} else if (method_str == "tools/call") {
if (!cJSON_IsObject(params)) {
ESP_LOGE(TAG, "tools/call: Missing params");
ReplyError(id_int, "Missing params");
return;
}
auto tool_name = cJSON_GetObjectItem(params, "name");
if (!cJSON_IsString(tool_name)) {
ESP_LOGE(TAG, "tools/call: Missing name");
ReplyError(id_int, "Missing name");
return;
}
auto tool_arguments = cJSON_GetObjectItem(params, "arguments");
if (tool_arguments != nullptr && !cJSON_IsObject(tool_arguments)) {
ESP_LOGE(TAG, "tools/call: Invalid arguments");
ReplyError(id_int, "Invalid arguments");
return;
}
DoToolCall(id_int, std::string(tool_name->valuestring), tool_arguments);
} else {
ESP_LOGE(TAG, "Method not implemented: %s", method_str.c_str());
ReplyError(id_int, "Method not implemented: " + method_str);
}
}
void McpServer::ReplyResult(int id, const std::string& result) {
std::string payload = "{\"jsonrpc\":\"2.0\",\"id\":";
payload += std::to_string(id) + ",\"result\":";
payload += result;
payload += "}";
Application::GetInstance().SendMcpMessage(payload);
}
void McpServer::ReplyError(int id, const std::string& message) {
std::string payload = "{\"jsonrpc\":\"2.0\",\"id\":";
payload += std::to_string(id);
payload += ",\"error\":{\"message\":\"";
payload += message;
payload += "\"}}";
Application::GetInstance().SendMcpMessage(payload);
}
void McpServer::GetToolsList(int id, const std::string& cursor, bool list_user_only_tools) {
const int max_payload_size = 8000;
std::string json = "{\"tools\":[";
bool found_cursor = cursor.empty();
auto it = tools_.begin();
std::string next_cursor = "";
while (it != tools_.end()) {
// 如果我们还没有找到起始位置,继续搜索
if (!found_cursor) {
if ((*it)->name() == cursor) {
found_cursor = true;
} else {
++it;
continue;
}
}
if (!list_user_only_tools && (*it)->user_only()) {
++it;
continue;
}
// 添加tool前检查大小
std::string tool_json = (*it)->to_json() + ",";
if (json.length() + tool_json.length() + 30 > max_payload_size) {
// 如果添加这个tool会超出大小限制,设置next_cursor并退出循环
next_cursor = (*it)->name();
break;
}
json += tool_json;
++it;
}
if (json.back() == ',') {
json.pop_back();
}
if (json.back() == '[' && !tools_.empty()) {
// 如果没有添加任何tool,返回错误
ESP_LOGE(TAG, "tools/list: Failed to add tool %s because of payload size limit", next_cursor.c_str());
ReplyError(id, "Failed to add tool " + next_cursor + " because of payload size limit");
return;
}
if (next_cursor.empty()) {
json += "]}";
} else {
json += "],\"nextCursor\":\"" + next_cursor + "\"}";
}
ReplyResult(id, json);
}
void McpServer::DoToolCall(int id, const std::string& tool_name, const cJSON* tool_arguments) {
auto tool_iter = std::find_if(tools_.begin(), tools_.end(),
[&tool_name](const McpTool* tool) {
return tool->name() == tool_name;
});
if (tool_iter == tools_.end()) {
ESP_LOGE(TAG, "tools/call: Unknown tool: %s", tool_name.c_str());
ReplyError(id, "Unknown tool: " + tool_name);
return;
}
PropertyList arguments = (*tool_iter)->properties();
try {
for (auto& argument : arguments) {
bool found = false;
if (cJSON_IsObject(tool_arguments)) {
auto value = cJSON_GetObjectItem(tool_arguments, argument.name().c_str());
if (argument.type() == kPropertyTypeBoolean && cJSON_IsBool(value)) {
argument.set_value<bool>(value->valueint == 1);
found = true;
} else if (argument.type() == kPropertyTypeInteger && cJSON_IsNumber(value)) {
argument.set_value<int>(value->valueint);
found = true;
} else if (argument.type() == kPropertyTypeString && cJSON_IsString(value)) {
argument.set_value<std::string>(value->valuestring);
found = true;
}
}
if (!argument.has_default_value() && !found) {
ESP_LOGE(TAG, "tools/call: Missing valid argument: %s", argument.name().c_str());
ReplyError(id, "Missing valid argument: " + argument.name());
return;
}
}
} catch (const std::exception& e) {
ESP_LOGE(TAG, "tools/call: %s", e.what());
ReplyError(id, e.what());
return;
}
// Use main thread to call the tool
auto& app = Application::GetInstance();
app.Schedule([this, id, tool_iter, arguments = std::move(arguments)]() {
try {
ReplyResult(id, (*tool_iter)->Call(arguments));
} catch (const std::exception& e) {
ESP_LOGE(TAG, "tools/call: %s", e.what());
ReplyError(id, e.what());
}
});
}
这里需要注意的是:
小智源代码在v1.7.0版中https://github.com/78/xiaozhi-esp32/releases/tag/v1.7.0增加了MCP协议作为默认控制协议,在此之前项目默认使用物联网 IOT framework作为控制协议。而且iot文件在后续更新中已经删除了,不在支持IOT framework作为控制协议。所以需要下载支持mcp协议的源码,我使用的是v2.0.3。


5.编译烧录
智能小车的控制只需要做一步的代码修改,非常简单。修改完成之后你需要编译源码然后烧录,如果你不会搭建esp32-idf在vscode或者cursor上的环境请看教程:【乐鑫教程】|使用 VS Code 快速搭建 ESP-IDF 开发环境 (Windows、Linux、MacOS)_哔哩哔哩_bilibili
不出意外的话你成功烧录之后唤醒小智ai就能经行语音控制小车了!当然你还可以登录你的小智控制台,定义小智的角色,功能。

更多推荐


所有评论(0)