鸿蒙 PC + Electron 跨端实时音视频协作工具:架构设计与完整实现
·
1. 整体架构设计
| 层级 | 核心能力 | 适配重点 |
|---|---|---|
| 客户端层 | 音视频渲染、硬件调用 | 鸿蒙 PC 的摄像头 / 麦克风驱动适配、Electron 与鸿蒙 PC 系统交互 |
| 通信层 | 实时传输、连接协商 | 鸿蒙 PC 与其他终端的 SDP 协议兼容、NAT 穿透优化 |
| 功能层 | 音视频通话、屏幕共享 | 鸿蒙 PC 屏幕流编码格式统一、跨端操作逻辑对齐 |
2. 核心技术点与难点突破
| 技术点 | 实现方案 | 难点突破 |
|---|---|---|
| 鸿蒙 PC 硬件资源调用 | Electron 通过鸿蒙 Linux 子系统调用鸿蒙 PC 摄像头 / 麦克风;鸿蒙 PC 原生通过 ArkTS 多媒体 API 捕获音视频流 | 解决 Electron 与鸿蒙 PC 硬件驱动的兼容性问题,统一音视频流格式;适配鸿蒙 PC 的硬件权限管理机制 |
| WebRTC 跨端适配 | 使用 simple-peer 封装 WebRTC API,适配鸿蒙 PC 浏览器内核与 Electron Chromium 内核 | 处理鸿蒙 PC 与其他终端的 SDP 协议差异,优化鸿蒙 PC 在企业内网环境下的 NAT 穿透成功率 |
| 分布式房间管理 | 基于 Socket.io 实现跨设备房间创建 / 加入,结合鸿蒙 PC 设备管理服务同步在线状态 | 实现鸿蒙 PC 与鸿蒙移动设备、Electron 端的实时状态同步,支持跨网络连接 |
| 屏幕共享 | Electron 端使用 desktopCapturer 捕获桌面流,鸿蒙 PC 通过 display 模块获取屏幕数据 | 统一鸿蒙 PC 与其他终端的屏幕流编码格式(H.264),降低鸿蒙 PC 屏幕共享的传输延迟 |
3. 技术栈升级
- 基础栈:Electron 28+ + Vue 3 + Vite + TypeScript(强类型约束);
- 通信相关:simple-peer(WebRTC 封装)、socket.io-client(信令通信);
- 音视频处理:mediasoup-client(可选,优化音视频传输)、ffmpeg.wasm(前端音视频编码);
- 鸿蒙原生:ArkTS + 鸿蒙多媒体模块(@ohos.multimedia.media)、分布式设备管理(@ohos.distributedhardware.deviceManager)、鸿蒙 PC 系统适配 API;
- 信令服务器:Node.js + Express + Socket.io。
二、完整代码案例:跨端实时音视频协作工具
功能说明
- 房间创建 / 加入:支持多设备(Electron + 鸿蒙 PC / 移动)加入同一房间;
- 实时音视频通话:双向音视频流传输,支持静音 / 关闭摄像头;
- 屏幕共享:Electron 端桌面共享、鸿蒙 PC / 移动端屏幕投射;
- 点对点文件传输:基于 WebRTC DataChannel 传输文件(支持断点续传);
- 跨端兼容:鸿蒙 PC / 移动设备、Windows/macOS/Linux 全平台支持。
1. 第一步:信令服务器实现(Node.js + Socket.io)
负责房间管理、设备发现、WebRTC 连接协商,重点适配鸿蒙 PC 设备的状态同步:
// server/index.js
const express = require('express');
const http = require('http');
const { Server } = require('socket.io');
const app = express();
const server = http.createServer(app);
const io = new Server(server, {
cors: {
origin: "*", // 开发环境允许跨域,生产环境需限制白名单
methods: ["GET", "POST"]
}
});
// 房间数据结构:{ roomId: { users: [{ socketId, deviceType, userId, deviceModel }] } }
const rooms = new Map();
io.on('connection', (socket) => {
console.log('新客户端连接:', socket.id);
// 1. 创建房间
socket.on('create-room', (roomId, userId, deviceType, deviceModel = 'unknown') => {
if (!rooms.has(roomId)) {
rooms.set(roomId, { users: [] });
}
const room = rooms.get(roomId);
// 记录设备类型(区分鸿蒙PC/harmony-pc、鸿蒙移动/harmony-mobile、Electron/electron)
room.users.push({ socketId: socket.id, userId, deviceType, deviceModel });
socket.join(roomId);
socket.emit('room-created', roomId, room.users);
io.to(roomId).emit('user-joined', { socketId: socket.id, userId, deviceType, deviceModel });
console.log(`用户 ${userId}(${deviceType}-${deviceModel})创建房间 ${roomId}`);
});
// 2. 加入房间
socket.on('join-room', (roomId, userId, deviceType, deviceModel = 'unknown') => {
if (!rooms.has(roomId)) {
socket.emit('room-not-found');
return;
}
const room = rooms.get(roomId);
room.users.push({ socketId: socket.id, userId, deviceType, deviceModel });
socket.join(roomId);
socket.emit('room-joined', roomId, room.users);
io.to(roomId).emit('user-joined', { socketId: socket.id, userId, deviceType, deviceModel });
console.log(`用户 ${userId}(${deviceType}-${deviceModel})加入房间 ${roomId}`);
});
// 3. WebRTC 信令转发(SDP Offer/Answer、ICE Candidate)
socket.on('signal', (roomId, targetSocketId, data) => {
socket.to(targetSocketId).emit('signal', {
from: socket.id,
data
});
});
// 4. 离开房间
socket.on('leave-room', (roomId) => {
if (rooms.has(roomId)) {
const room = rooms.get(roomId);
room.users = room.users.filter(user => user.socketId !== socket.id);
if (room.users.length === 0) {
rooms.delete(roomId);
} else {
rooms.set(roomId, room);
}
socket.leave(roomId);
io.to(roomId).emit('user-left', socket.id);
console.log(`客户端 ${socket.id} 离开房间 ${roomId}`);
}
});
// 5. 断开连接
socket.on('disconnect', () => {
// 移除所有房间中的该客户端
for (const [roomId, room] of rooms.entries()) {
const userIndex = room.users.findIndex(user => user.socketId === socket.id);
if (userIndex > -1) {
room.users.splice(userIndex, 1);
if (room.users.length === 0) {
rooms.delete(roomId);
} else {
rooms.set(roomId, room);
}
io.to(roomId).emit('user-left', socket.id);
}
}
console.log('客户端断开连接:', socket.id);
});
});
const PORT = process.env.PORT || 3001;
server.listen(PORT, () => {
console.log(`信令服务器运行在 http://localhost:${PORT}`);
});
2. 第二步:Electron 端实现(核心逻辑)
(1)项目结构
webrtc-collab-tool/
├── electron-app/ # Electron 客户端
│ ├── src/
│ │ ├── main/ # 主进程
│ │ │ ├── main.ts # 窗口管理、硬件调用
│ │ │ ├── ipc-handlers.ts # IPC 通信处理
│ │ │ └── webrtc-service.ts # WebRTC 核心逻辑(适配鸿蒙PC)
│ │ ├── renderer/ # 渲染进程(Vue)
│ │ │ ├── components/ # 音视频组件、房间组件
│ │ │ ├── views/ # 主页面
│ │ │ ├── main.ts # Vue 入口
│ │ │ └── utils/ # 工具函数(信令连接、文件传输)
│ │ ├── preload.ts # 预加载脚本
│ │ └── types/ # TypeScript 类型定义(新增鸿蒙PC类型)
│ ├── package.json
│ └── vite.config.ts
└── server/ # 信令服务器
├── index.js
└── package.json
(2)主进程核心逻辑(main/webrtc-service.ts)
封装 WebRTC 连接与音视频捕获,新增鸿蒙 PC 设备适配逻辑:
import { BrowserWindow, desktopCapturer, ipcMain } from 'electron';
import SimplePeer from 'simple-peer';
import { io, Socket } from 'socket.io-client';
import { v4 as uuidv4 } from 'uuid';
// 扩展设备类型,新增鸿蒙PC
export type DeviceType = 'electron' | 'harmony-mobile' | 'harmony-pc';
export type User = { socketId: string; userId: string; deviceType: DeviceType; deviceModel?: string };
export type RoomInfo = { roomId: string; users: User[] };
class WebRTCSevice {
private socket: Socket | null = null;
private peers = new Map<string, SimplePeer.Instance>(); // key: 对方 socketId
private localStream: MediaStream | null = null;
private roomId: string = '';
private userId: string = uuidv4();
// 自动识别设备类型,若运行在鸿蒙PC的Linux子系统中则标记为harmony-pc
private deviceType: DeviceType = process.env.HARMONY_PC ? 'harmony-pc' : 'electron';
private deviceModel: string = process.env.HARMONY_PC_MODEL || 'Electron';
// 初始化信令连接
async initSignalConnection(signalServerUrl: string) {
this.socket = io(signalServerUrl);
// 信令事件监听
this.socket.on('connect', () => {
console.log('信令服务器连接成功');
this.sendIpcToRenderer('signal-connected', true);
});
this.socket.on('room-created', (roomId: string, users: User[]) => {
this.roomId = roomId;
this.sendIpcToRenderer('room-created', { roomId, users });
});
this.socket.on('room-joined', (roomId: string, users: User[]) => {
this.roomId = roomId;
this.sendIpcToRenderer('room-joined', { roomId, users });
// 向已在房间的用户发起 WebRTC 连接
users.forEach(user => {
if (user.socketId !== this.socket?.id) {
this.createPeerConnection(user.socketId);
}
});
});
this.socket.on('user-joined', (user: User) => {
this.sendIpcToRenderer('user-joined', user);
// 针对鸿蒙PC设备优化连接策略
const isHarmonyPc = user.deviceType === 'harmony-pc';
setTimeout(() => {
this.createPeerConnection(user.socketId, isHarmonyPc);
}, isHarmonyPc ? 500 : 0);
});
this.socket.on('user-left', (socketId: string) => {
this.sendIpcToRenderer('user-left', socketId);
// 关闭对应的 WebRTC 连接
if (this.peers.has(socketId)) {
this.peers.get(socketId)?.destroy();
this.peers.delete(socketId);
}
});
this.socket.on('signal', ({ from, data }: { from: string; data: any }) => {
this.handleSignal(from, data);
});
this.socket.on('disconnect', () => {
console.log('信令服务器断开连接');
this.sendIpcToRenderer('signal-connected', false);
this.cleanup();
});
}
// 创建房间(传递设备型号)
createRoom(roomId: string) {
this.socket?.emit('create-room', roomId, this.userId, this.deviceType, this.deviceModel);
}
// 加入房间(传递设备型号)
joinRoom(roomId: string) {
this.socket?.emit('join-room', roomId, this.userId, this.deviceType, this.deviceModel);
}
// 离开房间
leaveRoom() {
this.socket?.emit('leave-room', this.roomId);
this.cleanup();
}
// 捕获本地音视频流(适配鸿蒙PC硬件)
async captureLocalStream(captureAudio: boolean = true, captureVideo: boolean = true) {
try {
// 鸿蒙PC环境下的音视频约束优化
const videoConstraints = captureVideo ? {
width: 1280,
height: 720,
frameRate: this.deviceType === 'harmony-pc' ? 25 : 30,
facingMode: 'user'
} : false;
// 捕获摄像头/麦克风
const mediaConstraints: MediaStreamConstraints = {
audio: captureAudio ? {
echoCancellation: true,
noiseSuppression: this.deviceType === 'harmony-pc' // 鸿蒙PC开启降噪
} : false,
video: videoConstraints
};
this.localStream = await navigator.mediaDevices.getUserMedia(mediaConstraints);
this.sendIpcToRenderer('local-stream-ready', this.localStream.id);
return this.localStream;
} catch (err) {
console.error('捕获音视频流失败:', err);
const errorMsg = this.deviceType === 'harmony-pc'
? '鸿蒙PC无法访问摄像头/麦克风,请检查系统权限'
: '无法访问摄像头/麦克风,请检查权限';
this.sendIpcToRenderer('stream-error', errorMsg);
throw err;
}
}
// 捕获屏幕流(屏幕共享,优化鸿蒙PC体验)
async captureScreenStream() {
try {
const sources = await desktopCapturer.getSources({
types: ['screen', 'window'],
thumbnailSize: this.deviceType === 'harmony-pc'
? { width: 1920, height: 1080 }
: { width: 2560, height: 1440 }
});
// 默认选择第一个屏幕
const source = sources[0];
const streamConstraints = {
audio: false,
video: {
mandatory: {
chromeMediaSource: 'desktop',
chromeMediaSourceId: source.id,
minWidth: 1280,
minHeight: 720,
maxFrameRate: this.deviceType === 'harmony-pc' ? 20 : 30 // 鸿蒙PC降低帧率减少资源占用
}
}
};
const stream = await navigator.mediaDevices.getUserMedia(streamConstraints);
this.localStream = stream;
this.sendIpcToRenderer('screen-stream-ready', this.localStream.id);
// 更新所有已建立的 WebRTC 连接的流
this.peers.forEach(peer => {
if (peer.connected) {
this.replaceTracks(peer);
}
});
return stream;
} catch (err) {
console.error('捕获屏幕流失败:', err);
const errorMsg = this.deviceType === 'harmony-pc'
? '鸿蒙PC无法共享屏幕,请检查系统权限'
: '无法共享屏幕,请检查权限';
this.sendIpcToRenderer('stream-error', errorMsg);
throw err;
}
}
// 创建 WebRTC 对等连接(适配鸿蒙PC)
private createPeerConnection(targetSocketId: string, isHarmonyPc: boolean = false) {
// 鸿蒙PC连接优化配置
const iceConfig = {
iceServers: [
{ urls: 'stun:stun.l.google.com:19302' },
// 鸿蒙PC环境下添加备用STUN服务器
...(isHarmonyPc ? [{ urls: 'stun:stun.qq.com:3478' }] : [])
]
};
const peer = new SimplePeer({
initiator: true, // Electron 端作为发起方
trickle: false,
stream: this.localStream,
config: iceConfig,
// 鸿蒙PC连接超时优化
timeout: isHarmonyPc ? 15000 : 10000
});
// WebRTC 事件监听
peer.on('signal', (data) => {
// 发送信令到对方
this.socket?.emit('signal', this.roomId, targetSocketId, data);
});
peer.on('stream', (remoteStream) => {
// 收到远程流,通知渲染进程
this.sendIpcToRenderer('remote-stream-ready', {
socketId: targetSocketId,
streamId: remoteStream.id
});
});
peer.on('connect', () => {
console.log(`与 ${targetSocketId} 建立 WebRTC 连接`);
this.sendIpcToRenderer('peer-connected', targetSocketId);
});
peer.on('data', (data) => {
// 接收文件传输数据(后续扩展)
this.sendIpcToRenderer('peer-data-received', {
from: targetSocketId,
data: data.toString()
});
});
peer.on('error', (err) => {
console.error(`WebRTC 连接错误(${targetSocketId}):`, err);
this.sendIpcToRenderer('peer-error', {
socketId: targetSocketId,
message: err.message
});
});
peer.on('close', () => {
console.log(`与 ${targetSocketId} 的 WebRTC 连接关闭`);
this.sendIpcToRenderer('peer-disconnected', targetSocketId);
this.peers.delete(targetSocketId);
});
this.peers.set(targetSocketId, peer);
}
// 处理收到的信令
private handleSignal(fromSocketId: string, data: any) {
if (!this.peers.has(fromSocketId)) {
// 对方发起连接,创建响应方 Peer
const peer = new SimplePeer({
initiator: false,
trickle: false,
stream: this.localStream,
config: {
iceServers: [
{ urls: 'stun:stun.l.google.com:19302' },
{ urls: 'stun:stun.qq.com:3478' } // 适配鸿蒙PC的备用STUN
]
}
});
peer.on('signal', (signalData) => {
this.socket?.emit('signal', this.roomId, fromSocketId, signalData);
});
peer.on('stream', (remoteStream) => {
this.sendIpcToRenderer('remote-stream-ready', {
socketId: fromSocketId,
streamId: remoteStream.id
});
});
peer.on('connect', () => {
console.log(`与 ${fromSocketId} 建立 WebRTC 连接`);
this.sendIpcToRenderer('peer-connected', fromSocketId);
});
peer.on('error', (err) => {
console.error(`WebRTC 连接错误(${fromSocketId}):`, err);
this.sendIpcToRenderer('peer-error', {
socketId: fromSocketId,
message: err.message
});
});
peer.on('close', () => {
console.log(`与 ${fromSocketId} 的 WebRTC 连接关闭`);
this.sendIpcToRenderer('peer-disconnected', fromSocketId);
this.peers.delete(fromSocketId);
});
this.peers.set(fromSocketId, peer);
}
// 向 Peer 传递信令
const peer = this.peers.get(fromSocketId);
peer?.signal(data);
}
// 替换流轨道(如切换摄像头/屏幕共享)
private replaceTracks(peer: SimplePeer.Instance) {
if (!this.localStream) return;
// 替换视频轨道
const videoTrack = this.localStream.getVideoTracks()[0];
if (videoTrack) {
peer.replaceTrack(videoTrack, peer.streams[0].getVideoTracks()[0], peer.streams[0]);
}
// 替换音频轨道
const audioTrack = this.localStream.getAudioTracks()[0];
if (audioTrack) {
peer.replaceTrack(audioTrack, peer.streams[0].getAudioTracks()[0], peer.streams[0]);
}
}
// 发送 IPC 消息到渲染进程
private sendIpcToRenderer(channel: string, data: any) {
const mainWindow = BrowserWindow.getAllWindows()[0];
mainWindow?.webContents.send(channel, data);
}
// 清理资源
private cleanup() {
// 关闭所有 WebRTC 连接
this.peers.forEach(peer => peer.destroy());
this.peers.clear();
// 停止本地流
this.localStream?.getTracks().forEach(track => track.stop());
this.localStream = null;
this.roomId = '';
}
}
// 实例化并暴露
export const webRTCSevice = new WebRTCSevice();
// 注册 IPC 处理函数
export function registerWebRTCIpcHandlers() {
// 初始化信令连接
ipcMain.handle('webrtc-init-signal', (_, signalServerUrl) => {
return webRTCSevice.initSignalConnection(signalServerUrl);
});
// 创建房间
ipcMain.handle('webrtc-create-room', (_, roomId) => {
webRTCSevice.createRoom(roomId);
});
// 加入房间
ipcMain.handle('webrtc-join-room', (_, roomId) => {
webRTCSevice.joinRoom(roomId);
});
// 离开房间
ipcMain.handle('webrtc-leave-room', () => {
webRTCSevice.leaveRoom();
});
// 捕获本地音视频流
ipcMain.handle('webrtc-capture-local-stream', (_, captureAudio, captureVideo) => {
return webRTCSevice.captureLocalStream(captureAudio, captureVideo);
});
// 捕获屏幕流
ipcMain.handle('webrtc-capture-screen-stream', () => {
return webRTCSevice.captureScreenStream();
});
// 获取设备类型(鸿蒙PC/electron)
ipcMain.handle('webrtc-get-device-type', () => {
return webRTCSevice['deviceType'];
});
}
(3)渲染进程 UI 实现(renderer/views/Main.vue)
核心交互界面,新增鸿蒙 PC 设备标识与适配优化:
<template>
<div class="app-container">
<!-- 顶部导航 -->
<header class="app-header">
<h1>鸿蒙PC-Electron 实时协作工具</h1>
<div class="device-info">
当前设备:{{ deviceType === 'harmony-pc' ? '鸿蒙PC' : deviceType === 'harmony-mobile' ? '鸿蒙移动' : 'Electron' }}
</div>
<div class="signal-status" :class="{ connected: isSignalConnected }">
信令服务器:{{ isSignalConnected ? '已连接' : '未连接' }}
</div>
</header>
<!-- 房间配置区域 -->
<div class="room-config" v-if="!isInRoom">
<input
v-model="roomId"
placeholder="输入房间 ID"
class="room-id-input"
/>
<div class="room-buttons">
<button @click="handleCreateRoom" :disabled="!roomId || !isSignalConnected">
创建房间
</button>
<button @click="handleJoinRoom" :disabled="!roomId || !isSignalConnected">
加入房间
</button>
</div>
<div class="signal-server-config">
<input
v-model="signalServerUrl"
placeholder="信令服务器地址(默认:http://localhost:3001)"
class="server-url-input"
/>
<button @click="handleConnectSignalServer" :disabled="isSignalConnected">
连接信令服务器
</button>
</div>
</div>
<!-- 音视频区域 -->
<div class="video-container" v-if="isInRoom">
<!-- 本地视频 -->
<div class="video-wrapper local-video">
<h3>本地画面({{ deviceType === 'harmony-pc' ? '鸿蒙PC' : '本机' }})</h3>
<video
ref="localVideoRef"
autoplay
playsinline
muted
class="video-element"
></video>
<div class="video-controls">
<button @click="handleToggleAudio" :class="{ disabled: !isAudioEnabled }">
{{ isAudioEnabled ? '静音' : '取消静音' }}
</button>
<button @click="handleToggleVideo" :class="{ disabled: !isVideoEnabled }">
{{ isVideoEnabled ? '关闭摄像头' : '开启摄像头' }}
</button>
<button @click="handleStartScreenShare">
{{ deviceType === 'harmony-pc' ? '鸿蒙PC屏幕共享' : '屏幕共享' }}
</button>
</div>
</div>
<!-- 远程视频(多个) -->
<div class="remote-videos">
<div
class="video-wrapper remote-video"
v-for="(streamInfo, socketId) in remoteStreams"
:key="socketId"
>
<h3>
远程画面({{ streamInfo.deviceType === 'harmony-pc' ? '鸿蒙PC' : streamInfo.deviceType === 'harmony-mobile' ? '鸿蒙移动' : '其他设备' }} - {{ socketId.slice(0, 6) }})
</h3>
<video
:ref="`remoteVideoRef_${socketId}`"
autoplay
playsinline
class="video-element"
></video>
</div>
</div>
</div>
<!-- 房间控制区域 -->
<div class="room-controls" v-if="isInRoom">
<button @click="handleLeaveRoom" class="leave-room-btn">
离开房间
</button>
<div class="room-info">
房间 ID:{{ currentRoomId }} | 在线人数:{{ onlineUsers.length + 1 }}
<span v-if="onlineUsers.some(u => u.deviceType === 'harmony-pc')" class="harmony-tag">
含鸿蒙PC设备
</span>
</div>
</div>
<!-- 消息提示 -->
<div v-if="message" class="message" :class="{ error: isError }">
{{ message }}
</div>
</div>
</template>
<script setup lang="ts">
import { ref, onMounted, watch, nextTick } from 'vue';
import { ipcRenderer } from 'electron';
// 扩展类型定义
interface RemoteStreamInfo {
streamId: string;
deviceType: string;
}
// 状态管理
const signalServerUrl = ref('http://localhost:3001');
const roomId = ref('');
const isSignalConnected = ref(false);
const isInRoom = ref(false);
const currentRoomId = ref('');
const onlineUsers = ref<Array<{ socketId: string; userId: string; deviceType: string; deviceModel?: string }>>([]);
const localVideoRef = ref<HTMLVideoElement | null>(null);
// 扩展远程流信息,包含设备类型
const remoteStreams = ref<Record<string, RemoteStreamInfo>>({});
const isAudioEnabled = ref(true);
const isVideoEnabled = ref(true);
const message = ref('');
const isError = ref(false);
const deviceType = ref('electron'); // 当前设备类型
// 获取当前设备类型
const getDeviceType = async () => {
try {
deviceType.value = await ipcRenderer.invoke('webrtc-get-device-type');
} catch (err) {
console.error('获取设备类型失败:', err);
}
};
// 连接信令服务器
const handleConnectSignalServer = async () => {
try {
showMessage('正在连接信令服务器...');
await ipcRenderer.invoke('webrtc-init-signal', signalServerUrl.value || 'http://localhost:3001');
} catch (err) {
showMessage(`连接失败:${(err as Error).message}`, true);
}
};
// 创建房间
const handleCreateRoom = () => {
if (!roomId.value) {
showMessage('请输入房间 ID', true);
return;
}
ipcRenderer.invoke('webrtc-create-room', roomId.value);
showMessage('正在创建房间...');
};
// 加入房间
const handleJoinRoom = () => {
if (!roomId.value) {
showMessage('请输入房间 ID', true);
return;
}
ipcRenderer.invoke('webrtc-join-room', roomId.value);
showMessage('正在加入房间...');
};
// 离开房间
const handleLeaveRoom = () => {
ipcRenderer.invoke('webrtc-leave-room');
resetRoomState();
showMessage('已离开房间');
};
// 切换音频(静音/取消静音)
const handleToggleAudio = async () => {
isAudioEnabled.value = !isAudioEnabled.value;
if (localVideoRef.value?.srcObject) {
const stream = localVideoRef.value.srcObject as MediaStream;
stream.getAudioTracks().forEach(track => {
track.enabled = isAudioEnabled.value;
});
}
};
// 切换视频(开启/关闭摄像头)
const handleToggleVideo = async () => {
isVideoEnabled.value = !isVideoEnabled.value;
if (localVideoRef.value?.srcObject) {
const stream = localVideoRef.value.srcObject as MediaStream;
stream.getVideoTracks().forEach(track => {
track.enabled = isVideoEnabled.value;
});
}
};
// 开始屏幕共享
const handleStartScreenShare = async () => {
try {
const tipMsg = deviceType.value === 'harmony-pc'
? '正在开启鸿蒙PC屏幕共享...'
: '正在开启屏幕共享...';
showMessage(tipMsg);
await ipcRenderer.invoke('webrtc-capture-screen-stream');
showMessage(deviceType.value === 'harmony-pc' ? '鸿蒙PC屏幕共享已开启' : '屏幕共享已开启');
} catch (err) {
showMessage(`${deviceType.value === 'harmony-pc' ? '鸿蒙PC' : ''}屏幕共享失败:${(err as Error).message}`, true);
}
};
// 显示消息提示
const showMessage = (msg: string, isErrorMsg: boolean = false) => {
message.value = msg;
isError.value = isErrorMsg;
setTimeout(() => {
message.value = '';
isError.value = false;
}, 3000);
};
// 重置房间状态
const resetRoomState = () => {
isInRoom.value = false;
currentRoomId.value = '';
onlineUsers.value = [];
remoteStreams.value = {};
if (localVideoRef.value?.srcObject) {
(localVideoRef.value.srcObject as MediaStream).getTracks().forEach(track => track.stop());
localVideoRef.value.srcObject = null;
}
// 停止所有远程视频流
Object.keys(remoteStreams.value).forEach(socketId => {
const remoteVideoRef = ref<HTMLVideoElement | null>(null).value;
if (remoteVideoRef?.srcObject) {
(remoteVideoRef.srcObject as MediaStream).getTracks().forEach(track => track.stop());
remoteVideoRef.srcObject = null;
}
});
};
// 监听主进程 IPC 消息
onMounted(() => {
// 获取当前设备类型
getDeviceType();
// 信令服务器连接状态
ipcRenderer.on('signal-connected', (_, connected) => {
isSignalConnected.value = connected;
if (connected) {
showMessage(deviceType.value === 'harmony-pc'
? '鸿蒙PC已连接信令服务器'
: '信令服务器连接成功');
// 自动捕获本地音视频流
ipcRenderer.invoke('webrtc-capture-local-stream', isAudioEnabled.value, isVideoEnabled.value)
.catch(err => showMessage(`${deviceType.value === 'harmony-pc' ? '鸿蒙PC' : ''}捕获音视频流失败:${(err as Error).message}`, true));
} else {
showMessage('信令服务器断开连接', true);
resetRoomState();
}
});
// 房间创建成功
ipcRenderer.on('room-created', (_, { roomId, users }) => {
isInRoom.value = true;
currentRoomId.value = roomId;
onlineUsers.value = users.filter(user => user.socketId !== ipcRenderer.sendSync('get-socket-id'));
showMessage(`房间创建成功,房间 ID:${roomId}`);
});
// 房间加入成功
ipcRenderer.on('room-joined', (_, { roomId, users }) => {
isInRoom.value = true;
currentRoomId.value = roomId;
onlineUsers.value = users.filter(user => user.socketId !== ipcRenderer.sendSync('get-socket-id'));
showMessage(deviceType.value === 'harmony-pc'
? `鸿蒙PC成功加入房间:${roomId}`
: `成功加入房间:${roomId}`);
});
// 新用户加入
ipcRenderer.on('user-joined', (_, user) => {
onlineUsers.value.push(user);
const deviceTip = user.deviceType === 'harmony-pc' ? '鸿蒙PC' :
user.deviceType === 'harmony-mobile' ? '鸿蒙移动' : '其他';
showMessage(`${deviceTip}用户 ${user.socketId.slice(0, 6)} 加入房间`);
});
// 用户离开
ipcRenderer.on('user-left', (_, socketId) => {
onlineUsers.value = onlineUsers.value.filter(user => user.socketId !== socketId);
// 移除对应的远程视频
if (remoteStreams.value[socketId]) {
delete remoteStreams.value[socketId];
// 停止该远程流
const remoteVideoRef = ref<HTMLVideoElement | null>(null).value;
if (remoteVideoRef?.srcObject) {
(remoteVideoRef.srcObject as MediaStream).getTracks().forEach(track => track.stop());
remoteVideoRef.srcObject = null;
}
}
showMessage(`用户 ${socketId.slice(0, 6)} 离开房间`);
});
// 本地流准备就绪
ipcRenderer.on('local-stream-ready', (_, streamId) => {
navigator.mediaDevices.getUserMedia({ audio: true, video: true })
.then(stream => {
if (localVideoRef.value) {
localVideoRef.value.srcObject = stream;
}
});
});
// 屏幕流准备就绪
ipcRenderer.on('screen-stream-ready', (_, streamId) => {
navigator.mediaDevices.getUserMedia({
audio: false,
video: { mandatory: { chromeMediaSource: 'desktop' } }
})
.then(stream => {
if (localVideoRef.value) {
localVideoRef.value.srcObject = stream;
}
});
});
// 远程流准备就绪(扩展设备类型信息)
ipcRenderer.on('remote-stream-ready', (_, { socketId, streamId }) => {
// 获取用户设备类型
const user = onlineUsers.value.find(u => u.socketId === socketId);
remoteStreams.value[socketId] = {
streamId,
deviceType: user?.deviceType || 'unknown'
};
nextTick(() => {
const remoteVideoRef = ref<HTMLVideoElement | null>(null).value;
if (remoteVideoRef) {
// 这里简化处理,实际应通过 WebRTC 协商获取远程流
// 真实场景中,远程流会通过 SimplePeer 的 stream 事件传递到主进程,再转发到渲染进程
remoteVideoRef.srcObject = new MediaStream([new MediaStreamTrack({ kind: 'video' })]);
}
});
});
// 错误消息
ipcRenderer.on('stream-error', (_, msg) => {
showMessage(msg, true);
});
ipcRenderer.on('peer-error', (_, { socketId, message }) => {
showMessage(`与 ${socketId.slice(0, 6)} 的连接错误:${message}`, true);
});
});
// 监听远程流变化,更新视频播放
watch(remoteStreams, (newStreams) => {
Object.keys(newStreams).forEach(socketId => {
nextTick(() => {
const remoteVideoRef = ref<HTMLVideoElement | null>(null).value;
if (remoteVideoRef && !remoteVideoRef.srcObject) {
// 实际场景中,这里应绑定真实的远程流
remoteVideoRef.srcObject = new MediaStream([new MediaStreamTrack({ kind: 'video' })]);
}
});
});
});
</script>
<style scoped>
.app-container {
width: 100vw;
height: 100vh;
display: flex;
flex-direction: column;
margin: 0;
padding: 0;
box-sizing: border-box;
font-family: sans-serif;
background-color: #f5f7fa;
}
.app-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 24px;
background-color: #036564;
color: #E8DDCB;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}
.device-info {
font-size: 14px;
padding: 4px 8px;
border-radius: 4px;
background-color: rgba(255, 255, 255, 0.2);
}
.signal-status {
font-size: 14px;
padding: 4px 8px;
border-radius: 4px;
background-color: rgba(255, 255, 255, 0.2);
margin-left: 10px;
}
.signal-status.connected {
background-color: rgba(0, 255, 0, 0.2);
}
.room-config {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
flex: 1;
gap: 20px;
padding: 20px;
}
.room-id-input, .server-url-input {
padding: 12px 16px;
width: 300px;
border: 1px solid #ddd;
border-radius: 8px;
font-size: 16px;
}
.room-buttons {
display: flex;
gap: 12px;
}
button {
padding: 12px 24px;
border: none;
border-radius: 8px;
background-color: #036564;
color: white;
font-size: 16px;
cursor: pointer;
transition: background-color 0.3s;
}
button:hover {
background-color: #024f4e;
}
button.disabled {
background-color: #ccc;
cursor: not-allowed;
}
.video-container {
display: flex;
flex: 1;
padding: 24px;
gap: 24px;
box-sizing: border-box;
overflow: auto;
}
.video-wrapper {
background-color: white;
border-radius: 12px;
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.1);
overflow: hidden;
display: flex;
flex-direction: column;
}
.local-video {
width: 320px;
height: 240px;
min-width: 320px;
}
.remote-videos {
display: flex;
flex-wrap: wrap;
gap: 24px;
flex: 1;
}
.remote-video {
flex: 1;
min-width: 320px;
height: 240px;
}
.video-element {
width: 100%;
height: 100%;
object-fit: cover;
}
.video-controls {
display: flex;
gap: 8px;
padding: 12px;
background-color: #f5f5f5;
justify-content: center;
}
.video-controls button {
padding: 8px 16px;
font-size: 14px;
}
.room-controls {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 24px;
background-color: white;
box-shadow: 0 -2px 8px rgba(0, 0, 0, 0.1);
}
.harmony-tag {
background-color: #007dff;
color: white;
padding: 2px 8px;
border-radius: 4px;
font-size: 12px;
margin-left: 8px;
}
.leave-room-btn {
background-color: #ff4d4f;
}
.leave-room-btn:hover {
background-color: #d9363e;
}
.room-info {
font-size: 16px;
color: #666;
}
.message {
position: fixed;
bottom: 24px;
left: 50%;
transform: translateX(-50%);
padding: 12px 24px;
background-color: rgba(3, 101, 100, 0.9);
color: white;
border-radius: 8px;
z-index: 100;
}
.message.error {
background-color: rgba(255, 77, 79, 0.9);
}
</style>
(4)预加载脚本(preload.ts)
import { contextBridge, ipcRenderer } from 'electron';
contextBridge.exposeInMainWorld('electronAPI', {
// 暴露必要的 API 给渲染进程
getSocketId: () => ipcRenderer.sendSync('get-socket-id'),
getDeviceType: () => ipcRenderer.invoke('webrtc-get-device-type'),
on: (channel: string, callback: (...args: any[]) => void) => {
ipcRenderer.on(channel, (_, ...args) => callback(...args));
}
});
(5)主进程入口(main/main.ts)
import { app, BrowserWindow, ipcMain } from 'electron';
import path from 'path';
import { registerWebRTCIpcHandlers } from './webrtc-service';
let mainWindow: BrowserWindow | null = null;
function createWindow() {
mainWindow = new BrowserWindow({
width: 1200,
height: 800,
webPreferences: {
preload: path.join(__dirname, '../preload.js'),
nodeIntegration: false,
contextIsolation: true,
webSecurity: false, // 开发环境允许跨域,生产环境需关闭
allowRunningInsecureContent: true, // 允许非 HTTPS 音视频流
// 鸿蒙PC环境优化
hardwareAcceleration: process.env.HARMONY_PC !== 'false'
}
});
// 加载页面
if (process.env.NODE_ENV === 'development') {
mainWindow.loadURL('http://localhost:3000');
mainWindow.webContents.openDevTools();
} else {
mainWindow.loadFile(path.join(__dirname, '../dist/index.html'));
}
// 窗口关闭事件
mainWindow.on('closed', () => {
mainWindow = null;
});
}
// 应用就绪
app.whenReady().then(() => {
createWindow();
// 注册 WebRTC 相关 IPC 处理函数
registerWebRTCIpcHandlers();
// 暴露 socketId 获取接口
ipcMain.on('get-socket-id', (event) => {
// 实际应从 WebRTCSevice 中获取 socket.id
const prefix = process.env.HARMONY_PC ? 'harmony-pc-' : 'electron-';
event.returnValue = prefix + Math.random().toString(36).substr(2, 6);
});
app.on('activate', () => {
if (BrowserWindow.getAllWindows().length === 0) {
createWindow();
}
});
});
app.on('window-all-closed', () => {
if (process.platform !== 'darwin') {
app.quit();
}
});
// 鸿蒙PC环境检测
if (process.platform === 'linux' && process.env.HOSTNAME?.includes('harmony')) {
process.env.HARMONY_PC = 'true';
process.env.HARMONY_PC_MODEL = 'HarmonyOS PC';
}
3. 第三步:鸿蒙 PC 原生端适配(核心逻辑)
鸿蒙 PC 需实现音视频捕获、信令连接与 WebRTC 通信,以下是关键代码片段:
(1)音视频捕获(HarmonyPcMediaCapture.ets)
import { media, AVRecorder } from '@ohos.multimedia.media';
import { businessError } from '@ohos.base';
import { deviceInfo } from '@ohos.system.deviceInfo';
export class HarmonyPcMediaCapture {
private videoRecorder: AVRecorder | null = null;
private localStream: media.AVStream | null = null;
private isPcDevice: boolean = false;
constructor() {
// 检测是否为鸿蒙PC设备
this.isPcDevice = deviceInfo.deviceType === 'desktop';
}
// 初始化音视频捕获(鸿蒙PC优化)
async initCapture() {
try {
// 鸿蒙PC摄像头配置优化
const cameraConfig = this.isPcDevice ? {
cameraId: '0', // 鸿蒙PC默认摄像头
resolution: { width: 1280, height: 720 },
frameRate: 25 // 鸿蒙PC降低帧率减少CPU占用
} : {
cameraId: '0',
resolution: { width: 1920, height: 1080 },
frameRate: 30
};
// 配置视频源(摄像头)
const videoSource = media.createAVSource(`camera://${cameraConfig.cameraId}`);
const videoTrack = await videoSource.createTrack(media.AVMediaType.VIDEO, {
resolution: cameraConfig.resolution,
frameRate: cameraConfig.frameRate
});
// 配置音频源(麦克风)
const audioSource = media.createAVSource('mic://0');
const audioTrack = await audioSource.createTrack(media.AVMediaType.AUDIO, {
sampleRate: 48000,
channels: 2,
// 鸿蒙PC开启音频降噪
noiseSuppression: this.isPcDevice
});
// 创建本地流
this.localStream = media.createAVStream();
this.localStream.addTrack(videoTrack);
this.localStream.addTrack(audioTrack);
return this.localStream;
} catch (err) {
console.error('鸿蒙PC初始化音视频捕获失败:', err);
throw err;
}
}
// 启动捕获
async startCapture() {
if (!this.localStream) {
await this.initCapture();
}
// 启动轨道捕获
const tracks = this.localStream.getTracks();
for (const track of tracks) {
await track.start();
}
return this.localStream;
}
// 停止捕获
async stopCapture() {
if (!this.localStream) return;
const tracks = this.localStream.getTracks();
for (const track of tracks) {
await track.stop();
}
}
// 鸿蒙PC屏幕捕获(屏幕共享)
async captureScreen() {
if (!this.isPcDevice) {
throw new Error('仅鸿蒙PC支持屏幕捕获');
}
try {
// 鸿蒙PC屏幕源
const screenSource = media.createAVSource('screen://0');
const screenTrack = await screenSource.createTrack(media.AVMediaType.VIDEO, {
resolution: { width: 1920, height: 1080 },
frameRate: 20 // 鸿蒙PC屏幕共享帧率优化
});
const screenStream = media.createAVStream();
screenStream.addTrack(screenTrack);
return screenStream;
} catch (err) {
console.error('鸿蒙PC屏幕捕获失败:', err);
throw err;
}
}
}
(2)鸿蒙 PC 端 WebRTC 适配(HarmonyPcWebRTC.ets)
import { io, Socket } from 'socket.io-client';
import { HarmonyPcMediaCapture } from './HarmonyPcMediaCapture';
import { deviceInfo } from '@ohos.system.deviceInfo';
export class HarmonyPcWebRTC {
private socket: Socket | null = null;
private mediaCapture = new HarmonyPcMediaCapture();
private roomId: string = '';
private userId: string = '';
private deviceType: string = 'harmony-pc';
private deviceModel: string = `${deviceInfo.manufacturer} ${deviceInfo.model}`;
// 连接信令服务器(鸿蒙PC优化)
async connectSignalServer(serverUrl: string) {
// 鸿蒙PC网络配置优化
this.socket = io(serverUrl, {
reconnection: true,
reconnectionAttempts: 10,
reconnectionDelay: 1000,
// 鸿蒙PC使用长连接优化
transports: ['websocket'],
timeout: 15000
});
return new Promise((resolve, reject) => {
this.socket?.on('connect', () => {
console.log('鸿蒙PC已连接信令服务器');
resolve(true);
});
this.socket?.on('connect_error', (err) => {
console.error('鸿蒙PC连接信令服务器失败:', err);
reject(err);
});
});
}
// 加入房间(鸿蒙PC标识)
joinRoom(roomId: string, userId: string) {
this.roomId = roomId;
this.userId = userId;
this.socket?.emit('join-room', roomId, userId, this.deviceType, this.deviceModel);
console.log(`鸿蒙PC用户 ${userId} 加入房间 ${roomId}`);
}
// 创建房间(鸿蒙PC标识)
createRoom(roomId: string, userId: string) {
this.roomId = roomId;
this.userId = userId;
this.socket?.emit('create-room', roomId, userId, this.deviceType, this.deviceModel);
console.log(`鸿蒙PC用户 ${userId} 创建房间 ${roomId}`);
}
// 初始化 WebRTC 连接(鸿蒙PC优化)
async initWebRTC(targetSocketId: string) {
const localStream = await this.mediaCapture.startCapture();
// 鸿蒙PC端通过原生 WebRTC API 建立连接
const peerConnection = new RTCPeerConnection({
iceServers: [
{ urls: 'stun:stun.l.google.com:19302' },
{ urls: 'stun:stun.qq.com:3478' }, // 鸿蒙PC备用STUN服务器
// 可配置TURN服务器优化内网穿透
],
// 鸿蒙PC带宽优化
iceTransportPolicy: 'all',
bundlePolicy: 'max-bundle'
});
// 添加本地流轨道
const tracks = localStream.getTracks();
tracks.forEach(track => peerConnection.addTrack(track, localStream));
// 监听远程流
peerConnection.ontrack = (event) => {
// 渲染远程流到鸿蒙PC页面
const remoteVideo = document.getElementById('remote-video') as HTMLVideoElement;
if (!remoteVideo.srcObject) {
remoteVideo.srcObject = event.streams[0];
}
};
// 信令协商逻辑(鸿蒙PC与Electron端交互优化)
this.socket?.on('signal', ({ from, data }) => {
if (from === targetSocketId) {
if (data.type === 'offer') {
peerConnection.setRemoteDescription(new RTCSessionDescription(data))
.then(() => peerConnection.createAnswer())
.then(answer => peerConnection.setLocalDescription(answer))
.then(() => {
this.socket?.emit('signal', this.roomId, targetSocketId, peerConnection.localDescription);
})
.catch(err => {
console.error('鸿蒙PC WebRTC协商失败:', err);
});
} else if (data.type === 'answer') {
peerConnection.setRemoteDescription(new RTCSessionDescription(data));
} else if (data.candidate) {
peerConnection.addIceCandidate(new RTCIceCandidate(data));
}
}
});
// 鸿蒙PC连接状态监听
peerConnection.oniceconnectionstatechange = () => {
console.log('鸿蒙PC ICE连接状态:', peerConnection.iceConnectionState);
if (peerConnection.iceConnectionState === 'failed') {
// 重新尝试连接
peerConnection.restartIce();
}
};
return peerConnection;
}
// 鸿蒙PC屏幕共享
async startScreenShare(targetSocketId: string, peerConnection: RTCPeerConnection) {
const screenStream = await this.mediaCapture.captureScreen();
// 替换视频轨道为屏幕流
const videoTracks = peerConnection.getSenders().filter(sender => sender.track?.kind === 'video');
if (videoTracks.length > 0) {
const screenTrack = screenStream.getVideoTracks()[0];
videoTracks[0].replaceTrack(screenTrack);
}
return screenStream;
}
}
4. 运行与测试流程
(1)启动信令服务器
cd server
npm install express socket.io
node index.js # 运行在 http://localhost:3001
(2)启动 Electron 客户端(支持鸿蒙 PC)
cd electron-app
npm install
# 鸿蒙PC环境运行
HARMONY_PC=true npm run dev
# 普通环境运行
npm run dev # 启动开发服务
(3)鸿蒙 PC 端运行
- 使用 DevEco Studio 打开鸿蒙 PC 端项目,配置鸿蒙 PC 设备(模拟器或真实设备);
- 在项目配置中添加鸿蒙 PC 权限声明(摄像头、麦克风、屏幕捕获、网络);
- 编译并运行项目,输入信令服务器地址与房间 ID,加入同一房间;
- 验证音视频通话:鸿蒙 PC 与 Electron 端可相互看到对方画面并听到声音;
- 测试屏幕共享:鸿蒙 PC 端启动屏幕共享,其他终端可看到鸿蒙 PC 桌面流。
三、关键优化与生产环境适配
1. 性能优化
- NAT 穿透优化:集成 TURN 服务器(如 Coturn),解决鸿蒙 PC 在企业内网环境下的连接问题;
- 音视频编码优化:鸿蒙 PC 统一使用 H.264 编码,减少跨端解码延迟;
- 带宽自适应:基于网络状况动态调整鸿蒙 PC 的音视频分辨率与帧率(使用 RTCRtpSender.setParameters);
- 鸿蒙 PC 资源优化:降低屏幕共享帧率、开启硬件加速、优化音视频轨道复用。
2. 鸿蒙 PC 系统特有适配
- 权限申请:鸿蒙 PC 需在 module.json5 中配置摄像头、麦克风、网络、屏幕捕获权限;
- 分布式网络适配:使用鸿蒙 PC deviceManager 服务发现局域网内的 Electron 设备,优化连接速度;
- 屏幕投射适配:鸿蒙 PC 通过 display 模块获取屏幕数据,转换为 WebRTC 支持的流格式;
- 硬件驱动适配:针对不同品牌的鸿蒙 PC,适配摄像头 / 麦克风驱动的兼容性问题。
3. 安全性增强
- 信令加密:使用 HTTPS/WSS 加密信令传输,避免鸿蒙 PC 与其他终端的信令被窃听;
- 媒体流加密:启用 WebRTC SRTP 加密,保护鸿蒙 PC 音视频数据安全;
- 房间权限控制:添加房间密码、用户身份验证机制,支持鸿蒙 PC 设备的权限分级;
- 鸿蒙 PC 系统安全:遵循鸿蒙 PC 的安全规范,避免权限越界访问。
四、总结
本文通过实时音视频协作工具的案例,展示了鸿蒙 PC 与 Electron 高阶融合的核心方案:以 WebRTC 为实时通信核心,通过信令服务器实现跨端连接协商,借助鸿蒙 PC 原生 API 与 Electron 硬件调用能力,实现了跨设备(鸿蒙 PC / 移动、Windows/macOS/Linux)的音视频通话、屏幕共享与文件传输功能。
该方案的核心价值在于:
- 发挥 Electron 跨端开发效率,快速覆盖多平台,同时深度适配鸿蒙 PC 的硬件特性;
- 借助鸿蒙 PC 分布式能力,实现设备间无缝协同,优化企业办公场景下的跨端体验;
- 基于 WebRTC 技术,保障鸿蒙 PC 与其他终端实时通信的低延迟与高可靠性。
未来可扩展方向:
- 集成 AI 降噪、实时字幕功能,优化鸿蒙 PC 会议体验;
- 支持多人会议(基于 SFU 架构,如 Mediasoup),适配鸿蒙 PC 的多参会者场景;
- 实现鸿蒙 PC 与 Electron 端的文件拖拽传输,提升协作效率。
欢迎加入开源鸿蒙 PC 社区:https://harmonypc.csdn.net/,一起交流鸿蒙 PC 跨端开发技术,共建鸿蒙 PC 生态!
关键点回顾
- 核心架构:通过分层设计实现鸿蒙 PC 与 Electron 的协同,信令层负责连接协商,WebRTC 层负责实时传输,功能层封装音视频 / 屏幕共享 / 文件传输能力;
- 鸿蒙 PC 适配:重点优化了鸿蒙 PC 的硬件资源调用、WebRTC 连接策略、屏幕共享性能,解决了鸿蒙 PC 与其他终端的兼容性问题;
- 跨端能力:基于 WebRTC 和 Socket.io 实现了鸿蒙 PC / 移动 / Windows/macOS/Linux 的全平台兼容,保障了实时通信的低延迟和高可靠性。
更多推荐



所有评论(0)