WebRTC 实时视频流
低延迟 H.264 硬件推流与驱动面会话信令
基于 AI 手机硬件屏幕推流与 WebRTC 媒体引擎,向上位机直接推送低延迟、60FPS 的实时视频流。网关负责驱动面信令交互。
4 步建立视频流会话
上位机 (Electron / Web) 网关服务 (:58166) AI 手机
│ │ │
├─ 1. POST /sessions (创建会话) ────►│── 启动屏幕推流引擎 ────────────────►│
│◄─ 返回 session_id + ice_servers ───┤ │
│ │ │
├─ 2. 本地创建 RTCPeerConnection │ │
├─ 3. POST /sessions/{id}/signal ───►│── 转发 SDP / ICE 信令 ────────────►│
│◄─ 接收远端 Answer 信令 ────────────┤ │
│ │ │
│================ 4. DTLS-SRTP P2P / TURN 媒体流直接直连 (100ms) ========│
会话驱动接口
1. 申请会话 (POST /robot/screen-stream/sessions)
curl -X POST "http://localhost:58166/robot/screen-stream/sessions" \
-H "Content-Type: application/json" \
-H "X-AppKey: <your-key>" \
-d '{
"deviceId": "848bd13e2f9",
"clientInstanceId": "client-uuid-001",
"scale": 0.5,
"fps": 20
}'
{
"code": 200,
"data": {
"session_id": "ses-screen-8f7a1b2c",
"expires_at": 1717172017000,
"ice_servers": [{ "urls": ["stun:turn.langhua.ai:3478"] }]
}
}
2. 交换信令 (POST /robot/screen-stream/sessions/{id}/signal)
将客户端生成的 WebRTC Offer 或 Candidate 发送给手机:
curl -X POST "http://localhost:58166/robot/screen-stream/sessions/ses-screen-8f7a1b2c/signal" \
-H "Content-Type: application/json" \
-H "X-AppKey: <your-key>" \
-d '{
"type": "offer",
"payload": "{\"type\":\"offer\",\"sdp\":\"...\"}"
}'
3. 会话续租与释放
- 续租:每 3~4 分钟调用一次
POST /robot/screen-stream/sessions/{id}/renew(默认租约 5 分钟)。 - 释放:关闭页面时调用
DELETE /robot/screen-stream/sessions/{id},手机端立即关闭编码器释放资源。
开箱即用单文件播放器 (viewer.html)
将以下代码保存为 viewer.html,填入 appKey 和 deviceId,双击在 Chrome 打开即可直接观看真机画面:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>AI 手机实时推流测试</title>
</head>
<body style="background:#111; color:#eee; font-family:sans-serif; text-align:center; padding:20px;">
<h2>AI 手机实时屏幕 (60 FPS)</h2>
<video id="phoneVideo" autoplay playsinline muted style="width:360px; height:800px; background:#000; border-radius:12px; border:2px solid #333;"></video>
<br/><br/>
<button id="startBtn" onclick="startStream()" style="padding:10px 24px; font-size:16px; cursor:pointer;">启动推流</button>
<script>
const GATEWAY = "http://localhost:58166";
const APP_KEY = "sk-lh-xxxxxx"; // 替换为你的 AppKey
const DEVICE_ID = "848bd13e2f9"; // 替换为你的 DeviceId
async function startStream() {
document.getElementById("startBtn").disabled = true;
// 1. 创建会话
const sessRes = await fetch(`${GATEWAY}/robot/screen-stream/sessions`, {
method: "POST",
headers: { "Content-Type": "application/json", "X-AppKey": APP_KEY },
body: JSON.stringify({ deviceId: DEVICE_ID, clientInstanceId: "test-client", scale: 0.5, fps: 20 })
});
const { session_id, ice_servers } = (await sessRes.json()).data;
// 2. 初始化 PeerConnection
const pc = new RTCPeerConnection({ iceServers: ice_servers });
pc.ontrack = (e) => {
document.getElementById("phoneVideo").srcObject = e.streams[0];
};
// 3. 生成并发送 Offer
const offer = await pc.createOffer({ offerToReceiveVideo: true });
await pc.setLocalDescription(offer);
await fetch(`${GATEWAY}/robot/screen-stream/sessions/${session_id}/signal`, {
method: "POST",
headers: { "Content-Type": "application/json", "X-AppKey": APP_KEY },
body: JSON.stringify({ type: "offer", payload: JSON.stringify(offer) })
});
console.log("WebRTC 握手完成,画面渲染中...");
}
</script>
</body>
</html>
并发驱动与抢占机制(开发者必读)
单设备独占策略(Single-Viewer Exclusive):
- 同用户平替:若同一用户在不同浏览器标签或新窗口重复打开同一台手机的画面,网关会自动向旧会话下发
BYE (session_replaced)并优雅关闭旧通道,确保新窗口立即接管画面。 - 跨用户防抢占:若其他账号尝试建立视频会话,网关将直接拒绝并返回错误
“该手机已有其他用户的屏幕实况会话”。