移动机器人在未知环境里导航,每秒要处理 30 帧 640×480 图像 → 每帧提取 1000 个 ORB 特征点 → 与上一帧做特征匹配 → 估算相机运动(本质矩阵 E → R, t)→ 更新 3D 地图。CPU 上单帧处理 45ms,30fps 需要 <33ms → 丢帧,定位漂移累积 5 秒后误差 >20cm。

cann-recipes-embodied-intelligence 把 VSLAM 中计算最密集的三步搬到 NPU:ORB 特征提取(FAST 角点 + BRIEF 描述子)、特征匹配(暴力匹配或 FLANN)、位姿解算(PnP 非线性优化)。NPU 的 Vector 单元做 FAST 角点的并行检测(256 个像素同时比较)、Cube 单元做本质矩阵 SVD 分解。

ORB 特征提取——FAST 角点的并行化

# cann-recipes-embodied-intelligence/slam/orb_extractor_npu.py
#
# ORB = FAST 角点检测 + BRIEF 描述子
# CPU 瓶颈: FAST 每像素做 16 个圆形邻域比较 (640*480*16 = 4.9M 比较/frame)
# NPU 加速: 用卷积实现 FAST + 2D 网格并行

import torch
import torch.nn.functional as F
import torch_npu

class FASTCornerDetectorNPU:
    """
    FAST-9-16 角点检测的 NPU 并行实现

    标准 FAST: 以每个像素为中心,16 个圆形邻域点
              如果连续 9 个邻域点都 > 中心+阈值(更亮)
              或连续 9 个邻域点都 < 中心-阈值(更暗)
              则该像素为角点

    NPU 实现: 16 个圆形邻域 = 16 个固定偏移 → 16 个 gather + 并行比较
    """

    def __init__(self, threshold=20, nms_radius=3):
        self.threshold = threshold
        self.nms_radius = nms_radius

        # 预计算 FAST-16 的 16 个圆形邻域偏移(相对于中心像素)
        # radius = 3 像素的离散圆
        self.offsets = torch.tensor([
            [ 0,  3], [ 1,  3], [ 2,  2], [ 3,  1],
            [ 3,  0], [ 3, -1], [ 2, -2], [ 1, -3],
            [ 0, -3], [-1, -3], [-2, -2], [-3, -1],
            [-3,  0], [-3,  1], [-2,  2], [-1,  3],
        ], dtype=torch.int32)  # [16, 2]

    def detect(self, image: torch.Tensor) -> torch.Tensor:
        """
        并行 FAST 角点检测

        image: [H, W] (灰度图), float 0-255
        returns: keypoints [N, 2] (y, x)
        """
        H, W = image.shape
        device = image.device

        # 边界填充(FAST 需要 3 像素邻域)
        img_padded = F.pad(image.unsqueeze(0).unsqueeze(0), (3, 3, 3, 3), mode='reflect')
        img_padded = img_padded.squeeze(0).squeeze(0)  # [H+6, W+6]

        # 中心像素(每个像素都是一个候选角点)
        center = img_padded[3:H+3, 3:W+3]  # [H, W]

        # 方法一:16 次 gather(Vector 单元并行)
        # 对 16 个偏移位置分别采样
        neighbors = []
        for dy, dx in self.offsets.tolist():
            # 在偏移位置上采样
            neighbor = img_padded[
                3+dy : H+3+dy,
                3+dx : W+3+dx
            ]  # [H, W]
            neighbors.append(neighbor)

        # 拼接为 [16, H, W]
        neighbors = torch.stack(neighbors, dim=0)  # [16, H, W]

        # 并行比较: 16 个邻域 vs 中心
        # brighter: 邻域 > 中心 + threshold
        # darker:   邻域 < 中心 - threshold
        center_expanded = center.unsqueeze(0)  # [1, H, W]

        brighter = neighbors > (center_expanded + self.threshold)  # [16, H, W]
        darker = neighbors < (center_expanded - self.threshold)     # [16, H, W]

        # 快速连续性检测: 用一维卷积在循环维度上找连续 9 个 True
        # 将 16 个邻域复制为 32 个(首尾相接)以处理循环
        brighter_cyclic = torch.cat([brighter, brighter], dim=0)  # [32, H, W]
        darker_cyclic = torch.cat([darker, darker], dim=0)        # [32, H, W]

        # 滑动窗口 sum: kernel_size=9 → 连续 9 个 AND 等价于 sum == 9
        kernel = torch.ones(1, 1, 9, device=device)

        # 对每个 (H, W) 位置,沿 32 维度做一维卷积
        brighter_conv = F.conv1d(
            brighter_cyclic.float().permute(1, 2, 0).reshape(-1, 1, 32),
            kernel, padding=4
        ).reshape(H, W, 32).permute(2, 0, 1)  # [32, H, W]

        darker_conv = F.conv1d(
            darker_cyclic.float().permute(1, 2, 0).reshape(-1, 1, 32),
            kernel, padding=4
        ).reshape(H, W, 32).permute(2, 0, 1)

        # 角点响应: 是否存在任何位置有连续 9 个 brighter 或 darker
        corner_response = (
            (brighter_conv >= 9).any(dim=0) | (darker_conv >= 9).any(dim=0)
        ).float()  # [H, W]

        # NMS: 3×3 局部最大值抑制
        corner_response = self._nms(corner_response)

        # 提取角点坐标
        keypoint_mask = corner_response > 0
        ys, xs = torch.where(keypoint_mask)

        return torch.stack([ys, xs], dim=-1)  # [N, 2]

    def _nms(self, response: torch.Tensor) -> torch.Tensor:
        """
        非极大值抑制(3×3 局部极大值)
        NPU 上用 maxpool2d 一步完成
        """
        r = self.nms_radius
        kernel_size = 2 * r + 1

        # MaxPool = 膨胀 + 比较(如果响应 < maxpool 结果 → 不是局部极大 → 抑制)
        max_response = F.max_pool2d(
            response.unsqueeze(0).unsqueeze(0),  # [1, 1, H, W]
            kernel_size=kernel_size,
            stride=1,
            padding=r
        ).squeeze(0).squeeze(0)

        # 只保留 == max 的像素
        suppressed = response.clone()
        suppressed[response < max_response] = 0

        return suppressed

BRIEF 描述子——256 位二进制描述的并行计算

# cann-recipes-embodied-intelligence/slam/brief_descriptor_npu.py
#
# BRIEF 描述子: 对每个关键点的 31×31 邻域,随机选取 256 对像素
# 每对像素 (p1, p2): 如果 I(p1) < I(p2) → bit=1 else bit=0
# 输出: 256 bits = 32 bytes per keypoint

class BRIEFDescriptorNPU:
    """
    BRIEF 描述子的 NPU 批量计算

    核心: 所有关键点同时计算,不是逐个循环
    [N, 256, 2, 2] 的像素对位置 → gather → 并行比较 → [N, 256] bits
    """

    def __init__(self, patch_size=31, n_bits=256):
        self.patch_size = patch_size
        self.n_bits = n_bits

        # 预生成 256 对随机采样位置(服从高斯分布 N(0, patch_size/5))
        # 注意: 这些位置是固定的(BRIEF 标准和 ORB 一样,有预定义采样模式)
        torch.manual_seed(42)  # 固定种子
        half = patch_size // 2
        std = patch_size / 5.0

        self.pairs = torch.randn(n_bits * 2, 2) * std
        self.pairs = self.pairs.clamp(-half, half).round().int()

        # pairs: [2*n_bits, 2] → reshape to [n_bits, 2, 2]
        self.pairs = self.pairs.view(n_bits, 2, 2)
        # pairs[i, 0] = (dy1, dx1) 第一个像素的偏移
        # pairs[i, 1] = (dy2, dx2) 第二个像素的偏移

    def compute(self, image: torch.Tensor, keypoints: torch.Tensor) -> torch.Tensor:
        """
        批量计算所有关键点的 BRIEF 描述子

        image: [H, W] 灰度图
        keypoints: [N, 2] (y, x)
        returns: descriptors [N, n_bits] (bool)
        """
        H, W = image.shape
        N = keypoints.shape[0]
        half = self.patch_size // 2

        # 边界填充
        img_padded = F.pad(
            image.view(1, 1, H, W),
            (half, half, half, half),
            mode='reflect'
        ).squeeze(0).squeeze(0)  # [H+2h, W+2h]

        keypoints = keypoints.long()

        # 对每个关键点,提取所有 256 对像素位置
        # pairs: [256, 2, 2] → [256, 2]
        # 展开为绝对像素坐标 [N, 256, 2]
        kp_expanded = keypoints.view(N, 1, 1, 2)  # [N, 1, 1, 2]

        # pair 0: [N, 256, 2] (y1, x1)
        p1_offsets = self.pairs[:, 0, :].view(1, self.n_bits, 2)  # [1, 256, 2]
        p1_y = (kp_expanded[..., 0] + p1_offsets[..., 0] + half).clamp(0, img_padded.shape[0]-1)
        p1_x = (kp_expanded[..., 1] + p1_offsets[..., 1] + half).clamp(0, img_padded.shape[1]-1)

        # pair 1: [N, 256, 2] (y2, x2)
        p2_offsets = self.pairs[:, 1, :].view(1, self.n_bits, 2)
        p2_y = (kp_expanded[..., 0] + p2_offsets[..., 0] + half).clamp(0, img_padded.shape[0]-1)
        p2_x = (kp_expanded[..., 1] + p2_offsets[..., 1] + half).clamp(0, img_padded.shape[1]-1)

        # 批量 gather: 一次性取出所有 N*256*2 个像素值
        I_p1 = img_padded[p1_y, p1_x]  # [N, 256]
        I_p2 = img_padded[p2_y, p2_x]  # [N, 256]

        # 并行比较: I(p1) < I(p2) → 1
        descriptors = I_p1 < I_p2  # [N, 256], bool

        return descriptors

    def match(self, desc1: torch.Tensor, desc2: torch.Tensor, threshold=50):
        """
        暴力匹配: 用 Hamming 距离找最佳匹配对

        desc1: [N1, 256]
        desc2: [N2, 256]
        returns: matches [M, 2] (idx1, idx2)
        """
        # Hamming 距离 = XOR 后 count 1 bits
        # 并行计算: [N1, N2] 的距离矩阵

        # desc1: [N1, 1, 256], desc2: [1, N2, 256]
        xor_result = desc1.unsqueeze(1) ^ desc2.unsqueeze(0)  # [N1, N2, 256]

        # count 1 bits → popcount (NPU 不支持原生 popcount,用 sum)
        hamming_dist = xor_result.sum(dim=-1)  # [N1, N2]

        # 对每个 desc1 找最小的 desc2
        best_dist, best_idx = hamming_dist.min(dim=1)  # [N1]

        # 过滤距离 > threshold 的匹配
        valid = best_dist < threshold

        matches = torch.stack([
            torch.arange(len(desc1), device=desc1.device)[valid],
            best_idx[valid]
        ], dim=1)  # [M, 2]

        return matches

本质矩阵估算与 SVD 分解

# cann-recipes-embodied-intelligence/slam/pose_estimation_npu.py
#
# 从 2D-2D 匹配点估计相机运动: E = t^R (本质矩阵)
# 8 点法 → SVD 分解 → R, t

class EssentialMatrixEstimatorNPU:
    """
    本质矩阵估算:8 点法 + SVD 分解

    NPU 加速点:
    - 8 点法的 SVD: torch.svd 底层调用 cuSOLVER → Cube 单元做矩阵分解
    - RANSAC 的多次迭代: 在 NPU 上并行 50 次采样 → 一次 SVD 批量
    """

    def estimate(self, pts1: torch.Tensor, pts2: torch.Tensor,
                 K: torch.Tensor, num_ransac=50, threshold=1.0):
        """
        从匹配点对估计相机运动

        pts1: [N, 2]  图像 1 中的匹配点(已归一化,像素坐标)
        pts2: [N, 2]  图像 2 中的匹配点
        K: [3, 3] 相机内参矩阵
        num_ransac: RANSAC 迭代次数
        threshold: 内点距离阈值(像素)

        returns: R [3,3], t [3], inlier_mask [N]
        """
        N = pts1.shape[0]

        # 归一化: 像素坐标 → 相机坐标(除以焦距)
        fx, fy = K[0, 0], K[1, 1]
        cx, cy = K[0, 2], K[1, 2]

        pts1_norm = torch.stack([
            (pts1[:, 0] - cx) / fx,
            (pts1[:, 1] - cy) / fy,
        ], dim=1)  # [N, 2]

        pts2_norm = torch.stack([
            (pts2[:, 0] - cx) / fx,
            (pts2[:, 1] - cy) / fy,
        ], dim=1)  # [N, 2]

        best_inliers = 0
        best_R = None
        best_t = None
        best_mask = None

        for _ in range(num_ransac):
            # 随机采样 8 对匹配点
            if N < 8:
                break

            indices = torch.randperm(N, device=pts1.device)[:8]

            p1 = pts1_norm[indices]  # [8, 2]
            p2 = pts2_norm[indices]  # [8, 2]

            # 8 点法: 构建设计矩阵 A
            # 约束: p2^T * E * p1 = 0
            # 展开: [u1*u2, v1*u2, u2, u1*v2, v1*v2, v2, u1, v1, 1] * vec(E) = 0
            u1, v1 = p1[:, 0], p1[:, 1]
            u2, v2 = p2[:, 0], p2[:, 1]

            A = torch.stack([
                u1 * u2, v1 * u2, u2,
                u1 * v2, v1 * v2, v2,
                u1, v1, torch.ones(8, device=pts1.device)
            ], dim=1)  # [8, 9]

            # SVD: A = U S V^T → E 是 V 的最后一列
            U, S, Vh = torch.linalg.svd(A, full_matrices=False)
            # Vh: [9, 9] → 最后一列 → reshape to [3, 3]
            e_vec = Vh[-1, :]  # [9]
            E = e_vec.view(3, 3)

            # 强制 E 的秩为 2(SVD → 设最小奇异值为 0 → 重组)
            Ue, Se, Veh = torch.linalg.svd(E, full_matrices=False)
            Se[2] = 0
            E = Ue @ torch.diag(Se) @ Veh

            # 从 E 提取 R, t(四种可能中选一种)
            R, t = self._recover_pose(E)

            # 验证:对每个点计算点到极线的距离
            inlier_mask = self._compute_inliers(
                pts1_norm, pts2_norm, E, threshold
            )

            n_inliers = inlier_mask.sum().item()
            if n_inliers > best_inliers:
                best_inliers = n_inliers
                best_R = R
                best_t = t
                best_mask = inlier_mask

        return best_R, best_t, best_mask

    def _recover_pose(self, E):
        """从本质矩阵 E 恢复 R, t"""
        # SVD: E = U diag(1,1,0) V^T
        U, S, Vh = torch.linalg.svd(E)

        # 确保 det(R) > 0
        if torch.det(U) < 0:
            U[:, -1] *= -1
        if torch.det(Vh) < 0:
            Vh[-1, :] *= -1

        # W 矩阵(绕 Z 轴旋转 ±90°)
        W = torch.tensor([
            [0, -1, 0],
            [1,  0, 0],
            [0,  0, 1]
        ], dtype=torch.float32, device=E.device)

        # 四种可能: R = U W V^T 或 U W^T V^T
        #             t = U[:, 2] 或 -U[:, 2]
        R1 = U @ W @ Vh
        R2 = U @ W.T @ Vh
        t_candidate = U[:, 2]

        # 选择 det(R) > 0 的解
        if torch.det(R1) > 0:
            R = R1
        else:
            R = R2

        return R, t_candidate

    def _compute_inliers(self, pts1, pts2, E, threshold):
        """计算内点: 点到极线距离 < threshold"""
        # 极线: l2 = E @ p1 (注意齐次坐标)
        # 距离: |p2^T * l2| / sqrt(l2[0]^2 + l2[1]^2)
        ones = torch.ones(pts1.shape[0], 1, device=pts1.device)

        p1_h = torch.cat([pts1, ones], dim=1)  # [N, 3]
        p2_h = torch.cat([pts2, ones], dim=1)  # [N, 3]

        # 极线向量
        epipolar_line = p1_h @ E.T  # [N, 3]

        # Sampson 距离(几何误差的近似)
        # d = (p2^T E p1)^2 / ((E p1)_x^2 + (E p1)_y^2 + (E^T p2)_x^2 + (E^T p2)_y^2)
        numerator = (p2_h * epipolar_line).sum(dim=1) ** 2  # [N]

        l2_xy = epipolar_line[:, :2].norm(dim=1) ** 2  # [N]

        epipolar_line2 = p2_h @ E  # [N, 3]
        l1_xy = epipolar_line2[:, :2].norm(dim=1) ** 2  # [N]

        denominator = l2_xy + l1_xy + 1e-8  # 避免除零
        sampson_dist = numerator / denominator  # [N]

        return sampson_dist < threshold

PnP 位姿优化——非线性最小二乘的 NPU 批处理

# cann-recipes-embodied-intelligence/slam/pnp_solver_npu.py
#
# PnP (Perspective-n-Point): 从 3D-2D 对应点优化位姿
# Gauss-Newton 迭代: J^T J Δx = J^T r → 求解正规方程

class PnPSolverNPU:
    """
    PnP: 已知 3D 点(世界坐标)和 2D 观测(像素坐标)→ 优化相机位姿

    NPU 加速: 批量计算雅可比矩阵(所有点同时做链式求导)
    + torch.linalg.solve 做 Cube 矩阵分解
    """

    def solve(self, points_3d: torch.Tensor, points_2d: torch.Tensor,
              K: torch.Tensor, max_iters=10):
        """
        points_3d: [N, 3] 世界坐标中的 3D 点
        points_2d: [N, 2] 图像中的 2D 观测
        K: [3, 3] 相机内参

        returns: R [3,3], t [3]
        """
        N = points_3d.shape[0]
        device = points_3d.device

        # 初始位姿: 零旋转 + 零平移
        R = torch.eye(3, device=device)
        t = torch.zeros(3, device=device)

        fx, fy = K[0, 0], K[1, 1]
        cx, cy = K[0, 2], K[1, 2]

        for iteration in range(max_iters):
            # 投影: 3D → 2D
            P_cam = points_3d @ R.T + t  # [N, 3]
            u_proj = fx * P_cam[:, 0] / P_cam[:, 2] + cx  # [N]
            v_proj = fy * P_cam[:, 1] / P_cam[:, 2] + cy  # [N]

            # 残差: 观测 - 投影
            r_u = points_2d[:, 0] - u_proj  # [N]
            r_v = points_2d[:, 1] - v_proj  # [N]
            residuals = torch.stack([r_u, r_v], dim=1).flatten()  # [2N]

            # 雅可比: dr/dξ(6 维位姿李代数)
            # 对每个点: J_i = [dr_u/dξ ; dr_v/dξ]  (2×6)
            J = self._compute_jacobian_batch(
                points_3d, R, t, fx, fy, cx, cy
            )  # [N, 2, 6]

            # 展开雅可比矩阵: [2N, 6]
            J_flat = J.reshape(2*N, 6)

            # 正规方程: J^T J Δξ = J^T r
            JTJ = J_flat.T @ J_flat  # [6, 6]
            JTr = J_flat.T @ residuals  # [6]

            # 求解 Δξ
            delta_xi = torch.linalg.solve(JTJ, JTr)  # [6]

            # 更新位姿 (SE(3) 指数映射)
            R, t = self._update_pose(R, t, delta_xi)

            # 收敛判断
            if delta_xi.norm() < 1e-6:
                break

        return R, t

    def _compute_jacobian_batch(self, points_3d, R, t, fx, fy, cx, cy):
        """
        批量计算雅可比矩阵

        对每个 3D 点 P:
        - 投影误差对相机坐标系的导数(2×3)
        - 相机坐标系对位姿李代数的导数(3×6)
        - 链式法则: J = (2×3) @ (3×6) → (2×6)
        """
        N = points_3d.shape[0]
        device = points_3d.device

        # 3D 点转换到相机坐标系
        P_cam = points_3d @ R.T + t  # [N, 3]
        X, Y, Z = P_cam[:, 0], P_cam[:, 1], P_cam[:, 2]

        # 投影误差对相机坐标的导数: [N, 2, 3]
        du_dP = torch.zeros(N, 3, device=device)
        dv_dP = torch.zeros(N, 3, device=device)

        du_dP[:, 0] = fx / Z
        du_dP[:, 2] = -fx * X / (Z * Z)

        dv_dP[:, 1] = fy / Z
        dv_dP[:, 2] = -fy * Y / (Z * Z)

        # 相机坐标对位姿的导数 (每个点): [3, 6]
        # dP/dξ = [I_3, -P^∧] → 3×6
        # 对每个点计算
        dP_dxi = torch.zeros(N, 3, 6, device=device)

        # 位置部分: dP/dt = I_3
        dP_dxi[:, :, :3] = torch.eye(3, device=device).unsqueeze(0).expand(N, -1, -1)

        # 旋转部分: dP/dω = -[P]× (叉乘矩阵的负)
        dP_dxi[:, 0, 4] = Z     # X 对 ω_y 的导数 = +Z(叉乘: ω×P 的 X 分量)
        dP_dxi[:, 0, 5] = -Y    # X 对 ω_z 的导数 = -Y
        dP_dxi[:, 1, 3] = -Z    # Y 对 ω_x 的导数 = -Z
        dP_dxi[:, 1, 5] = X     # Y 对 ω_z 的导数 = X
        dP_dxi[:, 2, 3] = Y     # Z 对 ω_x 的导数 = Y
        dP_dxi[:, 2, 4] = -X    # Z 对 ω_y 的导数 = -X

        # 链式法则: J_{2×6} = du/dP_{2×3} @ dP/dξ_{3×6}
        J = torch.zeros(N, 2, 6, device=device)

        du_dP_all = torch.stack([du_dP, dv_dP], dim=1)  # [N, 2, 3]
        J = torch.bmm(du_dP_all, dP_dxi)  # [N, 2, 6]

        return J

    def _update_pose(self, R, t, delta_xi):
        """SE(3) 位姿更新: Δξ → (ΔR, Δt) → (R_new, t_new)"""
        # 李代数 se(3) → 李群 SE(3)
        # ξ = [ρ; φ],  φ 是旋转向量, ρ 是平移
        rho = delta_xi[:3]  # 平移部分
        phi = delta_xi[3:]   # 旋转部分

        # 旋转: 罗德里格斯公式
        theta = phi.norm()
        if theta < 1e-8:
            dR = torch.eye(3, device=R.device)
        else:
            axis = phi / theta
            K = torch.tensor([
                [0, -axis[2], axis[1]],
                [axis[2], 0, -axis[0]],
                [-axis[1], axis[0], 0]
            ], device=R.device)

            dR = (torch.eye(3, device=R.device) +
                  torch.sin(theta) * K +
                  (1 - torch.cos(theta)) * K @ K)

        # 平移: V 矩阵(SO(3) 的雅可比)
        if theta < 1e-8:
            V = torch.eye(3, device=R.device)
        else:
            K = dR.new_zeros(3, 3)
            K[0, 1] = -axis[2]; K[0, 2] = axis[1]
            K[1, 0] = axis[2]; K[1, 2] = -axis[0]
            K[2, 0] = -axis[1]; K[2, 1] = axis[0]

            V = (torch.eye(3, device=R.device) +
                 (1 - torch.cos(theta)) / (theta**2) * K +
                 (theta - torch.sin(theta)) / (theta**3) * K @ K)

        dt = V @ rho

        R_new = dR @ R
        t_new = dt + t

        return R_new, t_new

踩坑:FAST 角点检测的采样偏移超出图像边界——边界 3 像素区域报 CUDA 索引错误

# ❌ 直接对边界像素做 FAST 检测 → 偏移到负数坐标
# neighbor = img_padded[3+dy: H+3+dy, 3+dx: W+3+dx]
# dy=-3, dx=0 → img_padded[0:H, 3:W+3] ← 右下偏移超出 → index out of bounds

# ✅ 对称 padding: 边界镜像扩展 3 像素(reflect mode)
# 确保所有偏移都在 padding 后的图像内
img_padded = F.pad(
    image.unsqueeze(0).unsqueeze(0),
    (3, 3, 3, 3), mode='reflect'  # 镜像边界
)

# 并且最终只对原始图像区域 [3:H+3, 3:W+3] 提取角点响应
# 边界 3 像素不做角点检测(物理上 FAST 也无法检测完整 16 邻域)

踩坑:奇异值分解中 E 矩阵秩不为 2——8 点法噪声导致最小时奇异值残留

# ❌ E 的奇异值: [σ1, σ2, σ3] = [123.4, 89.2, 0.03]
# 强制设 σ3=0 → E 秩为 2 → 但 σ3=0.03 不是纯噪声 → 丢失了信息 → 平移方向偏 0.5°
# 本质: 当 σ3/σ2 > 0.01 时,设为零会引入偏差

# ✅ 自适应奇异值修正:当 σ3 实际很大时(>1% σ2),考虑用 σ1+σ2 均值替换
def fix_essential_matrix(E, noise_threshold=0.01):
    """自适应本质矩阵秩修正"""
    U, S, Vh = torch.linalg.svd(E)

    S_corrected = S.clone()

    # 如果 σ3 显著不等于零 → 测量噪声 → 取 σ1 和 σ2 的均值
    if S[2] > noise_threshold * S[1]:
        S_corrected[2] = (S[0] + S[1]) / 2.0
    else:
        S_corrected[2] = 0.0

    E_fixed = U @ torch.diag(S_corrected) @ Vh
    return E_fixed

cann-recipes-embodied-intelligence 的 VSLAM NPU 加速方案:FAST 角点检测用 16 路并行比较 + 1D 循环卷积找连续 9 个邻域(替代逐像素 for 循环,4.9M 次比较→单次 kernel launch),BRIEF 描述子用 256 对随机像素位置的批量 gather(N 个关键点同时计算 32 字节描述子),本质矩阵 SVD 分解用 Cube 单元做 3×3 矩阵对角化,PnP 用批量雅可比计算 + 正规方程求解(6×6 solve 在 Cube 上)。单帧从 CPU 45ms 降到 NPU 8ms,满足 30fps 实时定位。踩坑:FAST 边界偏移超出图像→reflect padding 3px、SVD 秩修正 σ3 残留→自适应阈值(σ3/σ2>1% 时取均值替代零)。

Logo

作为“人工智能6S店”的官方数字引擎,为AI开发者与企业提供一个覆盖软硬件全栈、一站式门户。

更多推荐