ARRC: Advanced Reasoning Robot Control
Knowledge-Driven Autonomous Manipulation Using Retrieval-Augmented Generation.
Overview
We bridge the gap between high-level reasoning and low-level control by introducing ARRC (Advanced Reasoning Robot Control). This RAG-enabled robotic manipulation pipeline unifies perception, retrieval, and safe plan execution.
We deploy this system on a UFactory xArm 850 equipped with a RealSense D435 and a Dynamixel gripper, integrating retrieval of robot-centric safety heuristics and procedural templates at inference time. This design offers both adaptability and reliability, enabling the injection of new task knowledge or safety rules without retraining.
Software Architecture
The system architecture consists of three main components:
1. Perception: AprilTags combined with depth data from an Intel RealSense D435 provide marker-based detections fused to recover metric 3D poses in the robot frame.
2. Retrieval & Planning: A curated robotics knowledge base (movement primitives, templates, safety heuristics) is embedded and indexed. At inference time, relevant context is retrieved and passed to an LLM to generate structured JSON plans.
3. Execution: JSON plans are validated and executed via the XArm Python SDK. Execution is safeguarded by software safety gates, including workspace limits, speed caps, and gripper torque gating.
Results & Achievements
We benchmarked the system on tabletop manipulation tasks (scan, approach, pick–place).
Plan Validity: 80% success rate in generating valid, executable paths.
Approach Accuracy: 87.1% average accuracy in placing the gripper in the vicinity of the object.
Pick & Place: 80% success rate in full manipulation tasks.
Adaptability: The system demonstrated advanced reasoning by automatically transitioning scanning strategies (from horizontal to arc scan) when objects were occluded, successfully recovering and completing the task.