BrickCraft-Duo

Efficient Dual-Arm Skill Learning and Refinement for Compositional Long-Horizon Assembly

Jichuan Yu1, Zhenyu Xiao1, Ze Wang1, Ruixuan Liu2, Changliu Liu2, Chuxiong Hu1
1 Tsinghua University     2 Carnegie Mellon University
Abstract

Interlocking brick assembly provides a representative testbed for evaluating real-world robotic manipulation capabilities, where diverse structural designs, complex inter-step dependencies, intricate mechanical interactions and tight insertion tolerances pose substantial challenges. We present BrickCraft-Duo, a modular framework for long-horizon dual-arm collaborative assembly of interlocking bricks through data-efficient skill learning and composition. BrickCraft-Duo learns reusable single- and dual-arm assembly skills from diverse demonstrations, with bilateral symmetry alignment facilitating skill sharing across symmetric arms and assembly–support role assignments. Guided by stability-aware assembly reasoning, BrickCraft-Duo composes heterogeneous skills to achieve autonomous long-horizon execution, and further integrates human-in-the-loop correction for targeted skill refinement. The resulting system achieves long-horizon success rates of at least 60% and step-level completion rates of at least 95% across five real-world assembly tasks involving partially supported configurations, with horizons of up to nine steps.

Framework Overview
BrickCraft-Duo framework: a target brick structure is converted into an assembly plan and compiled into a dual-arm skill execution graph composed of Transit, Pick, UniAssembly, and CoAssembly skills

Given a target structure, BrickCraft-Duo autonomously reasons about a feasible assembly plan. It then instantiates and composes reusable skills into a skill execution graph for coordinated long-horizon execution.

UniAssembly vs. CoAssembly

Two arms are better than one! For partially supported structures, UniAssembly can cause the structure to deform or detach, necessitating dual-arm cooperative insertion.

UniAssembly
→
CoAssembly
UniAssembly
→
CoAssembly
Autonomous Long-Horizon Assembly
16×
Snake4 assembly steps
InitialRefined
Success Rate5/5—
Completion Rate100%—
16×
Gate8 assembly steps
InitialRefined
Success Rate1/54/5
Completion Rate85.0%97.5%
16×
Toy Gun4 assembly steps
InitialRefined
Success Rate4/5—
Completion Rate95.0%—
16×
Scale9 assembly steps
InitialRefined
Success Rate0/54/5
Completion Rate75.6%97.8%
16×
Eagle9 assembly steps
InitialRefined
Success Rate0/53/5
Completion Rate73.3%95.6%

Completion Rate: the ratio of successfully completed assembly steps to the total number of assembly steps in a long-horizon task.

Adaptation to Workspace Perturbations

BrickCraft-Duo does not require precise workspace calibration. The learned assembly skills can adapt to workspace perturbations with closed-loop visual feedback, enabling robust long-horizon assembly under spatial uncertainties.

Structure Perturbation
8×
8×
Baseplate Perturbation
8×
8×