---
title: ROS 2 camera tracking and robotic arm control
description: Understand a SHAPE example that combines hand tracking, ArUco markers and ROS 2 messages to control a robotic arm through visual programming logic.
url: "https://thetactigon.com/docs/ros-2-camera-tracking-robotic-arm/"
updated: "2026-10-27"
category: ROS2
---

# ROS 2 camera tracking and robotic arm control

Explore how ROS 2 camera tracking connects visual input to robotic actions in Tactigon SHAPE.

In this example, a camera detects a hand pointing at an ArUco marker. A ROS 2 node sends the marker information to SHAPE, where visual programming blocks use it to select a robotic arm’s pick or drop routine.

This guide explains the architecture and application logic. The coordinates and movement sequences belong to the documented demonstration and must be adapted to the actual robot and workspace.

## Before you begin

For the communication concepts and visual blocks used here, read:

- [Getting started with ROS 2 in Tactigon SHAPE](https://thetactigon.com/docs/ros-2-shape-getting-started/)
- **ROS 2 blocks in Tactigon SHAPE**

The camera-tracking example requires a camera, its ROS 2 environment and the corresponding custom message definitions. Controlling the arm also requires the robot integration used by the demonstration.

## How the camera-tracking system works

The vision node combines two technologies:

- **MediaPipe** tracks hand landmarks.
- **ArUco marker detection** identifies numbered markers in the camera image.

The application uses this information to determine which marker the user is pointing at. It publishes a pointing message for SHAPE to receive.

A marker’s identifier becomes an input to the program. The visual logic then decides which routine to execute.

![Camera tracking sends a marker identifier to SHAPE, which selects a pick or drop routine according to the holding variable.](https://thetactigon.com/wp-content/uploads/2026/10/shape-ros-2-camera-tracking-flow.webp)
Conceptual overview of the camera-tracking control flow.

## Understand the ROS 2 messages

The integration defines three custom message structures:

| Message | Content |
| --- | --- |
| `Point2D` | Image coordinates: `x` and `y` |
| `Marker` | Marker identifier and four corner coordinates |
| `MarkerList` | A collection of detected markers |

The marker identifier is the field used by the robotic arm example to distinguish the two target positions.

## Inspect the camera topics

The documented integration uses these topics:

| Topic | Purpose |
| --- | --- |
| `/image_raw` | Camera image input |
| `/camera_tracking/markers` | Detected markers |
| `/camera_tracking/pointing` | Marker selected through pointing |
| `/camera_tracking/image` | Image stream with diagnostic overlays |

The image stream helps you inspect detection; the pointing topic supplies the marker data used by the application logic.

## Receive the selected marker in SHAPE

The **On topic** block subscribes to `/camera_tracking/pointing` using the corresponding custom message type.

When a message arrives, the program extracts its `id` field from the payload and stores it in:

```
marker_pointed_id
```

In the example:

- Marker `0` identifies the right target.
- Marker `1` identifies the left target.

These assignments belong to the demonstration’s layout.

## Control the pick-and-drop sequence

The program uses a Boolean variable named `holding` to track whether the arm is carrying an object.

![SHAPE visual blocks receiving a ROS 2 marker identifier and controlling robotic arm pick-and-drop routines.](https://thetactigon.com/wp-content/uploads/2026/10/shape-ros-2-camera-tracking-blocks.webp)
Original SHAPE example showing initialisation, marker reception and robotic arm routines.

### Initialise the program

The setup routine prepares the gripper and arm, moves through the demonstration’s home sequence and sets:

```
holding = false
```

### Pick an object

When `holding` is `false`, a recognised target selects the pickup routine.

The arm moves to the target, closes the gripper and returns to its home position. The program then sets:

```
holding = true
```

### Drop the object

When `holding` is `true`, the selected target determines the drop routine.

The arm moves to the destination, releases the object and returns home. The program resets:

```
holding = false
```

### Reset the marker selection

After each completed routine, the example assigns:

```
marker_pointed_id = 999
```

This clears the previously stored selection so the main loop does not immediately repeat the action using that same value.

It does not, by itself, prevent a newly received pointing message from selecting another action.

## Adapt the example to your project

Keep the communication pattern separate from the robot-specific movement configuration.

Marker identifiers select routines; those routines contain the positions and commands for the actual arm. The demonstration’s coordinates are not universal robot settings.

Before reproducing the movement, confirm the robot model, coordinate conventions, reachable positions and operating procedure for your setup.

## Next steps

Use **ROS 2 blocks in Tactigon SHAPE** to review subscription and payload handling.

For the implementation reference, consult the [official camera-tracking and ROS 2 documentation](https://github.com/TactigonTeam/Tactigon-Shape/wiki/ros-2-robot-operating-system?utm_source=chatgpt.com).