Overview

The Franka Robot which has 7-dof picks up the lightsaber and knocks down enemies represented by red blocks while protecting the allies represented by blue blocks. These enemies and allies are arbitrarily positioned.

Description

1) Frame Positions (Computer Vision) The transforms of the camera and workspace from the robot are obtained using AprilTags. The frames are fixed to the most recent transform so that even if the AprilTag is removed or covered by a fallen enemy, we can still know the relative positions from each other from the latest calibration.

2) Detections (Computer Vision)
The RealSense's aligned depth image is used to deproject the pixels in the RGB image into real world coordinates. For reliability, several filters are used (depth, contour area, location, and opening/closing) to ensure only the relevant objects are detected. Enemies and allies are detected, and their centroids give their position. Their labelling is sorted in x direction. The count of enemies vanquished is also calculated.

3) Franka control
The Franka then performs knockout motion by first checking for a left swing, stab, and then right swing possibility to knock out enemies without harming any allies, while keeping the ally safety as the highest priority. It decides to NOT execute any motion if harm of any ally cannot be avoided.

After the Franka has held the lightsaber in the home position and the service to is called, the allies are added as collision object into the scene.

Future Scope

Develop a more robust motion planning to increase its ability to tackle plan and execute decisions for more edge cases and other position variations of enemies and allies.

Team Members:
Nick Morales
Megan Sindelar
Vaishnavi Dornadula
Sushma S Chandra