ALET (Automated Labeling of Equipment and Tools): A Dataset for Tool Detection and Human Worker Safety Detection

October 22, 2022

Fatih Can Kurnaz, Burak Hocaoğlu, Mert Kaan Yılmaz, İdil Sülo & Sinan Kalkan

Abstract

Robots collaborating with humans in realistic environments need to be able to detect the tools that can be used and manipulated. However, there is no available dataset or study that addresses this challenge in real settings. In this paper, we fill this gap with a dataset for detecting farming, gardening, office, stonemasonry, vehicle, woodworking, and workshop tools. The scenes in our dataset are snapshots of sophisticated environments with or without humans using the tools. The scenes we consider introduce several challenges for object detection, including the small scale of the tools, their articulated nature, occlusion, inter-class invariance, etc. Moreover, we train and compare several state of the art deep object detectors (including Faster R-CNN, Cascade R-CNN, YOLOv3, RetinaNet, RepPoint, and FreeAnchor) on our dataset. We observe that the detectors have difficulty in detecting especially small-scale tools or tools that are visually similar to parts of other tools. In addition, we provide a novel, practical safety use case with a deep network which checks whether the human worker is wearing the safety helmet, mask, glass, and glove tools. With the dataset, the code and the trained models, our work provides a basis for further research into tools and their use in robotics applications.

European Conference on Computer Vision (ECCV) 2020 Workshops.