Taxonomy of deformable object shape control

control of deformable objects (or shape control) solves the problem of autonomously manipulating deformable objects so that they acquire a desired target shape (Fig. 1). This definition, while straightforward and giving a valid general idea, is open-ended.

Mature disciplines, such as deformable solid mechanics, achieve the formalisation of some elements involved in the shape control problem (e.g. the characterisation of deformable materials from a physics point of view). However, these precise definitions stand in contrast to the wide variety of tasks and objects involved in many of the practical applications of shape control. Shape control tasks often arise in areas such as the manufacturing industry, food processing, or surgical procedures, where objects and shapes are complex and diverse, thus it becomes necessary to classify them in a more general and manageable manner. The same applies to the characterisation of other elements of the shape control problem, such as the object's available information (sensing), the nature of the employed control strategies or the shape error definition.

This paper addresses the need for a taxonomy of elements in shape control. With a growing body of literature, confusion arises in many concepts and definitions. To name a few examples: regarding the problem's dimensionality, there exists ambiguity between whether 3D shape control refers to dimensions of the workspace or dimensions of the object. When willing to compare methods, one faces numerous definitions of shape error, ranging from the comparison of discrete features to error definitions through surface point clouds or continuous curves. Other aspects such as the blending of mechanical and control concepts can lead to confusion, as seen with terms such as model-free control, which, from a control engineering point of view, often implies the use of a (non-mechanical) model, even if it is highly local or simplified.

Surveys in the Deformable Object Manipulation (DOM) literature (Herguedas et al., 2019; Sanchez et al., 2018; Zhu et al., 2022; Arriola-Rios et al., 2020; Yin et al., 2021) consider shape control as one of the many specific tasks that can be performed on non-rigid objects (e.g. cutting, folding, knotting, etc.). We now summarise their classification criteria of DOM elements.

In (Herguedas et al., 2019), authors classify state-of-the-art approaches into several categories based on criteria such as deformation models (precomputed, learned or model-free), object dimensionality (1D, 2D and 3D), control strategies (classical, robust or adaptive control), perception systems (force-based, vision-based or a combination of both) and the predominant actions involved (deformation or transport actions). They provide a comprehensive review of the different methodologies, highlighting their advantages, limitations, and areas of application.

Authors in (Sanchez et al., 2018) classify deformable objects into four categories according to their physical properties and geometry: uniparametric with no compression strength (cables, ropes), biparametric (flat) with large strain or displacement, biparametric without compressive strength (cloth type), and triparametric objects (sponges, food). They further classify research approaches according to the type of task performed on the deformable object, such as sensing, manipulation and task-specific actions.

Survey (Zhu et al., 2022) analyses challenges of robotic manipulation of deformable objects, covering design, sensing, modelling, planning, and control. It highlights the need to customise grippers to meet the specific requirements of deformable objects and classifies sensing techniques into visual, tactile and force methods. They distinguish between modelling approaches using frequently updated local models and complex global models using methods such as finite elements or neural networks. Planning strategies are also discussed, emphasising the need to take into account the high degrees of freedom of objects and advocating contact-centred planning. Control strategies are divided into model-based and model-free methods.

Focusing on the modelling of deformable objects, (Arriola-Rios et al., 2020) presents a structured exploration of deformable object manipulation in robotics across four key areas: shape representation, dynamics modelling, parameter learning and estimation, and manipulation strategies. Shape representation is divided into four types of models: implicit curves and surfaces (e.g. algebraic curves, level set methods), explicit parameterised representations (e.g. splines), free-form, and discrete representations (e.g. meshes), which provide the mathematical basis for modelling deformable objects. Dynamic modelling is classified into particle-based models (particle systems, mass-spring systems), constitutive models (finite element, volume, difference methods, boundary element method, long element method) and their approximations (modal analysis, active contours), describing in detail how these models simulate deformations under external forces. Parameter learning and estimation methods are classified into direct estimation, error minimisation (exhaustive searches, iterative methods, genetic algorithms, neural networks) and probabilistic methods. Finally, manipulation strategies involve planning object manipulation and controlling deformation, using both model-based and real-time sensor-based approaches.

A classification system is outlined in (Yin et al., 2021), incorporating modelling approaches, perception techniques, and manipulation strategies. With focus on physics-based modelling, (Yin et al., 2021) classifies deformation models into mass-spring systems, position-based dynamics and continuous (mechanic) models. Perception techniques are described in detail, distinguishing between types of sensors (vision, force, touch) and the different perceptual tasks they address, such as state estimation, identification of material properties, object detection, segmentation, and classification. Their classification of manipulation strategies elaborates on methodologies for planning, controlling and learning the manipulation of deformable objects. Furthermore, within their classification tables, they consider the dimensionality of the objects (1D, 2D, 3D), the types of sensors being used and whether single or multiple manipulators are involved.

The analysis of DOM studies (Herguedas et al., 2019; Sanchez et al., 2018; Yin et al., 2021; Zhu et al., 2022; Arriola-Rios et al., 2020) reveals an important gap in the literature. While general surveys (Herguedas et al., 2019; Sanchez et al., 2018; Yin et al., 2021; Zhu et al., 2022) provide a valuable general analysis of DOM, they fall short in offering the necessary depth and structure for shape control. In contrast, tutorials like (Arriola-Rios et al., 2020) offer a meticulous and in-depth examination of DOM components, such as deformable object modelling techniques, yet they maintain a highly specific focus. Hence, the literature either discusses DOM broadly or concentrates on very specific elements, lacking a comprehensive and hierarchical analysis of shape control as a relevant subdomain within DOM.

In our proposed taxonomy of shape control, we fill a gap between broad DOM surveys and very detailed and specific studies. By focusing on the fundamentals, our analysis avoids redundancy by omitting details on well-covered topics, such as existing sensor types and the classic literature on control, as discussed in (Herguedas et al., 2019), or shape representations, explored in depth in (Arriola-Rios et al., 2020). On the other hand, it incorporates relevant elements overlooked in existing surveys on deformable object manipulation (DOM), such as the definition of shape error for shape comparison, the information acquisition source and acquisition phase, and the intrinsic and extrinsic dimensionalities of the object. These aspects represent novel analyses not covered in previous DOM surveys (Herguedas et al., 2019; Sanchez et al., 2018; Yin et al., 2021; Zhu et al., 2022; Arriola-Rios et al., 2020). We also avoid unnecessary complexity by maintaining a holistic but practical format: our taxonomy follows a shape control system structure, with each main category serving as a fundamental building block. We ensure hierarchical coherence between elements of similar descriptive importance. The balance between generality and detail, the coherent hierarchy, the control-oriented modularity and the user-friendly figures and examples make this taxonomy accessible and suitable for characterising existing and future methods, serving as a valuable tool for shape control analysis.