Estratégias de aprendizado por esforço para controle ótimo online de manipuladores robóticos

Abstract

The inaccuracies and uncertainties in the parameters of a robot manipulator, such as payload variations, inaccurate measurements of the mass and/or inertia of the links, unknown backlash or friction in gears, among others, can significantly affect the robot performance, causing steady-state and trajectory following errors. Adaptive controllers are a suitable alternative for these systems since their main feature is the capability to learn online using real-time parameter estimation. Nevertheless, adaptive controllers are not usually designed to be optimal in the sense of minimizing cost functions as defined in the optimal control context, and thus, are not suitable to applications in which the use of optimal control strategies is required. In this work, an unified approach of adaptive control and optimal control based on concepts and methods of reinforcement learning is proposed aiming at the development of algorithms for the design of online optimal control systems with applications in robotic manipulator control. A parametric structure is used to approximate the value function in order to avoid the curse of dimensionality problem. The estimation of these parameters will be performed through the Recursive Least Squares (RLS) estimator at each time step. With respect to the control policy, two approaches will be implemented in the updating step: approximate policy improvement, where a representation by function approximators is used, and exact policy improvement, where control actions are calculated exactly through the value function. The main advantage of the proposed control methodology is that no prior information of the manipulator parameters is needed for its implementation, only the measurements of the states and the control signal are used. The evaluation of the control schemes is carried out in a UR10 robotic model on V-REP simulator for the regulation and tracking tasks, and payload variations