Industrial Robotic Arm Design enabled by AI-based Deep Reinforcement Learning using a combination of Deep Deterministic Policy Gradient (DDPG) and Hindsight Experience Replay (HER)

Abstract

Robotic arms have huge impact in Industrial process automation and help in significantly reducing the manufacturing time and cost of production. Most of the industrial robots deployed across various industries are programmed to perform repetitive tasks like welding, painting, packaging, moving or putting objects in predefined trajectories. This paper presents an implementation of additional intelligence imparted to a robotic arm for manufacturing purposes using Reinforcement Learning technique. In current scenario, the robotic arms are using inverse kinematics to operate in their environment. In general, the desired goal to reach is represented in Cartesian co-ordinates and it is converted to Joint angles using the inverse kinematics. The robotic arm should move it’s joints in certain angles and reach the final position. Solving the kinematics demands high computational power due to solving of multi-dimensional matrices as a part of calculating the joint angles. It is also passive calculation and there is no feedback system to check and correct the position during next iteration. This process is also prone to errors that arise due to matrix inversions during calculation. The Deep reinforcement learning method with a combination of DDPG and HER Algorithm’s using sparse rewards, proposed and implemented in this paper, uses rewards and penalty system to enable the system to learn directly from the surrounding environment and prevent any miscalculations. This process directly generates the joint angles and positions based on the intelligence attained through several training epochs and the current state is calculated based on the Q-table parameters which are computed using two different target Neural Networks known as Actor and Critic Networks.