Estudo semântico de palavras fora do vocabulário utilizando redes neurais recorrentes

Abstract

The process of recognizing computer text writing by means of Natural Language Processing (NLP) goes through some challenges when there are words that have not yet been categorized, which are called Out-of-Vocabulary (OOV). These are commonly content that make a representation, such as local slang or typing error. These types of content have grown exponentially as the Internet has become more popular, causing people to interact more assiduously through text. This paper presents six Neural Network (NN) based models for the treatment of these unknown words in the Portuguese language, which are Simple Recurrent Neural Networks (RNN), bidirectional RNN (BIRNN), Long Short-Term Memory (LSTM), bidirectional LSTM (BILSTM), Gated Recurrent Units (GRU) and bidirectional GRU (BIGRU). The models were trained using three different bases, but both in Portuguese. After training, a function was made that was able to categorize the OOVs, creating valid vectors. In addition, their meaning was also verified using cosine similarity and part-of-speech tagging. With all the tests, it was possible to obtain an accuracy of 99.99% with one of the bases using the GRU model