AI Language Processing Question:

Answer #1
It involves identifying and analyzing the structure of words. Lexicon of a language means the collection of words and phrases in a language. Lexical analysis is dividing the whole chunk of txt into paragraphs, sentences, and words.

Answer #2
Lexical Analysis is defined as the process of understanding the language at the lexicon level. ie. understanding the lexicons - words, tokens, ngrams etc. This involves dictionaries, lexicon tables and vocabulary build using domain knowledge