In recent times, artificial intelligence (AI) and machine learning (ML) have become hot topics among developers eager to incorporate these technologies into their projects. Although I’m not an expert in machine learning, my experience includes dabbling with it to improve the functionality of certain web projects.
Today, we’re going to have a look at a small segment of machine learning, focusing on Natural Language Processing (NLP). While my examples will be elementary, NLP can bring a wave of innovation to your projects, allowing for:
- Better search functions and result accuracy
- Improved chatbot conversations by understanding user interactions
- Text-to-speech features akin to Amazon Polly and other services
- Enhanced content creation tools, including grammar and style checks
Understanding NLP
According to Wikipedia, Natural Language Processing involves “the application of computational techniques to the analysis and synthesis of natural language and speech.” In simpler terms, NLP interprets and generates human language in a way that computers can understand.
Have you ever mistyped a search query on Google, only to be gently corrected? That’s NLP in action, parsing and making sense of our input despite errors.
NLP typically handles strings of text. Using an NLP library, developers like us can extract valuable information to perform various tasks. In this tutorial, we’ll be using “Natural,” a Node.js library designed for this purpose.
Let’s illustrate some of the handy methods in NLP that you can integrate into your projects swiftly.
An interesting example is the article title “The Secret Designer: First Job Horror” from Web Designer Depot:
I want to not just analyze the phrase but also isolate each word for detailed data extraction. This is where tokenization comes into play:
Tokenization
const nlp = require('natural');
const tokenizer = new nlp.WordTokenizer();
console.log(tokenizer.tokenize("The Secret Designer: First Job Horror"));
Our output is a straightforward JavaScript array:
[ 'The', 'Secret', 'Designer', 'First', 'Job', 'Horror' ]
The WordTokenizer breaks down the sentence into individual words, which can then be further processed. The “Natural” library offers several tokenizers for different purposes.
“Natural” encompasses a variety of algorithms contributed by brilliant minds to perform the functionalities we’ll discuss. If you’re curious about the details, the Natural GitHub repository provides all the information you need.
Measuring String Similarity
“Natural” uses the Levenshtein distance algorithm to assess how closely two strings resemble each other:
const nlp = require('natural');
console.log(natural.LevenshteinDistance("Daine","Dane"));
The output “1” indicates a small difference between the two strings. When misspellings occur, such as the insertion of letters, this is a useful method to offer correct spelling suggestions.
Approximate String Matching
This feature is akin to Levenshtein-distance calculations and is ideal for longer strings or names of entities like cities or people that may be misspelled in text.
Phonetic Comparison
When words are phonetically similar but have different spellings, the metaphone.compare() method is quite useful.
const nlp = require('natural');
const metaphone = nlp.Metaphone;
if(metaphone.compare('see', 'sea')) {
console.log('Phonetically they are identical!');
}
Spellchecking
Dynamic spellchecking is a powerful feature for developing word processors or enhancing spellcheck capabilities in your application.
const wordList = ['something', 'soothing']; const spellchecker = new nlp.Spellcheck(wordList);
Testing the spellchecker:
spellchecker.getCorrections('soemthing', 1); // ['something']
Lexical Database
“Natural” now integrates with Wordnet, a database from Princeton University, providing real-time word lookup with associated linguistic information such as verbs, adjectives, and synonyms.
Installing the Wordnet database in your Node.js projects is as simple as executing:
npm install wordnet-db
React developers, among others, could make the most out of this feature, as it aims to standardize dictionary lookups across various platforms.
Real-World Example
To cement our understanding, let’s build a command-line interface (CLI) tool that prompts for a word and returns its dictionary definition. For simplicity, I’ve omitted some detailed coding practices such as error handling, which can be implemented as needed in Node.js.
To begin, create a directory for your application. Navigate into it, and initialize your Node.js project:
npm init -y
The key part of the package.json file is the “main” entry, which should remain index.js. Create this file in the root of your directory.
Add some necessary packages with:
npm install --save commander wordnet-db natural
Commander.js facilitates the writing of CLI tools. We already discussed “Natural” earlier.
Open your index.js and include the following directive at the top:
#!/usr/bin/env node
This makes the file executable. To make your dictionary app callable from the command line, edit the package.json file by adding:
"preferGlobal": true, "bin": "./index.js"
Then, from your project directory, link your app using:
npm link
You can find the complete package.json and index.js in this Gist:
https://gist.github.com/dainemawer/d4dc972fd2c0db5e58615c13c17ca8aa
But let’s elaborate on the setup:
- We begin by importing the required modules such as commander and wordnet.
- The wordNetLookup function is deferred for later discussion.
- The application uses the program object to set up our tool, including the version, description, and command line interface.
- A command is configured to trigger a dictionary lookup using a provided word.
Execute your CLI tool with the command dictionary lookup “word” to perform a real-time lookup of a word from the WordNet database.
This NLP primer aims to introduce you to the possibilities of NLP in Node.js. Next time, we’ll dive into Named Entity Recognition (NER), which allows for identifying and extracting information about real-world entities such as cities and names.
