Getting Started with Natural Language Processing in Python

A significant portion of the data that is generated today is unstructured. Unstructured data includes social media comments, browsing history and customer feedback. Have you found yourself in a situation with a bunch of textual data to analyse, and no idea how to proceed?

The objective of this tutorial is to enable you to analyze textual data in Python through the concepts of Natural Language Processing (NLP). You will first learn how to tokenize your text into smaller chunks, normalize words to their root forms, and then, remove any noise in your documents to prepare them for further analysis.

Let's get started!

Prerequisites

In this tutorial, we will use Python's nltk library to perform all NLP operations on the text. At the time of writing this tutorial, we used version 3.4 of nltk. To install the library, you can use the pip command on the terminal:

pip install nltk==3.4

To check which version of nltk you have in the system, you can import the library into the Python interpreter and check the version:

import nltk
print(nltk.__version__)

To perform certain actions within nltk in this tutorial, you may have to download specific resources. We will describe each resource as and when required.

However, if you would like to avoid downloading individual resources later in the tutorial and grab them now in one go, run the following command:

python -m nltk.downloader all

Step 1: Convert into Tokens

A computer system can not find meaning in natural language by itself. The first step in processing natural language is to convert the original text into tokens. A token is a combination of continuous characters, with some meaning. It is up to you to decide how to break a sentence into tokens. For instance, an easy method is to split a sentence by whitespace to break it into individual words.

The post Getting Started with Natural Language Processing in Python appeared first on SitePoint.

Comments

Post a Comment

Popular posts from this blog

Visual Studio Code: A Power User’s Guide

Image
In this guide, you’ll learn how to take advantage of Visual Studio Code to supercharge your development workflow. This article is written for beginners who may be using Visual Studio Code for the first time. VS Code, as it’s commonly known, is considered a "lightweight" code editor. In comparison with full integrated development environment (IDE) editors which occupy gigabytes of disk space, VS Code only uses less than 200MB when installed. Despite the "lightweight" term, VS Code offers a massive number of features which keep increasing and improving with every new update. For this guide, we'll cover the most popularly used features. Every programmer has their own tool set which they keep updating whenever new workflows are discovered. If you want to learn every tool and feature VS Code has to offer, check out their official documentation . In addition, you may want to keep track of updates for new and improved features. Prerequisites In order to follow alo
Read more

6+ Best Websites for Free Fonts in 2020

Image
I firmly believe that, in order to strike the optimal balance between minimalism and visual appeal, a user interface should display no more than what’s absolutely necessary while making those necessary elements look visually appealing. Keep it simple, but make the simple look amazing. But, what does simple look like exactly? Well, strip away all of the over-the-top marketing gimmicks, redundant features, and cluttering visuals, and most of what’s left is the UI copy and text content. As boring as this sounds, let’s be honest: this is the most important aspect of any app or website. Information and the ability to navigate it is the entire reason users are using our app or website. So with the right typeface coupled with the right font styles (see “ Font vs typeface: the ultimate guide ”), we can create designs that are quite simple and yet visually appealing at the same time. This emphasizes the most important aspects of the design while usin
Read more

How to Contribute to Open Source TypeScript Projects

Image
If you think of the ideas of open source applied to information in an encyclopedia, you get to Wikipedia – lots and lots of small contributions that bubble up to something that’s meaningful. – Matt Mullenweg One of my favorite aspects of open source is the fact that anyone can contribute! It’s like “productive volunteering” (if you’ll humor me). It’s a fantastic way to build something bigger than you, give back to the community, and level up your skills. In this article, we’re going to talk about practical ways for you to get involved in open source TypeScript projects. We’ll first cover assessing your level then jump into how you can find opportunities. Assessing Your Level Before you get started contributing to open source, you want to find your comfort level. Where are you at skill-wise? What level of complexity are you wanting to solve? How much time do you want to spend? These are important questions to consider before diving in. This
Read more