Skip to main content

Command Palette

Search for a command to run...

Natural Language Processing

Updated
4 min readView as Markdown
Natural Language Processing

Debate Steps

Step 1. Learn and understand the topic

Ingest several billion passages from newspapers, books, and journals. (AI researchers call this collection of learned material a corpus.) Then, structure all that content so you can relate concepts to each other and evaluate them, even when they’re stated in different ways.

Step 2. Build a position

Create an opening speech made of short pieces of text pasted together from the corpus. Your speech should detail your position on the debate topic. It can’t be a jumble of phrases. It must present a compelling argument, in logical order, using good grammar.

Step 3. Organize your proof

Learn the deeper meaning of the facts that surround your topic. Decide what evidence is strongest and arrange your proof by themes. Adjust your arrangement each time new evidence arrives. This will help you find updated or completely new information that can score points against your opponent’s position.

Step 4. Respond to your opponent

Listen to your opponent’s arguments and rebuttals, then deliver a convincing rebuttal that refutes your opponent and further proves your case.

*To deal with the “messiness” of unstructured information, computers begin with one sentence at a time. This is called sentence segmentation. Computers then break the information into small chunks of information, called tokens, that can be individually classified. Once the tokens in text have been sorted into a structure based on what they mean, NLP can work with them.

The following activities show you how Groucho Marx’s joke can be tokenized into useful categories called entities and relationships. You’ll learn the meanings of these words as you continue.

*Entity

Relationship

Concept

Emotion detection identifies distinct human emotion types.

Sentiment analysis is a means of assessing if data is positive, negative, or neutral.

*Here are more applications that you might not have thought of:

  • Spotting a dangerous but difficult-to-detect flaw in an airplane’s wing

  • Monitoring water flow across a dairy farm to ensure it doesn’t reach nearby food crops

  • Counting the number of people in an unruly crowd

  • Classifying animal and plant populations to measure biodiversity in a forest

  • Performing lip-reading for people who cannot hear or speak

Although you might think these applications of computer vision might not affect you, they have the potential to benefit you, guide you, or even save your life.

Key points to remember

  • Machines require systems called natural language processing (NLP) to understand human language. Human language is unstructured. In NLP, machines segment sentences into small chunks of information, called a token. Machines classify and sort tokens into a structure so NLP can work with them to extract meaning.

  • With IBM Project Debater, the goal was to build an AI system that could help people make evidence-based, bias-free decisions on difficult topics where the answers aren’t obvious.

  • The four steps a debater AI system takes include:

    • Step 1: Learn and understand the topic

    • Step 2: Build a position

    • Step 3: Organize your proof

    • Step 4: Respond to your opponent

  • Emotion detection identifies distinct human emotion types. AI can be trained to classify emotions.

  • Sentiment analysis is a measure of the strength of an emotion. It results in assessing if data is positive, negative, or neutral.

  • Chatbots are ready to answer your questions!

    • The frontend interacts with the person asking questions. It listens (or reads) and speaks (or presents text).

    • The backend operates application logic and has enough memory to remember earlier parts of a conversation as dialog continues.

  • A chatbot identifies entities and intents, then uses what it has found to trigger a dialog.

    • An intent is a purpose, or the reason why a user is contacting the chatbot. Think of it as a verb or action to take.

    • An entity is a person, place, or thing. Think of it as a noun.

    • A dialog is a flowchart that illustrates the chatbot replies to the user intents.

  • With a convolutional neural network (CNN), an AI system can analyze images. With a generative adversarial network (GAN), an AI system can create new drawings and photos.

  • NLP and computer vision can be useful ways to extend human expertise.

1 views