AI for Beginners: Basic Machine-Learning Terminology

This article was originally written in Chinese and translated into English by AI.

Introduction

By using the Iris dataset to predict flower species, this lesson introduces the terminology and basic concepts of machine learning and provides an intuitive understanding of how machine learning is applied.

Main Content

Xiaobai is a recent graduate who knows very little about AI. As the saying goes, ignorance is bliss: despite his limited knowledge, he wants to work for an AI company.

After sending out many résumés, Xiaobai finally hears back from a data-mining company. Before interviewing him, the company asks him to complete a simple introductory task:

Use a machine-learning algorithm to analyze the Iris dataset. Choose your own angle and write an analysis report.

Xiaobai is completely baffled. What is Iris? How do you analyze data? How do you use a machine-learning algorithm?

As a newcomer, Xiaobai has only read the introductory article What Is AI?. He knows what machine learning is, but nothing beyond that.

So Xiaobai begins searching online...

Background

The Iris dataset, also known as the Iris flower dataset, was collected and organized by the biologist Ronald Fisher.

In 1936, Fisher recorded four attributes—sepal length, sepal width, petal length, and petal width—for three iris species: Iris setosa, Iris virginica, and Iris versicolor. He recorded 50 plants of each species, producing 150 samples in total. The first five records are shown below:

Sepal length	Sepal width	Petal length	Petal width	Species
6.4	2.8	5.6	2.2	2
5.0	2.3	3.3	1.0	1
4.9	2.5	4.5	1.7	2
4.9	3.1	1.5	0.1	0
5.7	3.8	1.7	0.3	0

Terminology

Each record is a sample (or instance). The first record—sepal length = 6.4, sepal width = 2.8, petal length = 5.6, petal width = 2.2, and species = 2—is one sample.

Together, these samples form the dataset used in our study.

We want to find a method that can predict the flower’s species when only the lengths and widths of its sepal and petals are known. We call this method a model or a hypothesis. The process of finding such a method from samples is called learning or training.

In this model, sepal length, sepal width, petal length, and petal width are like the independent variable x in a function. We call them attributes or features. The species we want to predict is like the dependent variable y, and we call it the label.

Here, the label is the flower species, and there are only three possibilities: species 0, 1, or 2. This is therefore a classification problem. If instead we used the species, sepal dimensions, and petal length to predict petal width, the width would be a continuous value with infinitely many possibilities. That would be a regression problem.

Whether the task is classification or regression, if every sample has both features and a label while we learn the model, the process is called supervised learning. If the samples have features but no labels—for example, if the dataset consists of the complete works of Lu Xun, where the text provides features but no label has been defined—and we want to discover latent patterns such as Lu Xun’s writing style, the process is called unsupervised learning.

The Machine-Learning Workflow

Suppose we have a task that we want to implement with machine learning, such as:

Predicting an iris species from the length and width of its sepals and petals.

What steps do we need to follow?

1. Collect and organize data
2. Preprocess the data (feature engineering)
3. Train the algorithm
4. Test the algorithm
5. Apply the algorithm

The first step is naturally to collect data, whether by scraping it from the internet, conducting an offline survey, or using another method.

Next, we analyze the data: check samples for anomalous values or missing values, examine the distribution of each attribute, and compare the values of different attributes. We then process the data: fill in or discard some samples, and split, combine, or discard certain attributes. We refer to this entire step as feature engineering.

To develop an intuitive understanding of the data, for example, we can select sepal length, sepal width, and petal width and plot them. Each point represents one sample, and red, blue, and yellow represent the three flower species. We can see clear differences among the species across these attributes.

Once we have selected the data, we can begin training the algorithm. We write a suitable machine-learning algorithm—later lessons will gradually explain the details—and train it. Recall from the previous lesson, What Is AI?, that machine learning uses data to learn an algorithm. Training the algorithm is the italicized part of that sentence.

Training alone is not enough. We must also see how the model performs in practice. Perhaps it has merely memorized the training data: it answers precisely for data it has seen, but gives random answers for anything unfamiliar.

Before training the model, we therefore set aside some of the original samples so that the machine-learning algorithm cannot study them. After it has learned from the remaining data, we ask it to predict the answers for the unseen samples. This tells us how well the machine-learning algorithm actually performs. The process is called testing the algorithm.

Finally, if the algorithm performs well, we can put it into practice—that is, apply the algorithm.

Code Example

Note: this example is only intended to provide an intuitive picture of the machine-learning workflow. Later articles will explain what every line means.

A more detailed version of the code, including plots, has been uploaded to Kaggle. After registering and signing in, click the “Fork” button to run the code online.

Packages that will appear frequently later:

• NumPy: a scientific-computing package that performs matrix calculations efficiently
• Matplotlib: a plotting toolkit
• scikit-learn: a machine-learning package containing many algorithms
• pandas: a data-analysis package
• TensorFlow: an AI framework
• Keras: a package for rapidly building neural networks

Machine learning and AI have many framework packages, including scikit-learn and TensorFlow. They can produce desired results without requiring you to understand the underlying algorithms. But if you want to study AI in depth, knowing how to call framework packages is nowhere near enough. You need to understand the mathematical principles and be able to implement the algorithms yourself.

Import the required packages and data:

import numpy as np
from sklearn import datasets
from sklearn.linear_model import LogisticRegression
iris=datasets.load_iris()

X = iris.data
y = iris.target

Split the data into training and test sets:

X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.3,random_state=0)

Call an existing algorithm from scikit-learn directly:

# Create the model
model = LogisticRegression()

# Fit the model
model.fit(X,y)

# Evaluate the model
model.score(X_test,y_test)

# Inspect its predictions
model.predict(X_test)

Does This Have Nothing to Do with AI?

“What is this? None of it is cool, and it has nothing to do with artificial intelligence! I wanted to study AI so I could build a robot that holds a conversation, or at least implement image recognition. This machine-learning stuff is just about studying data!” Xiaobai complains.

It is true that image and speech recognition today use deep neural networks, commonly known as deep learning. The classical machine-learning algorithms we study when beginning AI lean more toward data mining, such as predicting users’ purchasing behavior from big data. This is indeed far removed from the “intelligence” we may hope to create.

But if you read image-recognition papers published before 2012, you will find that most use classical machine-learning algorithms such as SVMs, logistic regression, and naive Bayes. Deep learning was once an obscure corner of machine learning. Only with the rapid growth of data and computing power did deep learning and its remarkable results return to view. In fields such as image and speech recognition, it gradually replaced other machine-learning algorithms.

Could we simply study “deep learning” and ignore “classical machine-learning algorithms”?

That would be like trying to run before learning to walk. The deep learning used for image and speech recognition originated in machine learning, and many of its ideas, concepts, and theories come from machine learning. Trying to skip machine-learning theory and jump directly into building AI would be like constructing a skyscraper without a solid foundation: it would inevitably collapse.

Anyone who wants to accomplish something in AI must therefore be patient and master the theoretical foundations one step at a time.

Over the next several lessons, we will use a small practical example to introduce each classical machine-learning algorithm in turn.

Summary and Further Reading

• Learned basic machine-learning terminology
  • Sample, attribute (feature), label, and dataset
  • Model (hypothesis) and training (learning)
  • Supervised and unsupervised learning
  • Classification and regression problems
• Learned the steps for applying machine learning
  1. Collect and organize data
  2. Preprocess the data (feature engineering)
  3. Train the algorithm
  4. Test the algorithm
  5. Apply the algorithm

Recommended Reading

• Basic machine-learning concepts: Chapter 1 of Hang Li’s Statistical Learning Methods
• Basic machine-learning concepts: Sections 1.2 and 1.3 of Zhihua Zhou’s Machine Learning

Recommended Video

• Google Machine Learning Tutorial: Seven Steps of Machine Learning

Author: M小白

To be continued...