Support Vector Machines (SVMs): Definitions & Applications

Instructor: Euan Russano

Euan has a Phd degree in Engineering and offers private training and tutoring in Programming and Engineering.

In this lesson you will learn what SVMs are and how they are useful to retrieve patterns and relations between data. You will also learn how to implement an SVM for data classification.

Definition

A Support Vector Machine (SVM), also referred to as a Support Vector Network (SVN) consists of a supervised model that can detect patterns and information in data for classification and regression. Its basic structure consists of mapping the data locations and then constructing margins between the categories obtained. These margins, or hyperplanes are placed as far apart as possible. The following figure illustrates a binary classification problem using SVM.

Figure 1 - SVM as a binary classifier model.
svm-binary

Some applications of SVMs are text categorization, image classification, handwriting recognition or machine learning problems in general.

History

The concept behind SVM first appeared in problems of classification. In 1963, Vapnik and Lerner introduced the Generalized Portrait algorithm, an initial approach for the development of SVM. This occurred just after Frank Rosenblatt developed a simple linear classifier called Perceptron which later came to be one building block of simple neural networks. It is important to notice at this point, that the Generalized Portrait algorithm formed a foundation for the more general, nonlinear algorithm currently used in SVM models.

Since SVM is, by nature, a linear model, it was only used for classification of linearly separable data. However, in 1992, Vapnik and his colleagues noted that, by using the Kernel Trick, the model could be used for the first time in non-linearly separated data. That was a huge development in the field.

SVM modelling framework became even more robust when, in 1995, Cortes and Vapnik introduced the Soft Margin Classifier, allowing SVM to accept some mis-classifications. This is usually regulated by a single hyperparameter (C), which dictates how 'soft' the model is.

In summary, we mentioned here four different SVM models used for classification, since its initial development:

  1. The Maximal Margin Classifier
  2. SVM kernelized (with Kernel Trick) for nonlinearly separable data
  3. The Soft Margin Classifier
  4. The Soft Margin Classifier with Kernel Trick

The application of the SVM model in regression took a little bit more time. An initial milestone was established in 1996, when Vapnik and colleagues proposed a modified version of SVM for regression. The name was also modified to reflect such changes, being called Support Vector Regression (SVR). This also incorporates the parameter C(soft margin) and the kernel trick to deal with non-linear regression.

Applications in Classification Problems

As already mentioned, SVM can be used in different areas of machine learning. Expanding more, some areas that find SVM application are:

  • Face Detection: Use of model to recognize face and non-face images and generate a boundary around the face (usually square).
  • Text Categorization: SVM can be used to classify texts in different categories, based on a score and compared with a threshold value. In the related area, it can be used for hypertext categorization.
  • Bioinformatics: Protein, disease and gene classification, as well as patients classification according genes and other biological features.
  • Generalized Predictive Control (GPC): SVM has found applications on the control of chaotic dynamics as the inner model of a GPC superior structure.
  • Geo and Environmental Sciences: Classification of material, weather, etc.

To unlock this lesson you must be a Study.com Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use Study.com

Become a Study.com member and start learning now.
Become a Member  Back
What teachers are saying about Study.com
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? Study.com has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account
Support