Margin boundaries (linear)

The distance between the decision boundary and the closest data points from either class.

The Margin boundary is a line parallel to the Decision boundary (linear), but offset. The distance between the positive and negative margin boundaries is the margin. A Maximum decision boundary is made so that it passes through the support vectors, such that margin is maximised, but the area between two margin boundaries do not contain any data points.

Classes of margin boundaries

Using Vector notation for weight and data

Let $h(x)=x\cdot w+b$ ($b$=bias) If $h(x)=0$ represents the decision boundary, then $h(x)=k,k\ne 0$ represents margin boundaries. Note that $\cdots +b$ and $\cdots -b$ in the formula is often interchangeable, as b can be negative or positive, and is adjusted during training anyways, just need to ensure its consistant.

More specifically

• Class +1 boundary
  • $h(x)=1$
• Class -1 boundary
  • $h(x)=-1$ The margin for this pair of margin boundaries is $m=\frac{2}{||w||}$, where $||w||$ is the magnitude of the weights (distance from the origin)