- Tags:
- Author:
- Link:
- Source date:
- Finished date:: 2021-11-16
Two types of regression algorithms: ones can extrapolate, the others canĀ“t
There are generally two ways a regression algorithm can make predictions: either by transforming theĀ featuresĀ or by transforming theĀ target.
**Feature-transforming algorithms** learn some mathematical function that takes features as an input and then combines and transforms them to produce an output that matches the target values in the training set. Linear regression and neural nets are of this kind. **They can extrapolate values.**
Target-transforming algorithms use the features to group the target values in the training set and make predictions by averaging values in a group; a set of feature just indicates which group to average. Decision trees and nearest neighbors are of this kind. They cannot extrapolate values.