Parameters and Hyperparameters

From Machine Learning
Jump to: navigation, search

Parameters

A parameter is internal to the model; they are required by the model when making predictions. Consider them the unknown variables of the equation.

Model parameters are often estimated using an optimisation algorithm and are learned from the data.

Parameter values are often saved as part of the learned model. Another model run on the same dataset may find and use different values.

A parametric model has a fixed number of parameters. A nonparametric model has a variable number of parameters – this includes models that for example can grow with an increasing dataset.


Hyperparameters

A hyperparameter is external to the model. They are often used in processes to help estimate model parameters.

They are often specified by the practitioner and can often be set using heuristics. Hyperparameters are often tuned for a given predictive modelling problem.

The value of a hyperparameter cannot be estimated from the data and is specified manually. Examples of hyperparameters include the learning rate for training a neural network or the k in k-nearest neighbours.