Terminology

From Machine Learning
Jump to: navigation, search

Refer to this section for any machine learning terms that may be confusing. There is a lot of very similar terms and language used in the field. The distinctions made in this section such as parameter vs hyperparameter are not terms used by all computer scientists but do provide a clearer way to refer to different levels in a machine learning model or process.

The "more information" hyperlinks link to other pages in this wiki, expanding on their respective sections.

Parameters and Hyperparameters

Parameters are defined by the data being used. Hyperparameters are manually tuned and apply to the model rather than the data [1]. Overall, the goal of defining parameters and hyperparameters is to optimise the accuracy of the prediction. Their values determine the accuracy and efficiency of the model. More information.

Features and Labels

Features and labels are terms that come up a lot in machine learning [2].

A feature is the information we have. It is a column in the data input – an individual measurable property of the object or phenomenon being analysed. When preparing data, it is important to filter out unnecessary features and select the relevant features so as not to slow down the training algorithm with redundant information.

A label refers to the information we want to predict, or the data we want to obtain. Labels use features to make predictions. When training a model, labelled data is provided, which indicates which feature or features correspond to a particular label. When the model is provided with new unlabelled data, it will assign the features (input) of the new data to labels as it has previously learnt.

For example, consider a model that predicts whether a person is male or female. The features (input information) might be properties such as age, weight and height. The labels are ‘male’ and ‘female’. The training dataset will be labelled.

This only applies to supervised learning. Unsupervised learning does not use labels like this – rather, the goal is for the model to find a correlation between age, weight and height, and may output two groups ‘A’ and ‘B’ that could correspond to the supervised method’s labels of ‘male’ and ‘female’, but may also find unexpected correlations in the data and form groups ‘A’, ‘B’ and ‘C’ which the operator will need to look at and define.

Note: the above (supervised) example is a classification problem as the output is a binary classification of either male or female.

Models and Algorithms

Model and algorithm are terms that, in machine learning, should not be used interchangeably (unless you enjoy confusion). More information.

Training, Validation and Testing datasets

To train and assess your machine learning model, the data you have should be split into two or three sets (the validation set is not always necessary, particularly for simpler classification models). It is part of the data cleaning and preparation that should take place before implementing a machine learning algorithm, and understanding how to split and use them is important. More information


Confusion (or Error) Matrices

These matrices are a way to visualise and quantify the performance of a machine learning model. Their main usage is from the statistical analysis of the results. The results take the form of number of True negatives and positives, and False negatives and positives. More information


Information Theory

Relevant to the mathematical and statistical aspects of understanding machine learning, information theory (particularly entropy and information gain) can be used to assess a machine learning model and understand its performance. Information gain calculations are particularly useful when wanting to find the significance of attributes in your data and are often part of refining a machine learning model. More information

References