Logistic Regression

Logistic regression is a linear method. Taking one or multiple input values (x), logistic regression models will provide one output value (y). The output value in logistic regression is a binary value. Due to the binary output, interpreting the results are not within the scope of these linear methods. Rather, the accuracy of the predictions is focused on.

Example (single x input): ⅇ^β₀+β₁x ÷ 1+ⅇ^{(β₀+β₁x)}

Logistic regression models the probability of an outcome. The probability is then transformed into a binary value based on the most likely outcome. This is all done by estimating the values of the coefficients (β_n).

Though the method is linear (as it assumes a linear relationship between the input and output variables), the prediction undergoes a non-linear transform to provide the output as a binary value.

In the following image, the straight line represents a linear solution and the s shape represents a logistic solution. The logistic s-shaped curve provides a more accurate representation of the probability of a datapoint being a 1 or a 0 in the image.

Preparing data for a logistic regression model

• The output must be a binary value. This model is used for binary classification problems.

• Logistic regression assumes no error in the output variable. Removing outliers and any misclassified information from the dataset is beneficial.

• The data should fit a rough Gaussian distribution to best represent the linear relationship between the input and output values. Transforming the dataset to expose this relationship will improve the performance of the model.

• The model can overfit if there are multiple highly correlated inputs variables

• If there are highly correlated input variables, or if the data is sparse, it is possible that the estimation calculations for the values of the coefficients will fail to converge.