The
Basics of Logistic Regression
The supervised learning
classification method logistic regression is used to predict the likelihood of
a target variable. Because the nature of the goal or dependent variable is
dichotomous, there are only two classifications. In basic terms, the dependent
variable is binary in nature, with data recorded as 1 (representing
success/yes) or 0 (representing failure/no). A logistic regression model
predicts P(Y=1) as a function of X mathematically. It's one of the most basic
machine learning algorithms, and it may be used to solve a variety of
classification issues including spam detection, diabetes prediction, cancer
diagnosis, and so on.
Logistic
Regression Types
In general, logistic
regression refers to binary logistic regression with binary target variables, but
it may also predict two additional types of target variables. Logistic
regression may be classified into two forms based on the number of categories:
binary and binomial.
In this form of
categorization, a dependent variable can only be one of two types: 1 or 0.
These variables might, for example, indicate success or failure, yes or no,
victory or loss, and so on.
Multinomial
The dependent variable might
have three or more potential unordered kinds or types with no quantitative
significance in this form of categorization. These variables may, for example,
represent "Type A," "Type B," or "Type C."
Ordinal
In this form of
categorization, the dependent variable can have three or more potential ordered
categories or quantitatively significant types. For example, these variables
may indicate “poor” or “good,” “very good,” or “excellent,” with scores ranging
from 0 to 3.
Assumptions
for Logistic Regression
v Before
we go into the implementation of logistic regression, we must first understand
the following assumptions.
v The
target variables in binary logistic regression must always be binary, and the
intended outcome is indicated by factor level 1.
v The
model should be free of multi-collinearity, which implies the independent
variables must be independent of one another.
v In
our model, we must add relevant variables.
v For
logistic regression, we need use a big sample size.
Models of Regression
Model of Binary Logistic
Regression Binary or binomial logistic regression is the most basic form of
logistic regression, in which the objective or dependent variable can only have
one of two types: 1 or 0.
Model
of Binary Logistic Regression
v Binary
or binomial logistic regression is the most basic form of logistic regression,
in which the objective or dependent variable can only have one of two types: 1
or 0. It allows us to describe a link between a binary/binomial target variable
and numerous predictor factors. In logistic regression, the linear function is
essentially utilised as an input to another function, such as g in the equation
h(x)=g(Tx), where 0h1 is a constant.
v g is
the logistic or sigmoid function, which can be written as g(z)=11+ezwherez=Tx.
v The
following graph can be used to depict a sigmoid curve. We can see that the
y-axis values are between 0 and 1.
What
is Logistic Regression and How Does It Work?
The linear regression model
and the logistic regression equation are quite similar.
Consider a model with one
predictor "x" and one Bernoulli response variable "," with
p denoting the probability of p=1. p = b0+b1x = p = b0+b1x = p = b0+b1x = p =
b0+b1x — 1
The right-hand side of the
equation (b0+b1x) is a linear equation that can hold values that are beyond the
range of the equation (0,1). We do know, however, that probability will always
be in the range of (0,1).
To get around this, we
forecast odds rather than probability.
Odds are the ratio of the
likelihood of an event occurring to the likelihood of it not occurring.
p/ (odds) (1-p)
It is possible to rewrite
equation 1 as follows:
eqn 2 = p/(1-p) = b0+b1x
We anticipate the logarithm
of odds to deal with negative values because odds can only be a positive value.
ln(p/(1-p)) log of chances
It's possible to rewrite
equation 2 as follows:
eqn 3 -------->
ln(p/(1-p)) = b0+b1x
We apply exponential on both
sides to obtain p from equation 3.
exp (b0+b1x) exp(ln(p/(1-p)) exp(ln(p/(1-p)) exp(ln(p/(1-p)) exp(ln(pe(b0+b1x) = eln(p/(1-p)).
Using the logarithm inverse
rule,
e(b0+b1x)=p/(1-p)
Algebraic operations that
are straightforward
e(b0+b1x) = (1-p) * p
p = e(b0+b1x)- p * e(b0+b1x)
= e(b0+b1x) = e(b0+b1x) = e(b0+b1x) =
On the right-hand side,
assuming p is common,
((e(b0+b1x))/p - e(b0+b1x))
p = p * ((e(b0+b1x)) p = p * ((e(b0+b1x)) p = p * ((e(
p = (1 + e(b0+b1x)/e(b0+b1x)
Similarly, for a logistic
model with ‘n' predictors, the equation is as follows:
p =
e-(b0+b1x1+b2x2+b3x3+—-+bnxn)
Notes at the End: Thank you for sticking with me all the way to the end. By the conclusion of this post, we should have a good understanding of how Logistic regression works and how to implement it in Python using the Scikit-learn package.
I hope you liked reading this essay, and please feel free to forward it on to your classmates.
0 Comments