Logistic Regression

Logistic Regression is a statistical method for analyzing a dataset in which there are one or more independent variables that determine an outcome. It is used to model relationships between a dependent variable and one or more independent variables by fitting a logistic curve to the data. Unlike linear regression, which is used to model a continuous outcome, logistic regression is used to model a binary outcome, i.e., a situation in which the outcome can take one of two possible values, such as yes/no or true/false. In logistic regression, a logistic function is used to model the probability that the dependent variable takes a certain value. The logistic function maps any real-valued number to a value between 0 and 1, which can be interpreted as the probability that the outcome is a certain value. Logistic regression is widely used in many fields, including biology, finance, and social sciences, for binary classification problems and can be extended to multi-class classification problems as well.

Mathematical Form Of  Logistic Regression :

Logit(p_i) = 1/(1+ exp^(-p_i))

ln(p_i/(1 - p_i)) = b_0 + b_1*X_1 + … + b_n*X_n

p_i is the predicted probability of the outcome taking a certain value b_0, b_1, b_2, ..., b_n are the coefficients of the logistic regression model.

The goal of logistic regression is to estimate the coefficients $b_0, b_1, b_2, b_n such that the logistic curve fits the data as well as possible. This is done using maximum likelihood estimation, which involves finding the coefficients that maximize the likelihood of the observed data given the model. Once the coefficients are estimated, the logistic regression model can be used to predict the probability of the outcome taking a certain value for any new values of the independent variables. This prediction can then be transformed into a binary decision (e.g., yes/no) by choosing a threshold value for the predicted probability.

Real Life Example of Logistic Regression :

1. Credit risk analysis :

Banks and financial institutions use logistic regression to assess the risk of a loan default based on factors such as the borrower's credit history, income, employment status, and loan amount.

2. Medical diagnosis :

Logistic regression can be used in medicine to diagnose a disease based on patient symptoms and test results.

3. Customer churn prediction :

Companies can use logistic regression to predict which customers are likely to leave and take their business elsewhere based on their purchasing history, demographics, and other factors.

4. Employee turnover prediction :

Employers can use logistic regression to predict which employees are likely to leave the company based on factors such as job satisfaction, salary, and years of service.

5. Election forecasting :

Political pollsters use logistic regression to predict the outcome of elections based on factors such as voter demographics, previous election results, and current political climate.



Coefficients Estimate :

Here we coefficient estimate Using Python

import pandas as pd

from sklearn.linear_model import LogisticRegression

from sklearn.model_selection import train_test_split



# Load the dataset

df = pd.read_csv("data.csv")



# Split the data into independent variables (X) and the dependent variable (y)

X = df.drop("purchased", axis=1)

y = df["purchased"]



# Split the data into training and test sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)



# Create a logistic regression model

model = LogisticRegression()



# Fit the model to the training data

model.fit(X_train, y_train)



# Make predictions on the test data

y_pred = model.predict(X_test)



# Print the accuracy score

print("Accuracy:", model.score(X_test, y_test))