Regression - Linear Regression, Logistic Regression.

Regression in Predictive Data Analytics

Linear RegressionLinear Regression1. Positive Regression 1. Negative Regression * [ ] Update mind map as needed

Definition: Linear regression is a statistical method that models the relationship between a dependent variable (target) and one or more independent variables (features) using a linear equation. The goal is to find the linear equation that best predicts the dependent variable from the independent variables.

Mathematical Representation: The linear regression model can be represented as: $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_nx_n + \epsilon $$

where:

( y ) is the dependent variable.
$$( x_1, x_2, \ldots, x_n )$$ are the independent variables.
$$( \beta_0 )$$ is the intercept.
$$( \beta_1, \beta_2, \ldots, \beta_n )$$ are the coefficients.
$$( \epsilon )$$ is the error term.

Implementation in Python using Scikit-learn: Here's how you can implement linear regression using Scikit-learn:

Import necessary libraries:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

Load the dataset:
```
data = pd.read_csv('data.csv')
```

Preprocess the data:

X = data[['feature1', 'feature2', 'feature3']]  # Independent variables
y = data['target']  # Dependent variable

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Train the model:

model = LinearRegression()
model.fit(X_train, y_train)

Make predictions and evaluate the model:

y_pred = model.predict(X_test)

# Calculate mean squared error and R-squared value
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print(f'Mean Squared Error: {mse}')
print(f'R-squared: {r2}')

Interpretation:

Coefficients ((\beta)): These indicate the change in the dependent variable for a unit change in the independent variable.
Intercept ((\beta_0)): This is the expected value of the dependent variable when all independent variables are zero.
R-squared: This measures the proportion of the variance in the dependent variable that is predictable from the independent variables.

Logistic RegressionLogistic RegressionRemove the missing value

Definition: Logistic regression is a classification algorithm used to predict the probability of a binary outcome (0 or 1) based on one or more independent variables. Unlike linear regression, the output of logistic regression is a probability value between 0 and 1.

Mathematical Representation: The logistic regression model uses the logistic function (also known as the sigmoid function) to map predicted values to probabilities: $$P(y=1|x) = \frac{1}{1 + e^{-(\beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_nx_n)}}$$

Implementation in Python using Scikit-learn: Here's how you can implement logistic regression using Scikit-learn:

Import necessary libraries:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, accuracy_score, precision_score, recall_score, f1_score

Load the dataset:
```
data = pd.read_csv('data.csv')
```

Preprocess the data:

X = data[['feature1', 'feature2', 'feature3']]  # Independent variables
y = data['target']  # Dependent variable (binary outcome)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Train the model:

model = LogisticRegression()
model.fit(X_train, y_train)

Make predictions and evaluate the model:

y_pred = model.predict(X_test)

# Calculate evaluation metrics
accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred)
recall = recall_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)

print(f'Accuracy: {accuracy}')
print(f'Precision: {precision}')
print(f'Recall: {recall}')
print(f'F1 Score: {f1}')

# Confusion matrix
cm = confusion_matrix(y_test, y_pred)
print(f'Confusion Matrix:\n{cm}')

Interpretation:

Coefficients ((\beta)): These indicate the log odds of the dependent variable for a unit change in the independent variable.
Odds Ratio: Exponentiating the coefficients gives the odds ratio, which indicates how the odds of the outcome increase (if >1) or decrease (if <1) with the independent variable.
Confusion Matrix: Shows the counts of true positive, true negative, false positive, and false negative predictions.
Accuracy, Precision, Recall, F1 Score: These metrics evaluate the performance of the classification model.

Conclusion

By understanding and implementing linear and logistic regression, you gain essential skills in predictive analytics. Linear regression helps predict continuous outcomes, while logistic regression is useful for binary classification problems. Both methods are fundamental tools in a data scientist's toolkit and provide a solid foundation for more advanced predictive modeling techniques.

MM - Regression - Linear Regression, Logistic Regression.MM - Regression - Linear Regression, Logistic Regression.Sure! Here's a list of keywords and short sentences to help you create mind maps for Linear Regression and Logistic Regression. Linear Regression Core Concepts Dependent Variable (y)**: The outcome being predicted. Independent Variables (x1, x2, ...)**: Predictors used to estimate y. Linear Equation**: $$( y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \ldots + \beta_nx_n + \epsilon )$$ Intercept ((\beta_0))**: Value of y when all x's are zero. Coefficients ((\beta))**: Effect of each predictor on y