TU MBS First Semester (MSC 514)
Unit 5: Correlation and Regression Analysis
These notes are specially prepared for Tribhuvan University (TU) MBS First Semester students. The content is exam-oriented and covers important concepts, formulas, interpretations, assumptions, and frequently asked examination questions.
Table of Contents
- 1. Correlation Analysis
- 2. Partial Correlation
- 3. Multiple Correlation
- 4. Coefficient of Determination
- 5. Regression Analysis
- 6. Linear Regression
- 7. Non-linear Regression
- 8. Multiple Regression
- 9. Standard Error of Estimate
- 10. Test of Regression Model
- 11. Test of Regression Coefficients
- 12. Autocorrelation
- 13. Multicollinearity
- 14. Residual Analysis
1. Correlation Analysis
Correlation measures the degree and direction of relationship between two or more variables. It indicates whether changes in one variable are associated with changes in another variable.
Examples
- Income and expenditure (Positive correlation)
- Price and demand (Negative correlation)
- Height and intelligence (No significant correlation)
Types of Correlation
- Positive Correlation
- Negative Correlation
- Zero Correlation
- Perfect Positive Correlation
- Perfect Negative Correlation
| Correlation Coefficient (r) | Interpretation |
|---|---|
| +1.00 | Perfect Positive Correlation |
| +0.80 | Strong Positive Correlation |
| +0.50 | Moderate Positive Correlation |
| 0 | No Correlation |
| -0.50 | Moderate Negative Correlation |
| -1.00 | Perfect Negative Correlation |
2. Partial Correlation
Partial correlation measures the relationship between two variables after removing the effect of one or more additional variables.
Example
- X = Sales
- Y = Profit
- Z = Advertisement
Partial correlation measures the relationship between Sales and Profit while controlling the effect of Advertisement.
- Removes the influence of third variables.
- Shows the actual relationship between two variables.
- Widely used in business and social science research.
3. Multiple Correlation
Multiple correlation measures the relationship between one dependent variable and two or more independent variables simultaneously.
Example
- Dependent Variable = Sales
- Independent Variables = Advertisement, Price
A higher value of R indicates a stronger combined relationship between the dependent variable and all independent variables.
4. Coefficient of Determination (R²)
The coefficient of determination measures the proportion of variation in the dependent variable explained by the independent variable(s).
Interpretation
Suppose R² = 0.81
- 81% variation is explained by the regression model.
- 19% variation remains unexplained due to other factors.
5. Regression Analysis
Regression analysis is a statistical technique used to estimate the functional relationship between dependent and independent variables and to predict future values.
Main Objectives
- Measure relationship between variables.
- Predict future values.
- Estimate the effect of independent variables.
- Support business decision-making.
Independent Variable (X) → Predictor variable.
- Correlation
- Regression
- Partial Correlation
- Multiple Correlation
- Coefficient of Determination
6. Linear Regression
Linear regression explains the relationship between a dependent variable and one or more independent variables using a straight-line equation. It is the most commonly used regression model in business, economics, and management research.
Where:
| Symbol | Meaning |
|---|---|
| Y | Dependent Variable |
| X | Independent Variable |
| a | Y-intercept (Constant) |
| b | Slope (Regression Coefficient) |
If b = 5, it means that for every one-unit increase in X, the value of Y increases by 5 units, assuming all other factors remain constant.
7. Non-linear Regression
Non-linear regression is used when the relationship between variables cannot be represented by a straight line. Instead, the data follow a curved pattern.
Common Non-linear Models
| Model | Equation |
|---|---|
| Quadratic | Y = a + bX + cX² |
| Exponential | Y = aebx |
| Logarithmic | Y = a + b ln(X) |
| Power Model | Y = aXb |
8. Multiple Regression
Multiple regression estimates the relationship between one dependent variable and two or more independent variables simultaneously.
Example
Sales = a + b₁(Advertisement) + b₂(Price) + b₃(Customer Income)
Advantages
- Provides more accurate prediction.
- Considers several influencing variables together.
- Widely used in business forecasting.
- Helps in policy and managerial decision-making.
9. Standard Error of Estimate
The Standard Error of Estimate (SEE) measures the average prediction error of the regression model. It indicates how closely the observed values lie around the regression line.
Where
| Symbol | Meaning |
|---|---|
| SSE | Sum of Squared Errors |
| n | Number of observations |
| k | Number of independent variables |
10. Test of Regression Model (F-Test)
The overall significance of a multiple regression model is tested using the F-test.
Hypotheses
| Hypothesis | Meaning |
|---|---|
| H₀ | All regression coefficients are equal to zero. |
| H₁ | At least one regression coefficient is not equal to zero. |
- If Fcalculated > Ftable, reject H₀.
- The regression model is statistically significant.
11. Test of Regression Coefficients (t-Test)
The significance of each individual regression coefficient is tested using the t-test.
Hypotheses
| Hypothesis | Meaning |
|---|---|
| H₀ | β = 0 (No significant effect) |
| H₁ | β ≠ 0 (Significant effect) |
- If |t| > t-table, reject H₀.
- The independent variable significantly affects the dependent variable.
12. Autocorrelation
Autocorrelation occurs when regression errors are correlated with one another. It mainly appears in time-series data.
Effects
- Inefficient regression estimates.
- Incorrect standard errors.
- Misleading hypothesis tests.
Detection
The Durbin-Watson (DW) Test is commonly used.
| DW Value | Interpretation |
|---|---|
| ≈ 2 | No autocorrelation |
| < 2 | Positive autocorrelation |
| > 2 | Negative autocorrelation |
13. Multicollinearity
Multicollinearity occurs when two or more independent variables are highly correlated with each other.
Problems
- Large standard errors.
- Unstable regression coefficients.
- Difficulty in interpreting individual variables.
- Less reliable statistical tests.
Detection
- Variance Inflation Factor (VIF)
- Tolerance Value
- Correlation Matrix
VIF greater than 10 generally indicates serious multicollinearity.
14. Residual Analysis
Residual analysis is performed to verify whether the assumptions of regression analysis are satisfied.
Main Assumptions Checked
| Assumption | Purpose |
|---|---|
| Linearity | Relationship should be linear. |
| Homoscedasticity | Constant variance of residuals. |
| Normality | Residuals should follow normal distribution. |
1. Linearity
The relationship between dependent and independent variables should be linear. This can be checked using scatter plots or residual plots.
2. Homoscedasticity
The variance of residuals should remain constant across all predicted values. If the spread changes, the problem is known as heteroscedasticity.
3. Normality of Errors
Residuals should follow a normal distribution for valid hypothesis testing and confidence interval estimation.
- Linearity
- Independence of Errors
- Homoscedasticity
- Normality of Errors
- No Multicollinearity
- No Autocorrelation (especially in time-series data)
15. Quick Revision Summary
| Topic | Key Point |
|---|---|
| Correlation | Measures the degree and direction of relationship between variables. |
| Partial Correlation | Measures the relationship between two variables after controlling other variable(s). |
| Multiple Correlation | Measures the relationship between one dependent variable and several independent variables. |
| Coefficient of Determination (R²) | Shows the proportion of variation explained by the regression model. |
| Linear Regression | Represents a straight-line relationship between variables. |
| Non-linear Regression | Represents a curved relationship between variables. |
| Multiple Regression | Uses two or more independent variables to predict one dependent variable. |
| Standard Error of Estimate | Measures prediction accuracy of the regression model. |
| F-Test | Tests the overall significance of the regression model. |
| t-Test | Tests the significance of an individual regression coefficient. |
| Autocorrelation | Correlation among regression errors, common in time-series data. |
| Multicollinearity | High correlation among independent variables. |
| Residual Analysis | Checks whether regression assumptions are satisfied. |
Important Formulas for TU Examination
- Simple Regression: Y = a + bX
- Multiple Regression: Y = a + b₁X₁ + b₂X₂ + ... + bₖXₖ
- Coefficient of Determination: R² = r²
- Coefficient of Non-determination: 1 − R²
- Residual: e = Y − Ŷ
- Standard Error of Estimate: SEE = √(SSE / (n − k − 1))
16. Frequently Asked TU Examination Questions
Very Short Questions (2 Marks)
- Define correlation.
- What is partial correlation?
- What is multiple correlation?
- Define regression analysis.
- What is autocorrelation?
- What is multicollinearity?
- What is coefficient of determination?
- What is residual?
- Define homoscedasticity.
- What is the standard error of estimate?
Short Questions (5 Marks)
- Explain linear regression with suitable examples.
- Differentiate linear and non-linear regression.
- Explain coefficient of determination and its interpretation.
- Discuss the importance of multiple regression.
- Explain partial and multiple correlation.
Long Questions (10 Marks)
- Explain multiple regression analysis with assumptions.
- Discuss the test of regression model and regression coefficients.
- Explain residual analysis in regression.
- Discuss multicollinearity and autocorrelation with suitable examples.
- Explain the assumptions of multiple regression analysis.
17. TU Examination Preparation Tips
- Always define the concept before writing its explanation.
- Draw diagrams or regression lines wherever appropriate.
- Write formulas correctly and define every symbol.
- Interpret statistical values such as R², t, and F clearly.
- Memorize all assumptions of regression analysis.
- Practice numerical problems using calculators.
- Use headings, bullet points, and examples to improve presentation.
Frequently Asked Questions (FAQ)
1. What is the difference between correlation and regression?
Correlation measures the strength and direction of a relationship, whereas regression estimates the relationship and predicts the value of a dependent variable.
2. What does R² represent?
R² represents the percentage of variation in the dependent variable explained by the independent variable(s).
3. Why is multiple regression important?
It improves prediction accuracy by considering the combined effect of multiple independent variables.
4. What is multicollinearity?
Multicollinearity occurs when independent variables are highly correlated, making regression coefficients unstable.
5. What is homoscedasticity?
Homoscedasticity means that the variance of regression residuals remains constant across all predicted values.
Conclusion
Correlation and Regression Analysis is one of the most important units in TU MBS Statistical Methods (MSC 514). Understanding concepts such as partial correlation, multiple correlation, coefficient of determination, regression models, hypothesis testing, multicollinearity, autocorrelation, and residual analysis is essential for both theoretical and numerical examinations. Mastering these topics will significantly improve performance in university examinations as well as in research and business data analysis.
Prepared for: TU MBS First Semester (MSC 514)
Statistical Methods – Unit 5: Correlation and Regression Analysis