Correlation and Regression R23 INFORMATION TECHNOLOGY PROBABILITY AND STATISTICS

 

Correlation and Regression:

1) Correlation

Correlation is a statistical concept that measures the degree and direction of relationship between two variables.

• If both variables increase together → Positive correlation

• If one increases while the other decreases → Negative correlation

• If there is no consistent pattern → No correlation

Example:

• Height and weight → usually positive correlation

• Price and demand → usually negative correlation

Correlation shows association, not causation.

---

2) Correlation Coefficient

A correlation coefficient is a numerical measure of correlation. It tells us:

• Strength of relationship

• Direction of relationship

The most common is Karl Pearson’s coefficient of correlation (r).

🔹 Karl Pearson’s Correlation Coefficient

$$r = \frac{Cov(X,Y)}{\sigma_X \sigma_Y}$$

Where:

• $Cov(X,Y)$ = Covariance between X and Y

• $\sigma_X$, $\sigma_Y$ = Standard deviations

🔹 Properties of r

• Range: –1 ≤ r ≤ +1

• $r = +1$ → Perfect positive correlation

• $r = -1$ → Perfect negative correlation

• $r = 0$ → No linear correlation

🔹 Interpretation

Value of r	Interpretation
0 to ±0.25	Very weak
±0.25 to ±0.50	Weak
±0.50 to ±0.75	Moderate
±0.75 to ±1	Strong

---

3) Rank Correlation

Rank correlation measures the relationship between variables based on their ranks, not actual values.

It is used when:

• Data is ordinal

• Exact values are not known

• Data is qualitative (like preferences, grades)

🔹 Spearman’s Rank Correlation Coefficient (ρ)

Developed by Charles Spearman.

$$\rho = 1 - \frac{6\sum d^2}{n(n^2 - 1)}$$

Where:

• $d$ = difference between ranks

• $n$ = number of observations

🔹 Properties

• Range: –1 ≤ ρ ≤ +1

• +1 → Perfect agreement in ranks

• –1 → Perfect disagreement in ranks

---

Difference Between Correlation Coefficient and Rank Correlation

Basis	Correlation Coefficient (Pearson)	Rank Correlation (Spearman)
Data type	Quantitative	Ordinal / Ranked
Uses actual values	Yes	No
Measures	Linear relationship	Monotonic relationship
Sensitivity to extreme values	High	Less sensitive