Probability

1. What is Probability?

Probability is a measure of the likelihood or chance that a particular event will occur. The probability P of an event A occurring is given by:

\[ P(A) = \frac{\text{Number of favorable outcomes for event } A}{\text{Total number of possible outcomes}} \]

Eg. Let’s see an example about Football Match Win Prediction. The problem is that we want to predict the probability that a football team will win their next match based on their historical win rate.

• Total matches played by the team: 50
• Matches won: 30
• Matches drawn: 10
• Matches lost: 10

– The probability of winning a match

\[ P(\text{win}) = \frac{30}{50} = 0.6 \]

– The probability of drawing a match

\[ P(\text{draw}) = \frac{10}{50} = 0.2 \]

– The probability of losing a match

\[ P(\text{loss}) = \frac{10}{50} = 0.2 \]

2. Complementary Probability

Complementary probability refers to the probability of the complement of an event occurring. If A is an event, the complement of A is A’, is the event that A does not occur. The sum of the probabilities of an event and its complement is always equal to 1.

\[ P(A’) = 1 – P(A) \]

• $\backslash (\; P(A)\; \backslash )$ is the probability of event A occurring.
• $\backslash (\; P(A’)\; \backslash )$ is the probability of event A not occurring.

Using above example data:

\[ P(\text{win}) = 0.6 \]

\[ \Rightarrow P(\text{not win}) = 1- 0.6 = 0.4 \]

3. Conditional Probability

Conditional probability is the probability of an event occurring given that another event has already occurred. It provides a way to update the probability of an event based on new information. The conditional probability of event A given event B is denoted as P(A|B) and is defined as:

\[ P(A|B) = \frac{P(A \cap B)}{P(B)} \]

• $\backslash (\; P(A\; \backslash cap\; B)\; \backslash )$is the probability that both events A and B occur.
• $\backslash (\; P(B)\; \backslash )$ is the probability that event B occurs, and it must be greater than 0.

Eg. We want to predict the probability that a football team will win their next match given that they are playing at home.

• Total matches played by the team: 100
• Matches played at home: 50
• Matches won: 60
• Matches won at home: 35

– The probability of playing at home

\[ P(\text{Home}) = \frac{50}{100} = 0.5 \]

– The probability of winning and playing at home

\[ P(\text{Win} \cap \text{Home}) = \frac{35}{100} = 0.35 \]

– The conditional probability of winning given that the team is playing at home

\[ P(\text{Win}|\text{Home}) = \frac{P(\text{Win} \cap \text{Home})}{P(\text{Home})} = \frac{0.35}{0.5} = 0.7 \]

4. Addition Rule in Probability

The addition rule in probability is used to find the probability of the union of two events, which is the probability that either event occurs. There are two forms of the addition rule.

If two events A and B cannot happen at the same time:

\[ P(A \cup B) = P(A) + P(B) \]

If two events A and B can happen at the same time:

\[ P(A \cup B) = P(A) + P(B) – P(A \cap B) \]

Eg. We have the following probabilities for a football team’s matches:

• The probability that the team wins a match is 40% => \(P(\text{Win}) = 0.4\)
• The probability that the team plays at home is 60% => \(P(\text{Home}) = 0.6\)
• The probability that the team wins and plays at home is 30% => \(P(\text{Win} \cap \text{Home}) = 0.3\)

We want to find the probability that the team either wins or plays at home:

\[ \Rightarrow P(\text{Win} \cup \text{Home}) = 0.4 + 0.6 – 0.3 = 0.7 \]

5. Bayes Theorem

Bayes Theorem is a fundamental concept in probability theory and statistics that describes how to update the probability of a hypothesis based on new evidence. For two events A is the hypothesis and B is the evidence:

\[ P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} \]

• \(P(A|B)\) probability of A given B is true.
• \(P(B|A)\) probability of B given A is true..
• \(P(A)\) is the independent probabilities of A.
• \(P(B)\) is the independent probabilities of B.

5.1. Proof of Bayes Theorem

The conditional probability of A given B is defined as:

\[ P(A|B) = \frac{P(A \cap B)}{P(B)} \]

\[ \Rightarrow P(A \cap B) = P(B) \cdot P(A|B) \]

Similarly, the conditional probability of B given A is:

\[ P(B|A) = \frac{P(A \cap B)}{P(A)} \]

\[ \Rightarrow P(A \cap B) = P(A) \cdot P(B|A) \]

So

\[ P(A) \cdot P(B|A) = P(B) \cdot P(A|B) \]

\[ \Rightarrow P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} \]

5.2. Bayes Theorem example

Suppose a football team has the following statistics:

• The team wins 40% of their matches => \(P(\text{Win}) = 0.4\)
• When they win a match, there is a 70% chance it was a home match => \(P(\text{Home}|\text{Win}) = 0.7\)
• The team plays 60% of their matches at home => \(P(\text{Home}) = 0.6\)

We want to calculate the probability that the team won a match given that they played at home.

\[ P(\text{Win}|\text{Home}) = \frac{P(\text{Home}|\text{Win}) \cdot P(\text{Win})}{P(\text{Home})} \]

\[ \Rightarrow P(\text{Win}|\text{Home}) = \frac{0.7 \cdot 0.4}{0.6} = \frac{0.28}{0.6} = 0.467 \]