20. Review the probability axioms, the union bound, the definition of probability and the Bayes Theorem

In this post, I will give an overview of the concepts which were already covered in the previous blog posts. First of all, let’s begin with the probability axioms, also known as Kolmogorov axioms. There are three of them and they are:

1. The probability of an event is a non-negative real number: P(E) ≥ 0
2. The probability that at least one of the elementary events 
    in the entire sample space will occur is 1: P(Ω) = 1
3. Any countable sequence of disjoint sets E1,E2,…En satisfies: 
        P(E1 U E2 U … U En)=P(E1)+P(E2)+ … +P(En)

The next concept which we are going to review is the union bound. In probability theory, Boole’s inequality, also known as the union bound, says that for any finite or countable set of events, the probability that at least one of the events happens is no greater than the sum of the probabilities of the individual events. In other words, if we have a countable set of events A1, A2, … , An, we can say that: P(A1 U A2 U … U An) ≤ P(A1)+P(A2)+ … +P(An).

Next is the definition of probability. There are a few definitions of probability, depending on the approach used in defining it. We will review the classic one which says: If an experiment can produce N mutually exclusive and equally likely outcomes, out of which n outcomes are favorable to the occurrence of event A, then the probability of A is denoted by P(A) and is defined as the ratio n/N. Thus the probability of A is given by:

  P(A) = Number of outcomes favorable to A/Number of possible outcomes = n/N

The last thing which we are going to review is the Bayes’ Theorem. The Bayes’ theorem describes the probability of an event, based on prior knowledge of conditions that might be related to the event. Bayes’ theorem is stated mathematically as the following equation:

              P(A|B)=P(B|A)P(A)/{(B)
  1. where A and B are events and P(B) ≠ 0.
  2. P(A B) is a conditional probability: the likelihood of event A occurring given that B is true.
  3. P(B A) is also a conditional probability: the likelihood of event B occurring given that A is true.
  4. P(A) and P(B) are the probabilities of observing A and B independently of each other; this is known as the marginal probability.