Bite-Sized Mathematics for Computer Science

Notes on MIT's 6.042J

Eda Eren
Table of Contents

Introduction

Prerequisites & Resources

Unit 1: Proofs

00. Introduction to Proofs

01. (Two) Proof Methods

02. The Well Ordering Principle

03. Logic & Propositions

04. Quantifiers and Predicate Logic

05. Sets

06. Binary Relations

07. Induction

08. State Machines — Invariants

09. Recursive Definitions

10. Infinite Sets

Unit 2: Structures

11. GCDs

12. Congruences

13. Euler's Theorem

14. RSA Encryption

15. Digraphs: Walks and Paths

16. Directed Acyclic Graphs

17. Partial Orders and Equivalence

18. Degrees & Isomorphism

19. Coloring & Connectivity

20. Trees

21. Stable Matching

Unit 3: Counting

22. Sums & Products

23. Asymptotics

24. Counting with Bijections

25. Repetitions & Binomial Theorem

Unit 4: Probability

27. Intro to Discrete Probability

28. Conditional Probability

29. Independence & Causality

30. Random Variables, Density Functions

31. Expectation

32. Deviation: Markov & Chebyshev Bounds

33. Sampling & Confidence

34. Random Walks & PageRank

Pigeonhole Principle, Inclusion-Exclusion

So, the pigeonhole principle. All that it says is this:

If there are more pigeons than pigeonholes, that means some holes must have more than 22 pigeons.

It is a mapping rule: if there is a total injection from AA to BB, then AB|A| \leq |B|.
That means, if A>B|A| \gt |B|, there is no total injection.

If we have nn pigeons and hh holes, then some hole has more than or equal to nh\lceil \frac{n}{h} \rceil (ceiling of nh\frac{n}{h}, ceiling being the rounded up value, e.g., 2.0001=3\lceil 2.0001 \rceil = 3).

For example, let's say we have 7 pigeons and 6 pigeonholes. Dividing 77 by 66 is 1.161.16, when we round up that value, we have 22 as a result. So, one of the pigeonholes has 22 pigeons in it.

Another example: ignoring leap years and assuming that one year has 365 days, the minimum number of people there has to be for at least 2 of them being born on the same day is 366.
Why is that? Well, imagine 365 people, there is a chance that each of them was born on different days. In order to make sure that at least 2 of them were born on the same day, there must be k+1k + 1 people, making it 366.

Let's look at another example from the practice section of the course.

At a certain school, there are 11 different classes offered to first-year students, and each student must enroll in exactly 4 of them. How many students must be in one class year to guarantee that at least 2 students will have the same schedule?

This is firstly an n choose k problem, nn being 11, kk being 4. In order to have at least 2 students have the same schedule, we need to add 1 to the result.

So, what we need to do is this: 11!4!(114)!+1\frac{11!}{4! \cdot (11 - 4)!} + 1.

We have 331 as the answer, so, there must be 331 students in one class year to guarantee that at least 2 students have the same schedule.


Let's take a look at the Principle of Inclusion-Exclusion.

We looked at the sum rule before, remember it says that if there are two disjoint sets, the size of their sum is the sum of the size of the first set and the size of the other set: AB=A+B|A \cup B| = |A| + |B|.

But, what if they are not disjoint? Like this:

Then, we would simply have to subtract the intersection from the above equation, and this is the principle of inclusion-exclusion:

AB=A+BAB|A \cup B| = |A| + |B| - |A \cap B|

The reason is that when we count A|A|, we have already counted the intersection, and when we count B|B|, that means we are doing it for the second time.

Another way to look at it is that when AA and BB are not disjoint, ABA \cup B is the union of AA and all that is in BB that is not in AA:

AB=A(BA)A \cup B = A \cup (B - A)

So, by the sum rule,

AB=A+BA|A \cup B| = |A| + |B - A|

From the picture, it looks like BA|B - A| is BAB|B| - |A \cap B|. If that's true, we can prove the Principle of Inclusion-Exclusion. So in order to do that, let's prove this lemma which says BA=BAB|B - A| = |B| - |A \cap B|.

We can break the set BB into two pieces, two disjoint sets: the intersection part, and the rest:

B=(BA)(BA)B = (B \cap A) \cup (B - A)

Therefore, by the sum rule:

B=BA+BA|B| = |B \cap A| + |B - A|

With a little arrangement, we see that

BA=BAB|B - A| = |B| - |A \cap B|

which was our starting point.

Plugging this definition of BA|B - A| into AB=A+BA|A \cup B| = |A| + |B - A|, we see:

AB=A+BAB|A \cup B| = |A| + |B| - |A \cap B|

Therefore, it proves our lemma, and the Principle of Inclusion-Exclusion. \blacksquare


What if we have three sets?
In that case, we would still subtract the intersections, but if we do, there will still be a small section where all the sets intersect that we need to add back in:

ABC=A+B+CABACBC+ABC|A \cup B \cup C| = |A| + |B| + |C| - |A \cap B|- |A \cap C| - |B \cap C| + |A \cap B \cap C|

This picture taken from the slides of the lecture helps visualize it:

Inclusion-Exclusion

One thing to realize is that intersections with odd number of sets occur positively, and those with even numbers occur negatively. For example, AB|A \cap B|, or AC|A \cap C|, or, BC|B \cap C| are intersections with 22 sets, and they show up with a negative sign. But, ABC|A \cap B \cap C| has three sets, and it has a positive sign.

So, we have nn sets that are overlapping, and want to know the size of their union, A1A2 ... An|A_1 \cup A_2 \cup \ ... \ \cup A_n|.
We can express it as the sum of the sizes of the intersections, the sizes of the intersection AiA_i's that are in this set of indices.
We need to specify the sign of that particular size of intersection, if it is odd, we want to add it with a positive sign; if it's even, we add it negatively. We can use (1)S+1(-1)^{|S| + 1}. If S|S| (the size of the intersection) is an odd number, it will be positive; if it's even, it'll be negative.

To write it down precisely:

S{1, 2, ..., n}(1)S+1 iSAi\displaystyle\sum_{\varnothing \neq S \subseteq \{1, \ 2, \ ..., \ n\}} (-1)^{|S| + 1} \ {\Big| \bigcap_{i \in S} A_i \Big|}