Bite-Sized Mathematics for Computer Science

Notes on MIT's 6.042J

Eda Eren
Table of Contents

Introduction

Prerequisites & Resources

Unit 1: Proofs

00. Introduction to Proofs

01. (Two) Proof Methods

02. The Well Ordering Principle

03. Logic & Propositions

04. Quantifiers and Predicate Logic

05. Sets

06. Binary Relations

07. Induction

08. State Machines — Invariants

09. Recursive Definitions

10. Infinite Sets

Unit 2: Structures

12. Congruences

13. Euler's Theorem

14. RSA Encryption

15. Digraphs: Walks and Paths

16. Directed Acyclic Graphs

17. Partial Orders and Equivalence

18. Degrees & Isomorphism

19. Coloring & Connectivity

20. Trees

21. Stable Matching

Unit 3: Counting

22. Sums & Products

23. Asymptotics

24. Counting with Bijections

25. Repetitions & Binomial Theorem

26. Pigeonhole Principle, Inclusion-Exclusion

Unit 4: Probability

27. Intro to Discrete Probability

28. Conditional Probability

29. Independence & Causality

30. Random Variables, Density Functions

31. Expectation

32. Deviation: Markov & Chebyshev Bounds

33. Sampling & Confidence

34. Random Walks & PageRank

GCDs

Let's start off with what we know about divisibility:

aa divides bb (a  ba \ | \ b) iff there is an integer kk such that ak=bak = b.

Then, we know that for all nn,

We also know some basic facts about divisibility:

Note: The number of the form sb+tcsb + tc is called the linear combination of bb and cc.


As long as aa and bb are not 00, they will have their greatest common divisor, and we can write it as gcd(a,b)\text{gcd}(a, b).

Euclid's Algorithm

We can find gcd\text{gcd}s with Euclid's Algorithm:

For b0b \neq 0,

gcd(a, b)=gcd(b,rem(a, b))\text{gcd}(a, \ b) = \text{gcd}(b, \text{rem}(a, \ b))

Okay, we know that a=bq+ra = bq + r, where qq is the quotient, and rr is the remainder. Notice that rr is 00 if bb divides aa.
If something divides both aa and bb, then it also divides rr. And, if something divides bb and rr, then it divides aa, too.

An easy example is to find the gcd\text{gcd} of 2626 and 2121:
gcd(26, 21)=gcd(21, 5)=gcd(5, 1)=1\text{gcd}(26, \ 21) = \text{gcd}(21, \ 5) = \text{gcd}(5, \ 1) = 1.

We can think of the Euclidean Algorithm as a state machine.
States will be a pair of nonnegative integers, N×N\mathbb{N} \times \mathbb{N}.
The start state is the original pair we want to find out the gcd\text{gcd} of, gcd(a, b)\text{gcd}(a, \ b). In the above example, it is gcd(26, 21)\text{gcd}(26, \ 21).
State transition rule is (x, y)(y,rem(x,y))(x, \ y) \longrightarrow (y, \text{rem}(x, y)) for y0y \neq 0.
And, here is the more interesting part: the preserved invariant is the gcd\text{gcd} itself, it stays constant. In the example, gcd(26, 21)\text{gcd}(26, \ 21) is 11, gcd(21, 5)\text{gcd}(21, \ 5) is also 11, and gcd(5, 1)\text{gcd}(5, \ 1) is also 11.
So, we can say that gcd(x, y)\text{gcd}(x, \ y) at any point is the same as the original value, gcd(a, b)\text{gcd}(a, \ b).

We can write a short recursive function for it:

def gcd(x, y):
    if y == 0:
        print(f'The result is: {x}')
        return x
    print(f'Calculating gcd({x}, {y})...')
    return gcd(y, x % y)

In this case gcd(26, 21) gives the output of:

Calculating gcd(26, 21)...
Calculating gcd(21, 5)...
Calculating gcd(5, 1)...
The result is: 1

There is also this fact:
gcd(a, b)=sa+tb\text{gcd}(a, \ b) = sa + tb says that the greatest common divisor of aa and bb is the linear combination of aa and bb.

If that's the case, we can also reason that an integer is a linear combination of aa and bb iff it is a multiple of gcd(a,b)\text{gcd}(a, b).

First, let's take a look at linear combinations more closely.

Let's say we want to write gcd(662, 414)\text{gcd}(662, \ 414) as a linear combination.
It has to be in the form: 662s+414t662s + 414t.
We can use Euclid's Algorithm, but let's go step by step:

662662 =414(1)+248= 414(1) + 248
414414 =248(1)+166= 248(1) + 166
248248 =166(1)+82= 166(1) + 82
166166 =82(2)+2= 82(2) + 2
8282 =2(41)+0= 2(41) + 0

You might already see the pattern.
Now, we're going to ignore the last row, and go reverse. We're going to rewrite everything with according to the remainder this time.

22 =1(166)2(82)= 1(166) - 2(82)
8282 =1(248)1(166)= 1(248) - 1(166)
166166 =1(414)1(248)= 1(414) - 1(248)
248248 =1(662)1(414)= 1(662) - 1(414)

Let's rewrite the same thing once again, but we'll plug in the values as shown.

So, we can still write 22 as =1(166)2(82)= 1(166) - 2(82), but let's plug in the value for 8282:

2=1(166)2(1(248)1(166))2 = 1(166) - 2(1(248) - 1(166))

Let's simplify it a bit:
2=1(166)2(248)+2(166)2 = 1(166) - 2(248) + 2(166).

And, even further:
2=2(248)+3(166)2 = -2(248) + 3(166).

Okay, now where are we going with this?

Let's continue plugging in values. This time for 166166:
2=2(248)+3(1(414)1(248))2 = -2(248) + 3(1(414) - 1(248)).

Now, simplify it: 2=3(414)5(248)2 = 3(414) - 5(248).

It's time to plug in the value for 248248:
2=3(414)5(1(662)+1(414))2 = 3(414) - 5(1(662) + 1(414)).

Simplify it: 2=5(662)+8(414)2 = -5(662) + 8(414).

In case you zoned out at some point during this process, let me repeat what we have as the end result: 5(662)+8(414)-5(662) + 8(414).
That's exactly what we were looking for in the first place, a number of the form 662s+414t662s + 414t!

The ss and tt are also called Bézout's coefficients. It is Bézout's identity that says we can write the greatest common divisor as a linear combination:

If a, bZ+a, \ b \in \mathbb{Z}^+, then  s, tZ\exists \ s, \ t \in Z such that gcd(a, b)=sa+tb\text{gcd}(a, \ b) = sa + tb.

The method we used to find out ss and tt is called the Extended Euclidean Algorithm (the "Pulverizer").