Lecture 10 - Sorting: complexity and recursion

Learning objectives

By the end of today, you should be able to:

• Sort by hand with selection sort, insertion sort, and merge sort
• Find the Big O of selection and insertion sort from their loops
• Read and write a recursive function: a base case plus a smaller version of the same problem
• Explain why merge sort is O(n log n) from its call tree
• Explain why the fastest sort for a person is not always the fastest for a computer

Part 1: Sorting the way people do

How do you sort a hand of cards?

You're dealt 8 cards. Put them in order, smallest to largest.

What do you actually do?

Selection sort: find the smallest, move it to the front

/// Sort 8 cards smallest to largest.
/// Find the smallest card left, swap it to the front, repeat.
fn selection_sort(mut cards: [i32; 8]) -> [i32; 8] {
    for front in 0..cards.len() {
        // Find where the smallest card is, from `front` to the end
        let mut smallest = front;
        for check in (front + 1)..cards.len() {
            if cards[check] < cards[smallest] {
                smallest = check;
            }
        }
        // Swap the smallest card into the front spot
        let temp = cards[front];
        cards[front] = cards[smallest];
        cards[smallest] = temp;
    }
    cards
}

fn main() {
    println!("{:?}", selection_sort([60, 30, 80, 10, 50, 20, 70, 40]));
}

A loop inside a loop: 7 + 6 + ... + 1 comparisons. O(n^2), like count_pairs on Wednesday.

Insertion sort: slide each card into place

/// Sort 8 cards smallest to largest.
/// Keep the left side sorted. Slide each new card left until it fits.
fn insertion_sort(mut cards: [i32; 8]) -> [i32; 8] {
    for next in 1..cards.len() {
        // Slide the card at `next` left while the card before it is bigger
        let mut spot = next;
        while spot > 0 && cards[spot - 1] > cards[spot] {
            let temp = cards[spot - 1];
            cards[spot - 1] = cards[spot];
            cards[spot] = temp;
            spot -= 1;
        }
    }
    cards
}

fn main() {
    println!("{:?}", insertion_sort([60, 30, 80, 10, 50, 20, 70, 40]));
}

Also a loop inside a loop: O(n^2) in the worst case.

Think-pair-share: does the starting order matter?

• What if the cards are already sorted? How many comparisons does each sort make?
• What if they're in reverse order?

Part 2: Recursion

What if you split the pile?

Sorting 16 cards alone is a lot of comparisons.

1. Split the pile in half and hand each half to a friend
2. Each friend splits their half and hands it off too
3. ...until everyone holds one card, which is already sorted

Every friend is doing the same job on a smaller pile.

A function that calls itself on a smaller version of the problem is recursive.

A recursive function you already know

Wednesday's sum_to, with a loop:

#![allow(unused)]
fn main() {
fn sum_to(n: u64) -> u64 {
    let mut total = 0;
    for i in 1..=n {
        total += i;
    }
    total
}
}

The same thing, with no loop:

fn sum_to(n: u64) -> u64 {
    if n == 0 {
        return 0;              // base case: nothing left to add
    }
    n + sum_to(n - 1)          // a smaller version of the same problem
}

fn main() {
    println!("{}", sum_to(3));
}

Tracing sum_to(3)

#![allow(unused)]
fn main() {
fn sum_to(n: u64) -> u64 {
    if n == 0 {
        return 0;
    }
    n + sum_to(n - 1)
}
}

What happens when we call sum_to(3)?

sum_to(3) = 3 + sum_to(2)
                = 2 + sum_to(1)
                        = 1 + sum_to(0)
                                = 0        base case, start returning
                        = 1 + 0 = 1
                = 2 + 1 = 3
          = 3 + 3 = 6

Every recursive function has two parts

1. A base case: a problem small enough to answer right away
2. A recursive step: call yourself on a problem that is smaller, closer to the base case

Write the base case first. Then ask: does every call get closer to it?

Think-pair-share: fill in the blanks

Wednesday's mystery function counted the digits in a number with a loop. Write it recursively:

/// Count the digits in n. count_digits(4096) is 4.
fn count_digits(n: u64) -> u64 {
    if n < 10 {
        return ___;
    }
    ___ + count_digits(___)
}

/// Count the digits in n. count_digits(4096) is 4.
fn count_digits(n: u64) -> u64 {
    if n < 10 {
        return 1;                  // one digit left
    }
    1 + count_digits(n / 10)       // this digit, plus the digits after chopping it off
}

fn main() {
    println!("{}", count_digits(4096));
}

What if there's no base case?

#![allow(unused)]
fn main() {
fn count_digits(n: u64) -> u64 {
    1 + count_digits(n / 10)
}

count_digits(115);
}

Part 3: Merge sort

Merging two sorted piles

Two piles, each already sorted, smallest on top:

Left:  20  50  80          Right:  10  30  90

Look at only the two top cards. Take the smaller one. Repeat.

20 vs 10  take 10     10
20 vs 30  take 20     10 20
50 vs 30  take 30     10 20 30
50 vs 90  take 50     10 20 30 50
80 vs 90  take 80     10 20 30 50 80
right pile left:      10 20 30 50 80 90

At most one comparison per card placed: merging is O(n).

Merge sort

To sort a pile:

1. Base case: one card? It's sorted. Done
2. Split the pile in half
3. Merge sort each half (recursion!)
4. Merge the two sorted halves

[60, 30, 80, 10, 50, 20, 70, 40]                 split
[60, 30, 80, 10]        [50, 20, 70, 40]         split
[60, 30]   [80, 10]     [50, 20]   [70, 40]      split
[60] [30]  [80] [10]    [50] [20]  [70] [40]     base case: one card each

[30, 60]   [10, 80]     [20, 50]   [40, 70]      merge
[10, 30, 60, 80]        [20, 40, 50, 70]         merge
[10, 20, 30, 40, 50, 60, 70, 80]                 merge

Why O(n log n)?

[10, 20, 30, 40, 50, 60, 70, 80]           merging this level touches 8 cards
[10, 30, 60, 80]  [20, 40, 50, 70]         8 cards
[30, 60] [10, 80] [20, 50] [40, 70]        8 cards

• Work per level: every card gets merged once, so about n
• Number of levels: halve 8 until you reach 1 card: 8, 4, 2, 1, so 3 levels so log n

n work on each of log n levels: O(n log n)

The picture of calls splitting into two smaller calls is a tree. You'll see a lot more of these.

Don't just memorize "merge sort is n log n." Draw the tree and count: how much work per level, times how many levels.

How much faster is that?

Comparisons for our 8 and 16 card starting orders (activity!):

Double the cards: selection does about 4x the comparisons. Merge does about 3x, and the gap keeps growing.

Quicksort and .sort()

Quicksort is another split-the-pile recursive sort. Pick one card, put smaller cards on its left and bigger on its right, then quicksort each side.

.sort(): in Rust you almost never write your own sort, you just:

fn main() {
    let mut cards = [60, 30, 80, 10, 50, 20, 70, 40];
    cards.sort();
    println!("{:?}", cards);
}

• .sort() is built on merge sort, and switches to insertion sort for 20 items or fewer
• .sort_unstable() is built on quicksort

Watch them sort

15 Sorting Algorithms in 6 Minutes

Watch for selection, insertion, merge, and quick. What shape does each one make as it works? For the ones we haven't talked about - can you guess what they do? (Heap sort, Radix sort, etc.)

Activity 10: Sorting race

• Groups of 5-6 (6 groups) - each get 4 packets
• Four people race at once, one each: insertion, selection, merge, and freestyle (your own strategy)
• Person 5 is the timer and fills in the Google Form. Person 6, if you have one, checks
• Same starting order for everyone. Round 1 with 8 cards, round 2 with 16
• Swap roles between rounds

Before each round: which sort will win?

Results

• Did the winner match the comparison counts?
• Which sort was easier for people than for a computer? Why?
• Did anything change between 8 and 16 cards? What about 1,000?
• Why would Rust's .sort() use insertion sort for small lists?

Appendix: merge sort in Rust

Just FYI if you're curious. Vec is a list that can grow, and &cards[..mid] means "a glance at the first half of cards". Both are coming later. Can you find the base case and the two recursive calls?

/// Merge two sorted piles into one sorted pile.
fn merge(left: &[i32], right: &[i32]) -> Vec<i32> {
    let mut result = Vec::new();
    let mut l = 0; // position in the left pile
    let mut r = 0; // position in the right pile
    while l < left.len() && r < right.len() {
        if left[l] <= right[r] {
            result.push(left[l]);
            l += 1;
        } else {
            result.push(right[r]);
            r += 1;
        }
    }
    // Once one pile is empty
    while l < left.len() {
        result.push(left[l]);
        l += 1;
    }
    while r < right.len() {
        result.push(right[r]);
        r += 1;
    }
    result
}

fn merge_sort(cards: &[i32]) -> Vec<i32> {
    if cards.len() <= 1 {
        return cards.to_vec();                 
    }
    let mid = cards.len() / 2;
    let left = merge_sort(&cards[..mid]);      
    let right = merge_sort(&cards[mid..]);     
    merge(&left, &right)
}

fn main() {
    println!("{:?}", merge_sort(&[60, 30, 80, 10, 50, 20, 70, 40]));
}