One logic,
three dialects.
The same algorithm, written three ways — in Python for clarity, Perl for text-and-glue expressiveness, and GNU Octave for numerical muscle. With proofs of why each works and live labs to watch them run.
Section 0Foundations & the Three Dialects
An algorithm is a finite, unambiguous procedure that transforms input into output in a finite number of well-defined steps. Three properties must hold: every step is definite, the procedure is effective (each step mechanically doable), and it always terminates.
We count elementary operations — a comparison, an arithmetic step, an array access — each at unit cost, on an idealized machine with random access to memory (the RAM model). This lets us talk about running time as a function of input size \(n\) without drowning in hardware detail.
Why three languages?
An algorithm is a pattern of thought, not a pile of syntax. Writing the same routine in three very different languages makes the pattern stand out from the dialect. Each language also has a temperament worth knowing:
Python reads like pseudocode — comprehensions, slices, and built-in dicts make intent obvious. Perl is the text-wrangler's tool: hashes, regexes, and map/grep make it terse for parsing and glue. Octave thinks in vectors and matrices, with 1-based indexing and logical-mask operations that turn loops into one-liners — ideal for numerical work.
Use the language switch at the top right (or the tabs on any code block) to read the whole guide in whichever dialect you prefer — every snippet flips at once.
A first taste: never loop when math will do
Summing \(1+2+\dots+n\) with a loop is \(O(n)\). Gauss's closed form is \(O(1)\) — the purest example of why choosing the right algorithm beats any amount of fast code.
def sum_to_n(n):
return n * (n + 1) // 2 # Gauss: O(1), no loop neededsub sum_to_n {
my $n = shift;
return $n * ($n + 1) / 2; # the closed form beats any loop
}function s = sum_to_n(n)
s = n * (n + 1) / 2; % closed form, constant time
endTwo things make a program good: an algorithm that is correct (provable) and efficient (analyzable). The labs let you see the behavior; the proofs tell you it holds for every input, not just the ones you tried.
Section 1Asymptotic Analysis
We care how cost scales, not the constants that change with every CPU. Asymptotic notation captures growth as \(n\to\infty\).
\(f(n)=O(g(n))\): there exist \(c,n_0>0\) with \(0\le f(n)\le c\,g(n)\) for all \(n\ge n_0\) (an upper bound).
\(f(n)=\Omega(g(n))\): a lower bound. \(f(n)=\Theta(g(n))\): both at once — same growth rate.
\(3n^2+5n+2 = O(n^2)\).
For all \(n\ge 1\), \(5n\le 5n^2\) and \(2\le 2n^2\), so \(3n^2+5n+2\le 3n^2+5n^2+2n^2 = 10n^2\). Taking \(c=10,\ n_0=1\) satisfies the definition, hence \(3n^2+5n+2=O(n^2)\). ∎
\(n^2 \ne O(n)\).
Suppose \(n^2=O(n)\). Then there exist \(c,n_0>0\) with \(n^2\le c\,n\) for all \(n\ge n_0\). Dividing by \(n\) gives \(n\le c\) for all \(n\ge n_0\). But \(n\) grows without bound, so any \(n>\max(c,n_0)\) breaks the inequality — a contradiction. So \(n^2\ne O(n)\). ∎
| Class | Name | Example |
|---|---|---|
| \(O(1)\) | constant | array index, hash lookup |
| \(O(\log n)\) | logarithmic | binary search |
| \(O(n)\) | linear | linear scan |
| \(O(n\log n)\) | linearithmic | merge sort |
| \(O(n^2)\) | quadratic | insertion sort (worst) |
| \(O(2^n)\) | exponential | naive subset search |
Section 2Recurrences & the Master Theorem
Divide-and-conquer algorithms describe their own cost recursively. Solving the recurrence gives the running time.
The recurrence \(T(n)=2T(n/2)+n,\ T(1)=1\) satisfies \(T(n)=O(n\log n)\).
Guess \(T(n)\le c\,n\log_2 n\) for some \(c>0\) and all \(n\ge 2\), and assume it for \(n/2\). Then \[T(n)=2T(n/2)+n\le 2\left(c\tfrac n2\log_2\tfrac n2\right)+n = c\,n(\log_2 n-1)+n = c\,n\log_2 n-(c-1)n.\] For any \(c\ge 1\) the term \(-(c-1)n\le 0\), so \(T(n)\le c\,n\log_2 n\), closing the induction. Hence \(T(n)=O(n\log n)\). ∎
For \(T(n)=a\,T(n/b)+f(n)\) with \(a\ge 1,\ b>1\), compare \(f(n)\) with \(n^{\log_b a}\):
Case 1. \(f(n)=O(n^{\log_b a-\varepsilon})\Rightarrow T(n)=\Theta(n^{\log_b a})\) (leaves dominate).
Case 2. \(f(n)=\Theta(n^{\log_b a})\Rightarrow T(n)=\Theta(n^{\log_b a}\log n)\) (balanced).
Case 3. \(f(n)=\Omega(n^{\log_b a+\varepsilon})\) with regularity \(\Rightarrow T(n)=\Theta(f(n))\) (root dominates).
Merge sort: \(T(n)=2T(n/2)+\Theta(n)\). Here \(a=b=2\), \(n^{\log_b a}=n\), \(f(n)=\Theta(n)\) — Case 2 gives \(\Theta(n\log n)\).
Binary search: \(T(n)=T(n/2)+\Theta(1)\). Here \(a=1,b=2\), \(n^{\log_2 1}=1\), \(f(n)=\Theta(1)\) — Case 2 gives \(\Theta(\log n)\).
Section 3Searching
Finding an element in a collection. One structural assumption — sortedness — collapses a linear scan into a logarithmic leap.
Linear search — \(O(n)\), no assumptions
def linear_search(a, target):
for i, x in enumerate(a): # 0-based index, value
if x == target:
return i # first match wins
return -1 # sentinel: not foundsub linear_search {
my ($a, $target) = @_; # array ref, value
for my $i (0 .. $#$a) { # 0 .. last index
return $i if $a->[$i] == $target;
}
return -1; # not found
}function idx = linear_search(A, target)
idx = -1; % sentinel: not found
for i = 1:numel(A) % Octave is 1-based
if A(i) == target
idx = i; return; % first match wins
end
end
endBinary search — \(O(\log n)\) on sorted data
Inspect the middle. If it is the target, done. If the target is smaller, it can only be in the left half; if larger, the right half. Each step discards half of what remains.
def binary_search(a, target):
lo, hi = 0, len(a) - 1
while lo <= hi:
mid = (lo + hi) // 2 # integer midpoint
if a[mid] == target:
return mid
elif a[mid] < target:
lo = mid + 1 # answer is to the right
else:
hi = mid - 1 # answer is to the left
return -1sub binary_search {
my ($a, $target) = @_;
my ($lo, $hi) = (0, $#$a);
while ($lo <= $hi) {
my $mid = int(($lo + $hi) / 2);
if ($a->[$mid] == $target) { return $mid; }
elsif ($a->[$mid] < $target) { $lo = $mid + 1; }
else { $hi = $mid - 1; }
}
return -1;
}function idx = binary_search(A, target)
% PRECONDITION: A is sorted ascending.
lo = 1; hi = numel(A); idx = -1;
while lo <= hi
mid = floor((lo + hi) / 2);
if A(mid) == target
idx = mid; return;
elseif A(mid) < target
lo = mid + 1; % answer is to the right
else
hi = mid - 1; % answer is to the left
end
end
endBinary search is correct and runs in \(O(\log n)\) time.
Correctness (invariant). The invariant: if the target is in the array, it lies in the window A[lo..hi]. Initially the window is the whole array. Each step inspects the middle: if it is less than the target, sortedness puts the target (if present) strictly to the right — the new window; the symmetric case handles a greater middle. The loop exits by returning a hit, or with an empty window, when the invariant says the target is absent and the not-found sentinel is correct.
Running time. The window size \(s=hi-lo+1\) at most halves each step, so after \(k\) steps it is \(\le n/2^k\). The loop stops once \(2^k>n\), i.e. after \(\le \lfloor\log_2 n\rfloor+1\) steps of \(O(1)\) work each: \(O(\log n)\). ∎
Section 4Sorting
The most-studied problem in computing — and a showcase of the trade-off between simple-but-slow and clever-but-subtle.
Insertion sort — \(O(n^2)\), lovely on nearly-sorted data
def insertion_sort(a):
a = a[:] # work on a copy
for i in range(1, len(a)):
key, j = a[i], i - 1
while j >= 0 and a[j] > key:
a[j + 1] = a[j] # shift larger elements right
j -= 1
a[j + 1] = key # drop key into the gap
return asub insertion_sort {
my @a = @{ $_[0] }; # copy the array
for my $i (1 .. $#a) {
my $key = $a[$i];
my $j = $i - 1;
while ($j >= 0 && $a[$j] > $key) {
$a[$j + 1] = $a[$j]; # shift right
$j--;
}
$a[$j + 1] = $key;
}
return \@a;
}function A = insertion_sort(A)
for i = 2:numel(A)
key = A(i); j = i - 1;
while j >= 1 && A(j) > key
A(j + 1) = A(j); % shift larger elements right
j = j - 1;
end
A(j + 1) = key; % drop key into the gap
end
endInsertion sort returns a sorted permutation of its input.
Invariant: at the start of the pass for index \(i\), the prefix before \(i\) holds its original elements in sorted order. Initialization: a one-element prefix is trivially sorted. Maintenance: the inner loop shifts every element greater than the key one slot right, then drops the key into the gap; the prefix is one longer and still sorted. Termination: when \(i\) passes the last index the whole array is sorted, and since only shifts and one insertion occurred it is a permutation of the input. ∎
Merge sort — \(\Theta(n\log n)\), always
Split in half, sort each half recursively, merge the two sorted halves in linear time.
def merge_sort(a):
if len(a) <= 1:
return a[:]
mid = len(a) // 2
L, R = merge_sort(a[:mid]), merge_sort(a[mid:])
out, i, j = [], 0, 0
while i < len(L) and j < len(R):
if L[i] <= R[j]: # <= keeps the sort stable
out.append(L[i]); i += 1
else:
out.append(R[j]); j += 1
out.extend(L[i:]); out.extend(R[j:])
return outsub merge_sort {
my @a = @{ $_[0] };
return [@a] if @a <= 1;
my $mid = int(@a / 2);
my $L = merge_sort([ @a[0 .. $mid - 1] ]);
my $R = merge_sort([ @a[$mid .. $#a] ]);
my @out;
my ($i, $j) = (0, 0);
while ($i < @$L && $j < @$R) {
if ($L->[$i] <= $R->[$j]) { push @out, $L->[$i++]; }
else { push @out, $R->[$j++]; }
}
push @out, @{$L}[$i .. $#$L] if $i <= $#$L;
push @out, @{$R}[$j .. $#$R] if $j <= $#$R;
return \@out;
}function A = merge_sort(A)
if numel(A) <= 1, return; end
mid = floor(numel(A) / 2);
L = merge_sort(A(1:mid));
R = merge_sort(A(mid+1:end));
A = merge(L, R);
end
function out = merge(L, R)
out = zeros(1, numel(L) + numel(R));
i = 1; j = 1; k = 1;
while i <= numel(L) && j <= numel(R)
if L(i) <= R(j)
out(k) = L(i); i = i + 1;
else
out(k) = R(j); j = j + 1;
end
k = k + 1;
end
while i <= numel(L), out(k) = L(i); i = i + 1; k = k + 1; end
while j <= numel(R), out(k) = R(j); j = j + 1; k = k + 1; end
endThe recurrence is \(T(n)=2T(n/2)+\Theta(n)\). By Case 2 of the Master Theorem (Section 2) this is \(\Theta(n\log n)\) — and unlike quicksort, the bound holds in the worst case too.
Quick sort — \(\Theta(n\log n)\) average, \(\Theta(n^2)\) worst
Pick a pivot, partition into smaller / equal / larger, recurse on the outer parts. Fast in practice; the worst case needs an adversarial input and a bad pivot rule.
def quick_sort(a):
if len(a) <= 1:
return a[:]
pivot = a[len(a) // 2]
less = [x for x in a if x < pivot]
eq = [x for x in a if x == pivot]
gre = [x for x in a if x > pivot]
return quick_sort(less) + eq + quick_sort(gre)sub quick_sort {
my @a = @{ $_[0] };
return [@a] if @a <= 1;
my $pivot = $a[int(@a / 2)];
my @less = grep { $_ < $pivot } @a;
my @eq = grep { $_ == $pivot } @a;
my @gre = grep { $_ > $pivot } @a;
return [ @{ quick_sort(\@less) }, @eq, @{ quick_sort(\@gre) } ];
}function A = quick_sort(A)
if numel(A) <= 1, return; end
pivot = A(floor(numel(A) / 2));
less = A(A < pivot); % logical indexing does the partition
eq = A(A == pivot);
gre = A(A > pivot);
A = [quick_sort(less), eq, quick_sort(gre)];
end| Algorithm | Best | Average | Worst | Stable? |
|---|---|---|---|---|
| Insertion | \(\Theta(n)\) | \(\Theta(n^2)\) | \(\Theta(n^2)\) | yes |
| Merge | \(\Theta(n\log n)\) | \(\Theta(n\log n)\) | \(\Theta(n\log n)\) | yes |
| Quick | \(\Theta(n\log n)\) | \(\Theta(n\log n)\) | \(\Theta(n^2)\) | no |
Any comparison-based sort needs \(\Omega(n\log n)\) comparisons in the worst case.
A comparison sort is a binary decision tree: each internal node is one comparison, each leaf one of the \(n!\) orderings. Every permutation must be reachable, so the tree has \(\ge n!\) leaves. A binary tree of height \(h\) has \(\le 2^h\) leaves, so \(2^h\ge n!\), giving \(h\ge\log_2(n!)\). By Stirling, \(\log_2(n!)=\Theta(n\log n)\). The height is the worst-case comparison count, so it is \(\Omega(n\log n)\). ∎
Section 5Hashing & Hash Tables
Give up order entirely and lookups, inserts, and deletes all become \(O(1)\) — on average. This is the structure behind every dictionary and set in the three languages.
A hash table stores key–value pairs in \(m\) buckets; a hash function \(h\) maps each key to a bucket. The load factor \(\alpha=n/m\) is the average keys per bucket. Two keys hitting one bucket is a collision, resolved by chaining (a list per bucket) or open addressing (probe onward).
All three languages give you a hash table as a first-class type — Python's dict, Perl's %hash, Octave's containers.Map. A word-frequency count shows the idiom in each:
from collections import defaultdict
words = ["fig", "ant", "fig", "cat", "ant", "fig"]
freq = defaultdict(int)
for w in words:
freq[w] += 1 # hash lookup + update, O(1) expected
print(freq["fig"]) # -> 3my @words = qw(fig ant fig cat ant fig);
my %freq;
$freq{$_}++ for @words; # autovivify + increment, O(1) expected
print "$freq{fig}\n"; # -> 3words = {"fig", "ant", "fig", "cat", "ant", "fig"};
freq = containers.Map("KeyType", "char", "ValueType", "double");
for i = 1:numel(words)
w = words{i};
if isKey(freq, w)
freq(w) = freq(w) + 1;
else
freq(w) = 1;
end
end
disp(freq("fig")) % -> 3Under simple uniform hashing, a search in a chaining table takes expected time \(\Theta(1+\alpha)\). With \(m=\Theta(n)\), every operation is \(\Theta(1)\) expected.
Simple uniform hashing means each key is equally likely to land in any of the \(m\) buckets, independently. For an unsuccessful search of key \(k\), the work is the length of bucket \(h(k)\). Define \(X_i=1\) when stored key \(i\) hashes there; \(\mathbb{E}[X_i]=1/m\), so by linearity the expected chain length is \(\sum_{i=1}^{n}1/m = n/m = \alpha\). Adding the \(O(1)\) to compute \(h(k)\) gives expected \(\Theta(1+\alpha)\). Choosing \(m=\Theta(n)\) makes \(\alpha=O(1)\). ∎
That \(O(1)\) is an expectation. The worst case is \(O(n)\) — an adversary who knows \(h\) can force every key into one bucket. Real implementations defend with randomized or universal hashing so no fixed input is reliably bad.
Section 6Dynamic Programming
When a problem has optimal substructure (its best solution is built from best sub-solutions) and overlapping subproblems (the same sub-solutions recur), solve each subproblem once and remember the answer.
Fibonacci — memoization vs tabulation
Naive recursion recomputes the same values exponentially often. Caching them collapses the cost to \(\Theta(n)\). Python and Perl cache top-down (memoization); the Octave version fills a table bottom-up (tabulation) — two faces of one idea.
def fib(n, memo=None): # top-down memoization
if memo is None:
memo = {}
if n < 2:
return n
if n in memo:
return memo[n]
memo[n] = fib(n - 1, memo) + fib(n - 2, memo)
return memo[n]{
my %memo; # closure: a private cache
sub fib {
my $n = shift;
return $n if $n < 2;
return $memo{$n} if exists $memo{$n};
$memo{$n} = fib($n - 1) + fib($n - 2);
return $memo{$n};
}
}function f = fib(n) % bottom-up tabulation
if n < 2, f = n; return; end
t = zeros(1, n + 1); % t(k) holds fib(k-1)
t(2) = 1;
for k = 3:n+1
t(k) = t(k-1) + t(k-2);
end
f = t(n + 1);
end0/1 Knapsack
Given items with weights and values and a capacity \(W\), choose a subset of maximum value that fits. Each cell asks: better to skip item \(i\), or take it and free up its weight?
def knapsack(weights, values, W):
n = len(weights)
dp = [[0] * (W + 1) for _ in range(n + 1)]
for i in range(1, n + 1):
for w in range(W + 1):
dp[i][w] = dp[i - 1][w] # skip item i
if weights[i - 1] <= w: # or take it
take = dp[i - 1][w - weights[i - 1]] + values[i - 1]
dp[i][w] = max(dp[i][w], take)
return dp[n][W]sub knapsack {
my ($w, $v, $W) = @_;
my $n = scalar @$w;
my @dp;
for my $i (0 .. $n) { $dp[$i][$_] = 0 for 0 .. $W; }
for my $i (1 .. $n) {
for my $cap (0 .. $W) {
$dp[$i][$cap] = $dp[$i - 1][$cap]; # skip item i
if ($w->[$i - 1] <= $cap) { # or take it
my $take = $dp[$i - 1][$cap - $w->[$i - 1]] + $v->[$i - 1];
$dp[$i][$cap] = $take if $take > $dp[$i][$cap];
}
}
}
return $dp[$n][$W];
}function best = knapsack(weights, values, W)
n = numel(weights);
dp = zeros(n + 1, W + 1); % dp(i+1,c+1): best value, first i items, cap c
for i = 1:n
for c = 0:W
dp(i+1, c+1) = dp(i, c+1); % skip item i
if weights(i) <= c % or take it
take = dp(i, c - weights(i) + 1) + values(i);
if take > dp(i+1, c+1)
dp(i+1, c+1) = take;
end
end
end
end
best = dp(n + 1, W + 1);
end\(dp[i][w]=\max\bigl(dp[i-1][w],\ dp[i-1][w-w_i]+v_i\bigr)\). The table is \((n+1)\times(W+1)\) and each cell is \(O(1)\), so the algorithm runs in \(\Theta(nW)\) time — efficient when \(W\) is modest, though it is pseudo-polynomial in the value of \(W\).
Section 7Greedy Algorithms
Make the locally optimal choice at each step and never reconsider. Simple and fast — but correct only when the problem has the greedy-choice property, which must be proven.
Activity selection
Given activities with start and finish times, pick the largest set that do not overlap. The greedy rule: always take the next activity that finishes earliest.
def activity_select(acts): # acts: list of (start, finish)
acts = sorted(acts, key=lambda p: p[1]) # sort by finish time
chosen, last_finish = [], float("-inf")
for s, f in acts:
if s >= last_finish: # compatible with the last pick
chosen.append((s, f))
last_finish = f
return chosensub activity_select {
my $acts = shift; # arrayref of [start, finish]
my @sorted = sort { $a->[1] <=> $b->[1] } @$acts; # by finish time
my @chosen;
my $last = -1e15;
for my $p (@sorted) {
my ($s, $f) = @$p;
if ($s >= $last) { # compatible with the last pick
push @chosen, $p;
$last = $f;
}
}
return \@chosen;
}function chosen = activity_select(acts) % acts: rows of [start, finish]
[~, idx] = sort(acts(:, 2)); % sort by finish time
acts = acts(idx, :);
chosen = []; last = -inf;
for i = 1:rows(acts)
if acts(i, 1) >= last % compatible with the last pick
chosen(end+1, :) = acts(i, :);
last = acts(i, 2);
end
end
endChoosing the compatible activity with the earliest finish time yields a maximum-size set.
Let \(g\) finish earliest. Take any optimal solution \(O\) and let \(o\) be its earliest-finishing activity. Since \(g\) finishes no later than \(o\), swapping \(o\) for \(g\) keeps every activity compatible and the count unchanged, so some optimal solution contains \(g\). Removing \(g\) and the activities it overlaps leaves a smaller subproblem; by induction the greedy choice extends to a global optimum. ∎
For 0/1 knapsack, taking the best value-per-weight item first can be strictly suboptimal — that needs the DP of Section 6. For coins \(\{1,3,4\}\) making 6, greedy takes \(4+1+1\) (three coins) while the optimum is \(3+3\) (two). Never trust a greedy algorithm without an exchange-argument proof.
Section 8Graph Algorithms
Graphs model networks — roads, links, dependencies. We store them as adjacency lists: Python a dict of lists, Perl a hash of arrayrefs, Octave a cell array of neighbor vectors.
Breadth-first search — shortest paths in unweighted graphs
Explore in rings of increasing distance with a FIFO queue. BFS visits every vertex and edge once: \(O(V+E)\).
from collections import deque
def bfs(adj, s): # adj: dict node -> list of neighbors
seen, order, q = {s}, [], deque([s])
while q:
u = q.popleft() # FIFO queue
order.append(u)
for v in adj[u]:
if v not in seen:
seen.add(v)
q.append(v)
return ordersub bfs {
my ($adj, $s) = @_; # adj: hashref node -> arrayref of neighbors
my %seen = ($s => 1);
my @order;
my @q = ($s);
while (@q) {
my $u = shift @q; # FIFO queue
push @order, $u;
for my $v (@{ $adj->{$u} }) {
unless ($seen{$v}) {
$seen{$v} = 1;
push @q, $v;
}
}
}
return \@order;
}function order = bfs(adj, s) % adj: cell array, adj{u} = neighbor vector
n = numel(adj);
seen = false(1, n); seen(s) = true;
queue = [s]; order = [];
while ~isempty(queue)
u = queue(1); queue(1) = []; % dequeue front (FIFO)
order(end+1) = u;
for v = adj{u}
if ~seen(v)
seen(v) = true;
queue(end+1) = v; % enqueue
end
end
end
endDijkstra — shortest paths with non-negative weights
Repeatedly finalize the closest unvisited vertex and relax its edges. (Python uses a binary heap; the Perl and Octave versions scan for the minimum — clearer, and fine for modest graphs.)
import heapq
def dijkstra(adj, s): # adj: dict u -> list of (v, weight)
dist = {u: float("inf") for u in adj}
dist[s] = 0
pq = [(0, s)] # min-heap of (distance, node)
while pq:
d, u = heapq.heappop(pq)
if d > dist[u]:
continue # stale entry
for v, w in adj[u]:
if d + w < dist[v]: # relax the edge
dist[v] = d + w
heapq.heappush(pq, (dist[v], v))
return distsub dijkstra {
my ($adj, $s) = @_; # adj: hashref u -> arrayref of [v, w]
my %dist = map { $_ => 1e15 } keys %$adj;
$dist{$s} = 0;
my %done;
while (1) {
my ($u, $best) = (undef, 1e15);
for my $node (keys %$adj) { # pick closest unfinished node
next if $done{$node};
($u, $best) = ($node, $dist{$node}) if $dist{$node} < $best;
}
last unless defined $u;
$done{$u} = 1;
for my $e (@{ $adj->{$u} }) {
my ($v, $w) = @$e;
$dist{$v} = $dist{$u} + $w if $dist{$u} + $w < $dist{$v}; # relax
}
}
return \%dist;
}function dist = dijkstra(adj, s) % adj{u} = rows of [neighbor, weight]
n = numel(adj);
dist = inf(1, n); dist(s) = 0;
done = false(1, n);
for it = 1:n
u = -1; best = inf;
for k = 1:n % pick closest unfinished node
if ~done(k) && dist(k) < best
best = dist(k); u = k;
end
end
if u == -1, break; end
done(u) = true;
E = adj{u};
for r = 1:rows(E)
v = E(r, 1); w = E(r, 2);
if dist(u) + w < dist(v) % relax the edge
dist(v) = dist(u) + w;
end
end
end
endOn a graph with non-negative edge weights, Dijkstra computes correct shortest-path distances.
Claim: when a vertex \(u\) is finalized, \(\operatorname{dist}(u)\) is its true shortest distance. Suppose not, and let \(u\) be the first vertex finalized with a too-large value. Its true shortest path leaves the finalized set at an edge \((x,y)\) with \(x\) finalized (correctly, as \(u\) is the first error) and \(y\) not. When \(x\) was finalized its edges were relaxed, so \(\operatorname{dist}(y)\le\operatorname{dist}(x)+w(x,y)\), the true distance to \(y\), which by non-negativity is \(\le\) the true distance to \(u \le \operatorname{dist}(u)\). Then \(y\) would have been finalized before \(u\) — contradiction. ∎
A single negative edge breaks the argument: finalizing greedily can lock in a value a later cheaper path would lower. With negative weights use Bellman–Ford (\(O(VE)\)), which also detects negative cycles.
Section 9Three Dialects — Idiom Cheat Sheet
The same intent, said three ways. Keep this beside you while translating an algorithm from one language to another.
| Task | Python | Perl | Octave |
|---|---|---|---|
| Length of a list | len(a) | scalar @a | numel(a) |
| First element | a[0] | $a[0] | a(1) % 1-based! |
| Last element | a[-1] | $a[-1] | a(end) |
| Slice first k | a[:k] | @a[0..$k-1] | a(1:k) |
| Append x | a.append(x) | push @a, $x | a(end+1) = x |
| Map f over a | [f(x) for x in a] | map { f($_) } @a | arrayfun(@f, a) |
| Keep where p(x) | [x for x in a if p(x)] | grep { p($_) } @a | a(p(a)) % logical index |
| Empty dict / hash | d = {} | my %h; | containers.Map() |
| Set / get by key | d[k] = v; d[k] | $h{$k} = $v; $h{$k} | m(k) = v; m(k) |
| Sort ascending | sorted(a) | sort { $a <=> $b } @a | sort(a) |
| Sort by key f | sorted(a, key=f) | sort { f($a) <=> f($b) } @a | [~,i]=sort(f(a)); a(i) |
| Join with commas | ','.join(xs) | join(',', @xs) | strjoin(xs, ',') |
| Integer range 1..n | range(1, n+1) | (1 .. $n) | 1:n |
| Read a line | input() | $line = <STDIN>; | line = fgetl(stdin); |
Python and Perl arrays are 0-based; Octave is 1-based. Every loop bound, slice, and midpoint shifts by one when you port between them — the single most common source of off-by-one bugs across these three languages.
Section 10Complexity Reference
A pocket card for the road.
| Operation | Time |
|---|---|
| Array index / hash lookup (expected) | O(1) |
| Binary search (sorted) | O(log n) |
| Linear scan | O(n) |
| Merge sort / heap sort | O(n log n) |
| Quick sort (average / worst) | O(n log n) / O(n²) |
| Insertion sort (worst) | O(n²) |
| BFS / DFS | O(V + E) |
| Dijkstra (binary heap) | O((V + E) log V) |
| 0/1 Knapsack (DP) | O(n·W) |
| Comparison-sort lower bound | Ω(n log n) |
1 · State the input size your bound is in terms of. 2 · Prove correctness with an invariant before optimizing. 3 · Write the recurrence, then reach for the Master Theorem. 4 · Mind the 0-based / 1-based boundary when porting. 5 · The best optimization is almost always a better asymptotic class, not a faster constant.