cs161 Lecture 24: DHTs (Chord)
Very large system, impacts what we mean by scaling in the last lecture.
Performance - Worry a lot abot *decreased* performance as we grow
Two issues are confusing this:
Bandwidth / Latency
Per node / per system
Managability - Decentralize to get constant scaling
Availability - Need to handle constant join/leave (compare Porcupine)
Load balancing is important, noone can handle even a small fraction of total
Node lookup is the only operation
Don't assume content is with the key
Applications?
mirroring/cdn
ditributed storage/backup
distribued indexes
file system (nice optimization if read-only?)
Why is this "easy" on the Internet? (IP addresses have structure)
Consistent hashing (load balancing)
draw picture
each node gets K/N keys (k goes to lowest N >= k)
a join/leave moves K/N keys
vs hashing to buckets
virtual servers to avoid imbalance [was actually (1+O(log n))K/N]
Lookup: O(n) by checking successor
Lookup: O(log n) by checking finger tables
O(log n) messages, what about time?
try to use nearby finger entries
Join: invariants to keep in mind
1. Successor must be corrrect
2. successor(k) must hold k
3. Finger tables are correct (desired)
join(n): O(log^2 n)
ask some node to find n's successor
set n's pred, set n's successor's pred to n
n's fingers (carefully to maintain O(log^2 n) (log(n) maybee < |fingers|)
n AS a finger
Transfer keys (all keys are on the successor)
Stabilization
Perfect joins are very hard with concurrent join/leaves
Constantly ask you successor for its pred
Change your successor if your successor's pred is closer to you
Availability
Successor-list O(log n)?
Evaluation is mostly just math, except:
dealing with failure: pretty irrelevant to reality
RON data: too small to be worthwhile?
Attacks
Attack data by being in charge of it (why hard?)
Speedup
Hop count - d * fingers gives log_1+d(N)
Care about latency, not hop count
recursive could be faster (no roundtrips)
k finger entries per interval
use shorter finger if it's much closer
Looukp path length is actually 1/2 log n.