cs161 Lecture 24: DHTs (Chord)

Very large system, impacts what we mean by scaling in the last lecture.
 Performance - Worry a lot abot *decreased* performance as we grow
  Two issues are confusing this:
   Bandwidth / Latency
   Per node / per system
 Managability - Decentralize to get constant scaling
 Availability - Need to handle constant join/leave (compare Porcupine)
   Load balancing is important, noone can handle even a small fraction of total

Node lookup is the only operation
 Don't assume content is with the key
 Applications?
  mirroring/cdn
  ditributed storage/backup
  distribued indexes
  file system (nice optimization if read-only?)
 Why is this "easy" on the Internet? (IP addresses have structure)
  
Consistent hashing (load balancing)
 draw picture
  each node gets K/N keys (k goes to lowest N >= k) 
  a join/leave moves K/N keys
 vs hashing to buckets
 virtual servers to avoid imbalance [was actually (1+O(log n))K/N]

Lookup: O(n) by checking successor

Lookup: O(log n) by checking finger tables

O(log n) messages, what about time?
 try to use nearby finger entries

Join: invariants to keep in mind
 1. Successor must be corrrect
 2. successor(k) must hold k
 3. Finger tables are correct (desired)

join(n): O(log^2 n)
 ask some node to find n's successor
 set n's pred, set n's successor's pred to n
 n's fingers (carefully to maintain O(log^2 n) (log(n) maybee < |fingers|) 
 n AS a finger
 Transfer keys (all keys are on the successor)

Stabilization
 Perfect joins are very hard with concurrent join/leaves
 Constantly ask you successor for its pred
 Change your successor if your successor's pred is closer to you

Availability
 Successor-list O(log n)?

Evaluation is mostly just math, except:
 dealing with failure: pretty irrelevant to reality
 RON data: too small to be worthwhile?

Attacks
 Attack data by being in charge of it (why hard?)

Speedup
 Hop count - d * fingers gives log_1+d(N)
 Care about latency, not hop count
   recursive could be faster (no roundtrips)
   k finger entries per interval
   use shorter finger if it's much closer

Looukp path length is actually 1/2 log n.