Programming Throwdown

Patrick Wheeler and Jason Gauci


Intro topic:  Buying a Car

News/Links:

Book of the Show


Tool of the Show
Topic: Vector Databases (~54 min)

  • How computers represent data traditionally
    • ASCII values
    • RGB values
  • How traditional compression works
    • Huffman encoding (tree structure)
    • Lossy example: Fourier Transform & store coefficients
  • How embeddings are computed
    • Pairwise (contrastive) methods
    • Forward models (self-supervised)
  • Similarity metrics
  • Approximate Nearest Neighbors (ANN)
  • Sub-Linear ANN
    • Clustering
    • Space Partitioning (e.g. K-D Trees)
  • What a vector database does
    • Perform nearest-neighbors with many different similarity metrics
    • Store the vectors and the data structures to support sub-linear ANN
    • Handle updates, deletes, rebalancing/reclustering, backups/restores
  • Examples
    • pgvector: a vector-database plugin for postgres
    • Weaviate, Pinecone 
    • Milvus