You hold your phone up in a noisy room, record a few seconds, and the song appears. It feels like magic, but the core idea is surprisingly approachable. I built SoundFind, a local-first, Shazam-style web app, to identify songs from your own indexed catalogue. Here is how it works.
Step 1: turn sound into a picture
Audio is a waveform: pressure over time. To find patterns, we convert it into a spectrogram using a short-time Fourier transform. Time runs left to right, frequency bottom to top, and brightness shows how loud each frequency is at each moment.
Step 2: keep only the peaks
A spectrogram holds far too much detail, and most of it is fragile. Background noise, a bad microphone and room echo all change it. What survives is the set of strongest points: local peaks that stand out from their neighbourhood. Plot only those and you get a sparse "constellation map" of the song.
Step 3: hash pairs of peaks
A single peak is not distinctive. Pairs are. For each anchor peak, take a few peaks in a target zone just after it and build a hash from:
- the anchor's frequency,
- the paired peak's frequency,
- the time gap between them.
Store each hash with the song and the anchor's time. A song becomes thousands of small, robust fingerprints.
Step 4: match by time alignment
When you record 5 to 8 seconds, SoundFind computes the same hashes from the clip and looks them up in the catalogue. Plenty of hashes collide by chance. The trick is the timing: for the right song, many matches share the same offset between the clip's time and the song's time. Count matches per song per offset, and the true match stands out as a tall spike while random collisions stay scattered.
The system around the algorithm
The algorithm is half the work. The other half is a product people can use:
- Web app: Next.js 15 with TypeScript. The browser records the clip.
- Data: Supabase for auth, Postgres and storage, including each user's history.
- Matcher: a Python FastAPI service using librosa, NumPy and SciPy for the signal processing.
- Security: the Next.js route forwards requests and injects the admin API key server-side, so it never reaches the browser.
- Deployment: the web app on Vercel, the matcher in Docker on Hugging Face Spaces.
One lesson from building it: heavy audio processing does not belong inside short-lived serverless functions. Moving the matcher into its own service solved timeouts and kept the web app fast.
Tuning the details
A few parameters decide how robust and how fast the system is:
- Peak neighbourhood size: a larger neighbourhood keeps fewer, stronger peaks, which resists noise but stores less detail.
- Fan-out: how many target peaks each anchor pairs with. More pairs mean better recall and a bigger index.
- Target zone: how far ahead in time and frequency to look for partners.
- Hash precision: how finely frequencies and time gaps are quantised. Too fine and noise breaks matches; too coarse and unrelated songs collide.
Indexing the catalogue
Matching is only as good as the catalogue. Every track goes through the same ingest pipeline:
- Decode the file and convert it to mono at a fixed sample rate.
- Compute the spectrogram and find the peaks.
- Generate the peak-pair hashes with their anchor times.
- Store them in Postgres in batches, indexed by hash, so lookups stay fast as the catalogue grows.
In SoundFind, ingest runs through an admin route, so new tracks can be added without touching the code.
Limits worth knowing
Fingerprinting recognises a specific recording. It struggles with live versions, covers, heavy pitch or tempo changes and very short clips in loud rooms. That is also why it is fast and precise: it is not trying to understand the music, only to recognise this exact recording.
Why it works so well
Peaks survive noise, hashes make lookups fast, and time alignment turns thousands of weak clues into one confident answer. The same thinking applies well beyond music: find stable features, index them cleverly, and let consistency across many small matches do the deciding.
See the SoundFind case study or how I approach machine learning projects.
