← All posts · Michael Baffour Awuah

open source · MIT · reproducible by design

Find the homologues
the standard search
walks right past.

A profile-HMM discovery pipeline that reaches the twilight zone, searches genomes in all six frames, validates every hit as a real ORF, and iterates to convergence — from a single command, with the same answer every time.

one command in six-frame search golden-file tests locked env
# run_manifest.json — the shape of one run
commands
1
from seed FASTA to a finished, labelled package
databases
11
phage / viral sources in the streamable registry
reading frames
6
translated per genome, so nothing hides out-of-frame
evidence cols
37
per hit in hits.tsv — coordinates to sequence
stdout · capabilities

The whole discovery loop, compiled into one command

Build, calibrate, search, validate, iterate, characterise, package — every step automated and reproducible, backed by tests so a code change can't quietly change the biology.

> profile alignment · conserved columns weighted heavily, floppy loops loosely ~18% identity · twilight zone
seed·terminase LDSKVGTIDRLNEAHG
hit·NC_048… IDTRLGSVDKM-EARG
hit·frame·−2 VDSKIGNLDRFSE-HG
hit·antisense LDARVGTMDKLTEAKG
conserved / catalytic hydrophobic polar basic gap
one command

Seed FASTA in, package out

Give it seeds and it does the rest — builds the HMM, picks and searches databases, validates hits, iterates, and assembles a labelled output folder. On first run it installs the tools it needs via conda.

sensitivity

Profile HMMs reach the twilight zone

Position-specific scoring weights a conserved catalytic residue heavily and a floppy loop loosely, detecting homology down to ~15–25% identity — the remote range where pairwise BLAST quietly fails.

six-frame

Finds the unfindable

Genome databases are translated in all six frames, so homologues encoded antisense or out-of-frame to predicted genes — the ones standard annotation misses entirely — become searchable. Some proteins surface only this way.

orf validation

Proves they're real genes

Each hit is reconstructed frame-correctly from genomic coordinates, required to carry no internal stop codons, and checked to sit in a real coding locus. An HMM score alone is never enough — the evidence lands in the table.

convergence

Iterate until it stops

Deduplicate, re-seed the model, search again — and stop early, automatically, when the hit set and the model stabilise. The pipeline records the round-by-round counts and the reason it stopped.

controls · roc

Quantify the false-positive rate

Composition-matched shuffled seeds and unrelated proteomes calibrate the bit-score threshold, giving sensitivity, specificity, FPR and an ROC AUC — so "it's just composition bias" gets a number, not a shrug.

regression

Golden-file tests

A stored baseline snapshots the biological invariants that must stay stable for a fixed seed and pinned databases — representative accessions, confidence tiers, bit scores, ORF invariants, tree topology — so a refactor can't silently move the answer.

pinned env

Locked environment

An environment.lock.yml pins every tool — HMMER 3.4, MAFFT, trimAl, Prodigal, CD-HIT, IQ-TREE, MEME/FIMO, clinker — and each run also records the live versions it actually used in its manifest.

evidence

Tables, not just sequences

A 37-column hits.tsv gives organism, genomic coordinates, ORF-validation metrics, HMM statistics, and both nucleotide and amino-acid sequence per hit — plus a deduplicated "found in N databases" provenance column.

figures

Alignment, tree & synteny

A MAFFT L-INS-i alignment, an IQ-TREE ML tree with the seeds placed and marked, and per-cluster gene-neighbourhood figures — all written as editable SVG and PDF alongside a 300-dpi PNG.

overprinting

Demonstrated, not suspected

The opt-in interrupted-homolog search reads through premature stops, then tests whether each stop is synonymous in an open antisense frame — the signature of overprinting — reporting a strong / partial / none verdict.

scan mode

Does this genome carry it?

Scan one genome — a local FASTA or an NCBI accession, fetched for you — and get present / absent, coordinates, ORF validation, the flanking genes, and a genome-map figure with your gene in bold gold.

package

Provenance-rich output

Every run assembles a labelled PACKAGE/ with a run_manifest.json, a written METHODS.md, GFF3 browser tracks, named GenBank neighbourhoods, the profile .hmm, and an HTML summary.

$ cat WHY_THIS_EXISTS.md
the story

Hidden in plain sight

// My lab once found a gene that had been sitting in plain sight the whole time. It wasn't buried in some exotic organism nobody had sequenced. It was right there, in genomes people had already looked at, unremarked — because the standard searches simply hadn't looked hard enough to see it. Once you know a thing like that is possible, you can't un-know it. You start wondering how many other interesting gene families are out there, quietly overlooked, one better search away from being obvious.

That feeling is what this tool is built around. Not "run BLAST and move on," but: what would it take to reliably find the homologues everyone else misses, and to do it in a way I could actually stand behind six months later?

Because the honest problem isn't that the tools are bad. HMMER, MAFFT, Prodigal, IQ-TREE, clinker — these are excellent, battle-tested pieces of software. The problem is the seam between them. A remote-homologue search is a real workflow: build a profile, calibrate against controls, search a set of databases in six frames, reconstruct and validate every hit as a genuine ORF, deduplicate, re-seed, iterate to convergence. Stitch that together by hand and by the third round the pipeline lives in your shell history and your short-term memory — and both are lossy.

// the interesting stuff was hidden because the search stopped one frame too early.

This matters most in phage biology, where so much of the interesting stuff is genuinely hidden. Some of the proteins I care about are encoded antisense or out-of-frame to a predicted gene, so a search over annotated proteins never sees them at all. You have to translate the raw genome in all six frames to reach them. Do that by hand across a dozen databases and several iterations and the bookkeeping — not the biology — becomes the hard part. And bookkeeping is exactly where reproducibility goes to die.

So HMM Homologue Finder packages the whole loop into one command. You hand it a seed FASTA. It builds the HMM, calibrates the threshold against positive and negative controls, searches the catalog in six frames, reconstructs and validates each hit's ORF, deduplicates, re-seeds, and repeats — stopping automatically when the hit set and model stabilise. Then it characterises what it found: alignment, tree, synteny figures, GFF3 tracks, a written methods file, and a full manifest.

Two decisions mattered most. First, it gives the same answer every time for a fixed seed and pinned databases — the environment is locked, the stochastic steps are seeded, and a golden-file test snapshots the biological invariants so a refactor can't silently move the result. Second, a hit has to earn its place: reconstructed frame-correctly, no internal stops, sitting in a real coding locus, with an ROC curve to answer "it's just composition bias" with a number rather than a shrug. A validated hit is still only a candidate — function needs the bench — but it's one you can defend.

One honest note: I'm a scientist who got good at building tools with AI, not a career software engineer. I built this with Claude — the science and the decisions about what to validate are mine; the AI just made it real faster than I could alone. And building it forced me to understand the problem more deeply than using tools ever did. You cannot fudge in code what it means for a six-frame hit to be a "real" gene. The test suite makes you say exactly what you mean.

Michael Baffour Awuah  ·  $ you can just build things
getting started

Clone it, set it up once, point it at your seeds

The stages run in order — each only makes sense after the last. Full instructions live in docs/INSTALL.md and docs/USAGE.md.

01

Clone the repository

The search engine is bundled in engine/, so the repo is self-contained — nothing else to download to make it run.

git clone … && cd hmm-homologue-finder
02

Set up once

Creates the hmm-discovery conda environment and installs every tool — HMMER, MAFFT, Prodigal, IQ-TREE and the rest — at pinned versions. One time only.

bash setup.sh
03

Smoke-test the install

Run the example seeds to confirm everything is wired up before a real, database-downloading run. Fast, offline, and a good first-machine check.

bash run.sh --smoke
04

Preload the databases

Optional. Databases stream, get six-frame-translated, and cache — so later searches spend their time searching, not downloading.

python preload_databases.py
05

Run discovery

Hand it your seed FASTA. It builds the HMM, searches the databases you pick, validates ORFs, and iterates to convergence — no babysitting.

bash run.sh --seeds my_seeds.fasta --iterate
06

Open the package

Everything lands in a labelled PACKAGE/: evidence tables, alignment, tree, synteny figures, GenBank files, the profile .hmm, a written METHODS.md, and an HTML results summary.

runs on macOS Linux Windows · WSL2 — the bioinformatics tools ship for macOS/Linux, so Windows goes through WSL2.
$ bash run.sh --seeds your_family.fasta --iterate

Go find the ones
everyone else missed.

Free, open source, and reproducible. Clone it, point it at a seed FASTA, and get the same answer every time.