Model & metrics

Everything below is measured on a held-out test set the model never saw during training — not estimated, not rounded up. The rule-based detector is reported alongside, on the same test set.

Random Forest — holdout test results

0.9081

Accuracy

0.9341

Precision

0.8718

Recall

0.9019

F1 score

0.9691

ROC-AUC

0.0579

False-positive rate

Confusion matrix (test set)	Predicted legitimate	Predicted phishing
Actually legitimate	1384	85
Actually phishing	177	1204

Test set: 2850 URLs (1381 phishing). The model is a Random Forest (250 trees, max depth 24), trained on 45 URL-only features.

Headline verdict (both detectors combined)

Headline score = 0.5 × rule score + 0.5 × model probability × 100. Thresholds (Low ≤ 20, Medium ≤ 30) were tuned on the validation set.

3-level accuracy (combined)	0.8558
High-verdict false-alarm rate (benign URLs called High)	0.0449
High-verdict recall (phishing URLs called High)	0.8392
Rule-only 3-level accuracy (same thresholds)	0.5228

The test truth has only two levels (phishing / benign), so “3-level accuracy” counts a Medium verdict as wrong. The High false-alarm rate and recall describe the headline verdict more fairly.

Most important features

Feature	Importance
`subdomain_total_length`	0.1276
`host_length`	0.0931
`entropy_host`	0.0801
`url_length`	0.0742
`tld_is_suspicious`	0.0633
`path_length`	0.0595
`digit_ratio_url`	0.0498
`longest_digit_run`	0.0473
`registered_domain_length`	0.0468
`entropy_path`	0.0395

Data & splits

Phishing URLs	9495	PhishTank online-valid.csv (verified=yes, online=yes)
Benign URLs	9789	Tranco top-1m.csv.zip (top 100k domains, full URLs synthesised with realistic paths) + 240 synthetic benign shortener URLs (de-biasing step)
Synthetic augmentation (training split only)	890	Seeded synthetic phishing URLs (scripts/augment.py) teaching the model the deep tricks that are rare in the real feeds: homoglyph/ASCII/near-miss brand mutations, '@'-trick decoys, file-extension TLDs, open-redirect laundering, invisible Unicode characters, and login pages on free hosting.
Collected		2026-11-09
Split (train / validation / test)	13531 / 2902 / 2850	registered domain (no domain leaks across splits)

Honest limitations

Benign URLs are synthesised from Tranco domain names, so their paths are simpler than real browsing; the model may partly key on URL length. The rule-based detector needs no training data and is reported alongside as a cross-check.
URL-only analysis cannot catch phishing hosted on legitimate or compromised domains.
PhishTank skews toward freshly reported, short-lived phishing pages; performance on older or targeted attacks may differ.
Most checkers that return only a verdict — including the top-ranked ones — explain nothing, and some reject bare domains outright. PhishLens accepts any URL shape and always shows its reasons; that is the gap it fills, not a claim of perfect detection.

Reproduce everything: python scripts/train.py downloads nothing new (datasets live in data/), rebuilds the model, and rewrites these numbers.