Prefilter: drop before the slow checks
argus-curator 0.15.2.dev2+g97021d6 @ 97021d6PyPI (opens in new tab)Source (opens in new tab)Changelog (opens in new tab)
Imported from docs/prefilter.md on argus-curator's dev branch at that commit. Fixes go upstream, not here.
SingGuard takes seconds an image, and scenes and garments add their own pass.
When some images are of no use whatever the later checks say, the prefilter
drops them first, so those passes never run on them. Each rule is opt-in, on
the request as "prefilter": {...} and on the CLI as a flag:
The rules run once, right after identities (so a set age counts), on the
images that passed the hard filters. A dropped image stays in the results as
a reject with reject_reason prefilter:nsfw, prefilter:no-face or
prefilter:no-age (the first that applies, in that order); hard rejects keep
their own reason. A hard reject gets no face or age reading, so the under-18
rule (minor-exposed) could not speak for it, and it is not sent to NudeNet, EraX or SingGuard
either; min_short_side cannot go below 512 for the same reason. A dropped
image has been through faces, NudeNet and EraX, since the rules read them, but
not SingGuard. No rejected image, dropped or hard, is kept or exported, whatever
its score, and not when named in an export's selection; include_rejected is
ignored. Its keep reason (/scan/{id}/select, the CSV report) is its
reject_reason, e.g. prefilter:nsfw. A dropped image no longer counts
toward its identity: face_clusters[].size and the representative leave it
out, an identity with only dropped images is removed from face_clusters, and
its rows and set_age.cluster_id keep naming it. The pooled attributes and the
set age were read before the drop and still include it. A streaming scan sends
the dropped images in a prefilter chunk, with the recounted face_clusters.
A rule without the check it needs
(drop_nsfw with NudeNet off or no ARGUS_CURATOR_NUDENET_URL) is refused
with HTTP 400 (CLI exit 2). The summary records the rules as
prefilter_config, and /health reports prefilter.
# Faces with an age only, and no SingGuard pass on nudity NudeNet already saw
argus-curator scan /data/inbox --mivolo --drop-nsfw --drop-no-age --csv report.csv