The aim is a shared test set of honest defects, a model that reports uncertainty, and a deployment that a night shift can live with.
If your demo only works on a white background, this is not your collaboration.
The shared test set is the substance of it. Every participant contributes labelled images of genuine rejects — including the marginal cases where two inspectors disagreed, which are the most valuable and the least commonly shared. In return each participant can benchmark any model, their own or a supplier's, against data collected in somebody else's plant.
That last point is the commercial argument. A model that performs well on your own data and poorly on a second site is a model that has learned your lighting. Establishing that before a procurement decision is worth considerably more than the effort of contributing.
How it runs
The mechanics, stated plainly.
Contribute, then benchmark
Participants contribute a labelled defect set from their own line. Access to the pooled benchmark follows contribution, not membership.
Monthly technical calls
An hour, online, on calibration, drift and labelling practice. Recorded for participants who were on shift.
Open method, closed data
Methods and results are published openly. The images stay within the group, under agreement, with faces and identifying marks removed.
What you provide
- A labelled set of genuine rejects, including the marginal and disputed cases
- Access for a site visit, so that lighting and workflow can be understood
- A named quality engineer who will define what counts as a defect
- Tolerance for a result showing that your current system underperforms
What you get
- Benchmarking of any model against defect data from multiple sites
- A calibration and labelling method you can apply to your own procurement
- An honest view of whether vision is the right answer for your bottleneck
- Introductions to groups who have deployed this without theatre
Terms
Agreed before anybody contributes anything, which is the only moment it works.
- Contributed imagery is anonymised and remains within the participant group
- Methods, metrics and aggregate results may be published openly
- No participant is identified in any publication without written consent
- Commercial suppliers may join only where an end-user participant sponsors them