Benchmark¶

MVTec AD Dataset ¶

Model		Avg	Carpet	Grid	Leather	Tile	Wood	Bottle	Cable	Capsule	Hazelnut	Metal Nut	Pill	Screw	Toothbrush	Transistor	Zipper
STFPM	ResNet-18	0.893	0.954	0.982	0.989	0.949	0.961	0.979	0.838	0.759	0.999	0.956	0.705	0.835	0.997	0.853	0.645
STFPM	Wide ResNet-50	0.876	0.957	0.977	0.981	0.976	0.939	0.987	0.878	0.732	0.995	0.973	0.652	0.825	0.5	0.875	0.899
PatchCore	ResNet-18	0.819	0.947	0.722	0.997	0.982	0.988	0.972	0.810	0.586	0.981	0.631	0.780	0.482	0.827	0.733	0.844
PatchCore	Wide ResNet-50	0.877	0.981	0.842	1.0	0.991	0.991	0.985	0.868	0.763	0.988	0.914	0.769	0.427	0.806	0.878	0.958
PaDiM	ResNet-18	0.891	0.945	0.857	0.982	0.950	0.976	0.994	0.844	0.901	0.750	0.961	0.863	0.759	0.889	0.920	0.780
PaDiM	Wide ResNet-50	0.950	0.995	0.942	1.0	0.974	0.993	0.999	0.878	0.927	0.964	0.989	0.939	0.845	0.942	0.976	0.882
DFM	ResNet-18	0.894	0.864	0.558	0.945	0.984	0.946	0.994	0.913	0.871	0.979	0.941	0.838	0.761	0.95	0.911	0.949
DFM	Wide ResNet-50	0.891	0.978	0.540	0.979	0.977	0.974	0.990	0.891	0.931	0.947	0.839	0.809	0.700	0.911	0.915	0.981
DFKDE	ResNet-18	0.762	0.646	0.577	0.669	0.965	0.863	0.951	0.751	0.698	0.806	0.729	0.607	0.694	0.767	0.839	0.866
DFKDE	Wide ResNet-50	0.774	0.708	0.422	0.905	0.959	0.903	0.936	0.746	0.853	0.736	0.687	0.749	0.574	0.697	0.843	0.892

Model		Avg	Carpet	Grid	Leather	Tile	Wood	Bottle	Cable	Capsule	Hazelnut	Metal Nut	Pill	Screw	Toothbrush	Transistor	Zipper
STFPM	ResNet-18	0.951	0.986	0.988	0.991	0.946	0.949	0.971	0.898	0.962	0.981	0.942	0.878	0.983	0.983	0.838	0.972
STFPM	Wide ResNet-50	0.903	0.987	0.989	0.980	0.966	0.956	0.966	0.913	0.956	0.974	0.961	0.946	0.988	0.178	0.807	0.980
PatchCore	ResNet-18	0.935	0.979	0.843	0.989	0.934	0.925	0.956	0.923	0.942	0.967	0.913	0.931	0.924	0.958	0.881	0.954
PatchCore	Wide ResNet-50	0.955	0.988	0.903	0.990	0.957	0.936	0.972	0.950	0.968	0.974	0.960	0.948	0.917	0.969	0.913	0.976
PaDiM	ResNet-18	0.968	0.984	0.918	0.994	0.934	0.947	0.983	0.965	0.984	0.978	0.970	0.957	0.978	0.988	0.968	0.979
PaDiM	Wide ResNet-50	0.979	0.991	0.970	0.993	0.955	0.957	0.985	0.970	0.988	0.985	0.982	0.966	0.988	0.991	0.976	0.986

Model		Avg	Carpet	Grid	Leather	Tile	Wood	Bottle	Cable	Capsule	Hazelnut	Metal Nut	Pill	Screw	Toothbrush	Transistor	Zipper
STFPM	ResNet-18	0.932	0.961	0.982	0.989	0.930	0.951	0.984	0.819	0.918	0.993	0.973	0.918	0.887	0.984	0.790	0.908
STFPM	Wide ResNet-50	0.926	0.973	0.973	0.974	0.965	0.929	0.976	0.853	0.920	0.972	0.974	0.922	0.884	0.833	0.815	0.931
PatchCore	ResNet-18	0.896	0.933	0.857	0.995	0.964	0.983	0.959	0.790	0.908	0.964	0.903	0.916	0.853	0.866	0.653	0.898
PatchCore	Wide ResNet-50	0.923	0.961	0.875	1.0	0.989	0.975	0.984	0.832	0.908	0.972	0.920	0.922	0.853	0.862	0.842	0.953
PaDiM	ResNet-18	0.916	0.930	0.893	0.984	0.934	0.952	0.976	0.858	0.960	0.836	0.974	0.932	0.879	0.923	0.796	0.915
PaDiM	Wide ResNet-50	0.951	0.989	0.930	1.0	0.960	0.983	0.992	0.856	0.982	0.937	0.978	0.946	0.895	0.952	0.914	0.947
DFM	ResNet-18	0.919	0.895	0.844	0.926	0.971	0.948	0.977	0.874	0.935	0.957	0.958	0.921	0.874	0.933	0.833	0.943
DFM	Wide ResNet-50	0.918	0.960	0.844	0.990	0.970	0.959	0.976	0.848	0.944	0.913	0.912	0.919	0.859	0.893	0.815	0.961
DFKDE	ResNet-18	0.872	0.864	0.844	0.854	0.960	0.898	0.942	0.793	0.908	0.827	0.894	0.916	0.859	0.853	0.756	0.916
DFKDE	Wide ResNet-50	0.875	0.907	0.844	0.905	0.945	0.914	0.946	0.790	0.914	0.817	0.894	0.922	0.855	0.845	0.722	0.910