A Contrast Decrease test was performed on resnet-18
, in which
a 19% failure rate was observed.
In at least one case, the model's prediction changed 0.62. This caused the label to change from ballpoint, ballpoint pen, ballpen, Biro to loupe, jeweler's loupe.
This test measures the robustness of the model to Contrast Decrease transformations. It does this by taking a sample input, decrease image contrast, and measuring the behavior of the model on the transformed input.
Production inputs can have unusual variations amongst many different dimensions, ranging from lighting changes to sensor errors to compression artifacts. It is important that the models are robust to the introduction of such variations.
resnet-18
pkg:huggingface/microsoft/resnet-18@2f536bd335677c6b111b3d103af458ef57a6145e
This report was automatically generated by the scanning engine rime-0.21.0rc4.post195+git.2a88076b.d
on 2023-01-07 14:16.