A Randomize Pixels With Mask test was performed on
resnet-50, in which
a 36% failure rate was observed.
In at least one case, the model's prediction changed 0.96. This caused the label to change from boa constrictor, Constrictor constrictor to bolo tie, bolo, bola tie, bola.
This test measures the robustness of the model to Randomize Pixels With Mask transformations. It does this by taking a sample input, randomizing pixels with fixed probability, and measuring the behavior of the model on the transformed input.
Production inputs can have unusual variations amongst many different dimensions, ranging from lighting changes to sensor errors to compression artifacts. It is important that the models are robust to the introduction of such variations.
This report was automatically generated by the scanning engine
rime-0.21.0rc4.post195+git.2a88076b.d on 2023-01-07 17:28.