The situation
Standard deep networks fail when objects are partly hidden, and adding synthetic occlusion during training only goes so far.
CompNet: classification that holds up under occlusion
A generative compositional model inside deep convolutional networks that classifies objects from their visible parts, even when much of them is hidden.
Occlusion-robust image classification
01 · Context
02 · Challenge
What had to be true
Keep classification accurate at high levels of occlusion by focusing on the parts of the object that can actually be seen.
03 · What we built
The system
- A generative compositional model integrated into deep convolutional neural networks (DCNNs)
- A fully generative approach that models feature distributions with and without occlusion, instead of occlusion augmentation
- Occluder localisation by estimating the likelihood of occlusion at each image location
- Classification driven by the non-occluded parts of the object
04 Architecture
How it works, step by step.
Step through the system, or let it play.
- An image arrives
- DCNN extracts features
- Compositional model scores parts
- Occluders are localised
- Classified from visible parts
05 · Outcome
Validated on Occluded-COCO-Vehicles (MS-COCO) and PASCAL3D+: 99% accuracy with no occlusion and 91% with 60–80% of the object hidden, well ahead of standard deep networks.
06 · Stack
Next case · Construction · Computer visionCustom vehicle detection at construction sites