The DeepStack integration has been widely popular. The value of AI in surveillance is it improves accuracy to detect objects and reduces false alerts. AI will draw attention to activity that matters by identifying objects that could be missed by humans because of lack of attention or hard to see at night. AI also suppresses events that do not matter such as moving shadows from the wind, changes in sunlight, flies, birds, squirrels, rain etc. AI is becoming a very effective tool for outdoor cameras.
- ai-alerts_optimized.png (63.88 KiB) Viewed 38888 times
The above image shows the user experience across all the BI modalities: console, web interface and mobile interface. Consistent across all modalities are annotations on the alert image identifying objects found and icons in the alert list to easily identify alerts / cancelled alerts.
Annotation colors
- Orange: High level of confidence (>67%).
- Yellow: Confidence level <67%, but meets your threshold, e.g. 50%.
- Blue: Static object/Occupied state. See above for details.
- Red: Cancelled alert based on "To cancel" field in Camera settings.
- Green: Objects of no interest.
For example a boat appeared in the scene but boat is not listed as an object of interest.
Alerts list Icons:
Person
Car
Green check: AI confirmation, i.e. AI found an object, but no unique icon for object.
Flag: Same Flag function as before.
There is a user setting to automatically flag confirmed AI alerts.
Camera settings -> Trigger tab -> AI button -> Auto-flag confirmed alerts (Default is unselected)
Cancelled symbol: nothing found
Occupied State / Static Object (details below)
Static object
A static object is an object that is NOT moving. The most common example is not to keep triggering on a mailbox that the AI keeps misinterpreting as a person. BI comes with advanced alerting by overlapping motion with objects before sending an alert. This reduces the majority of false alerts due to static objects.
Static objects will only have a blue annotation. Occupied state will include the time symbol.
- occupied state.PNG (77.39 KiB) Viewed 38888 times
Occupied state
There still exists some stubborn scenes where Occupied State provides even more value. For example, a parked car in the driveway underneath a tree for shade could be challenging. The shadows from the tree could trigger motion and the motion boxes could very well overlap with the car causing alerts. Occupied state should prevent such alerts based on logic described below.
Furthermore a parked car is NOT a permanently static object like a mailbox. When the car leaves and comes back, BI should alert on the departure/return. The occupied state algorithm is trying to account for all these cases.
Occupied state has the timer icon AND uses the blue annotation.
Logic behind Occupied State
Same object type: +/- 10% confidence level
Same location: +/- 5%
Same object color (removed because many cameras go black&white, i.e. IR, at night. v5.5.1.17)
Occupied state list: BI maintains two lists during an alert.
- Current alert list: objects identified in current alert
- Static object list: A list of objects that it perceives to be static, e.g. a parked car, based off of past alerts.
When an alert occurs, each object(s) in the current alert list is first compared against all the objects in the static object list. If an object in the current alert matches any object in the static object list , then the object is considered static, e.g. parked car, and BI will NOT fire an alert based on this object. The static object icon will be displayed if all objects are determined to be static.
Identifying new objects
If an object on the current alert list does NOT match any object in the static object list an alert is fired. The new object will be placed in the static object list. This is how the software distinguishes between parked cars and cars that drove by. This is also how the static object list gets updated.
Removing static objects
Static objects continue to remain in the static object list if the object is matched. With every match, the timestamp on the object is updated.
If an alert arrives and the static object list has objects that were NOT matched by any of the current alert objects AND those static objects are older than 2 min, those objects are purged from the static object list. We know AI is not perfect which is why we created the 2 min window. A static object detected in one frame may not be detected in another frame due to lighting changes etc. In order to remove these inaccuracies, we assume as the camera alerts, at least one frame will identify the static object correctly within 2 min or earlier. FYI, engineering keeps tweaking the window. It started at 30s and moved to 2 min based on false alarms.
Static Objects is a setting that can be turned off if necessary (you want more sensitivity). This may be a good idea in high traffic areas like a camera facing a door or a car entry gate.
- static object.png (24.23 KiB) Viewed 38888 times
Camera settings -> Trigger tab -> Artificial Intelligence (default settings)
- deepstack user experience_default settings.png (69.37 KiB) Viewed 38888 times
Click on the Help button for details on each setting. Below is some insight based on tickets.
Custom models
Users often get confused how to enable/disable models on a per camera basis.
DeepStack has two default models: objects, faces
Suppose you created/added two custom models: dark, combined
Camera 1 only runs dark: Custom models = dark. This is an inclusion list since no :0 values and dark is the only specified model.
Camera 2 runs faces and objects.
List of Objects Available for detection
DeepStack has two default models: objects, faces
Suppose you created/added two custom models: dark, combined
Camera 1 only runs dark: Custom models = dark. This is an inclusion list since no :0 values and dark is the only specified model.
Camera 2 runs faces and objects.
Custom models = dark:0, combined:0. Exclusion list that runs all the models except dark and combined, i.e. only the default objects will be run.
Custom models = faces, objects. Inclusion list that only runs faces and objects, i.e. dark and combined will not be run.
Camera 3 runs faces, objects and combined: Custom model = dark:0. Run all the models except dark.Custom models = faces, objects. Inclusion list that only runs faces and objects, i.e. dark and combined will not be run.
Best to check deepstack.cc for current list of objects.
Code: Select all
person, bicycle, car, motorcycle, airplane,
bus, train, truck, boat, traffic light, fire hydrant, stop_sign,
parking meter, bench, bird, cat, dog, horse, sheep, cow, elephant,
bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase,
frisbee, skis, snowboard, sports ball, kite, baseball bat, baseball glove,
skateboard, surfboard, tennis racket, bottle, wine glass, cup, fork,
knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot,
hot dog, pizza, donut, cake, chair, couch, potted plant, bed, dining table,
toilet, tv, laptop, mouse, remote, keyboard, cell phone, microwave,
oven, toaster, sink, refrigerator, book, clock, vase, scissors, teddy bear,
hair dryer, toothbrush.Motion triggers vs Alerts
With Artificial Intelligence there is now a wider gap between Motion Triggers, e.g. the camera triggered because a deer walked across the lawn, and an Alert, i.e. the AI correctly did not recognize the deer as a person, and therefore cancelled the alert.
The AI setting "Hide cancelled alerts on timeline and 'all alerts'" is on by default, i.e. most users after setting up their Motion and AI, do not want to be bothered by the "nothing found" alerts, because that is why they installed AI in the first place.
However, I do see users in the country side that like to record wildlife such as bears, coyotes, deer etc when they approach their property. BI AI will cancel the Alert, however motion triggers are still always recorded. This implementation was chosen just in case the AI missed something, you still had the motion trigger recording. It turns out, this implementation provides added benefit for the users in the country side that still like to capture recordings of wildlife. The "Hide cancelled alerts on timeline and 'all alerts'" can be unselected if you want "nothing found" alerts still listed in the Alerts folder. Furthermore, cancelled alerts are always listed in the cancelled alerts folder regardless of the "Hide cancelled alerts on timeline and 'all alerts'" setting.