Smart Video 101

The most basic version of camera-based analytics is video motion detection (VMD). Motion detection works by analyzing changes between pixels from one frame to the next, triggering an alert whenever the difference exceeds the sensitivity threshold.

Strengths:

• Low-cost, does not require additional computing power provided by an on-premise device or a cloud-based service.
• Can be effective when deployed in the right environment; for example, indoor environments outside of business hours.
• Can be layered to reduce bandwidth and compute consumption on AI-based analytics.

Weaknesses:

• Highly prone to false positives in most deployment environments due to possible sensitivity to swaying vegetation, animals, and even changing shadows and lighting.
• Can overwhelm a monitoring operation if set up in areas that are prone to false positives.
• Can lead to missed alerts if motion sensitivity is set too low (in an effort to reduce false positives).

Takeaway: In most situations, video motion detection alone is insufficient in setting up an effective AI video surveillance system. However, when layered as a pre-processing layer for AI-based analytics, VMD can significantly improve cost and bandwidth efficiencies by filtering out frames that do not need to be processed.

Instead of analyzing changes in pixels between frames, basic AI analytics analyzes each frame independently and looks for specific objects that a deep learning algorithm can identify. Compared with VMD, basic AI analytics offer an additional layer of nuance, for example: “that is a human” or “that is a vehicle.”

Strengths:

• Adds an additional degree of nuance over VMD, which can reduce the number of false positives processed by security staff.

Weaknesses:

• Basic analytics cannot determine the behavioral context of object interactions. For example, basic analytics can identify human beings and ignore squirrels, but it can’t filter any further (i.e., “there are a group of three people loitering in the stairwell” or “a person is carrying an AR-15 on the East Wing of the building”). These capabilities require more advanced models of analytics.

Advanced analytics takes the concept of “object detection” a step further with its ability to detect small objects in grainy security camera feeds and draw conclusions based on how objects in surveillance feeds interact.

Examples of advanced analytics may include:

• Gun Detection
• Mask Detection
• Social Distancing Detection
• Loitering Detection
• Crowd Detection
• Slip/Fall Detection
• Fight Detection

Takeaway: With advanced analytics, security professionals can dramatically improve “incident-driven response”—all the way down to the camera level. For example, a Director of Security for a housing development may want to set up alerts for loitering incidents in the stairwells, gun detection on all indoor cameras covering hallways and lobbies, and human detection on perimeter cameras outside of business hours. In instances such as these, advanced analytics can configure each camera to support each security operations objective.

Facial recognition is a form of advanced analytics that first detects a face, then attempts to match that face to a database of other faces. Facial recognition requires a much higher resolution (pixels-per-foot) than other advanced analytics.

As a result, facial recognition often requires specialized hardware and can be exceptionally expensive—a leading provider of facial recognition technology charges $800-$1,200 per camera a month, excluding the hardware price.

Additional use cases for this type of analytics is limited and may include:

• Automated access control
• Enforcing a list of people that must be denied entry
• Interdiction of repeat shoplifters

One thing to note is that the deployment of facial recognition has become increasingly controversial and contested across the country. In September of 2020, the City of Portland passed a ban on facial recognition. What’s more, a school district in Upstate New York was sued by parents after signing a $2.5 million facial recognition contract, forcing the district to stop using the technology after Governor Cuomo issued a moratorium.

Takeaway: With concerns about civil liberties and privacy rising, any decision to adopt facial recognition analytics should be deliberated thoroughly.

NVR systems are similar to DVRs but are deployed with digital cameras. Instead of connecting to the NVR via coaxial cables, cameras are connected via ethernet cable or wirelessly. Like DVRs, NVR appliances are designed for smaller deployments–usually up to 32 channels.

Analytics can be deployed with NVRs, but usually require the analytics provider to build an API integration with the NVR manufacturer to enable streaming of camera feeds to either an on-premise or cloud-based GPU server.

In cases where integration cannot be built, an analytics provider can pull a secondary stream directly from individual cameras into an analytics server. Still, the alerts will have to be delivered separately instead of through the NVR’s user interface.

A VMS is enterprise software typically installed on a local desktop computer or server that enables the customization and management of surveillance camera feeds. A VMS’s functionality is similar to an NVR’s but allows for additional flexibility since it’s not restricted to a fixed appliance.

The Benefits Of Adopting a VMS Over an NVR Appliance:

• There are no upfront hardware costs—as long as you have a desktop computer that can host the VMS or run the VMS on a virtual machine.
• A VMS has the flexibility to support more channels by adding additional network switches.
• VMS software’s growing popularity over NVR appliances means that you would be more likely to receive continued updates and technical support.
• Analytics providers will be more likely to have API integrations with the major VMS brands than NVR devices. These integrations can enable reduced bandwidth, compute consumption, and the ability to pull video through a single server instead of dozens, hundreds, or even thousands of individual cameras.

The Potential Cons of Adopting a VMS: A VMS typically requires more technical and networking expertise to install, which may not be a problem for larger organizations, but could present a layer of additional complexity for smaller ones.

Takeaway: If analytics support is a significant contributing factor to the decision between a VMS or an NVR appliance, we recommend that you opt for installing a VMS. As VMS continues to take market share, NVRs are gradually losing ground and will likely follow the path of analog cameras in near-obsolescence in the future.

Image compression is an excellent way for analytics solutions to minimize the amount of bandwidth required. Modern compression methods, such as *H.264 and *H.265, dramatically lowers the bitrate needed for video transmission by sampling several partial frames (p-frames) for every complete frame (i-frames) sampled.

*Most cameras / VMSs manufactured within the past decade will support H.264, if not H.265.

Most video management systems and certain cameras provide Video Motion Detection (VMD) capabilities, meaning that they can trigger an event alarm whenever the camera detects motion. As mentioned in The Different Types of Video Analytics Explained, VMD is not a practical standalone solution in most situations due to the high frequency of false positives.

Still, it’s important to note that cameras (and VMSs) can be configured to send frames to an analytics server upon the detection of motion, reducing bandwidth consumption by 80% – 90% in certain cases.

Limiting frame sampling is another technique that analytics providers often leverage to minimize bandwidth consumption. Top-performing analytics models don’t need 10 or 20 frames per second (fps) to deliver accurate results. For most analytics solutions, sampling at 1 fps is all that is necessary to trigger accurate detections.

However, some forms of advanced analytics, such as mask and gun detection, will require up to 3 fps to minimize the probability of a missed detection on a small object that may only appear briefly in a surveillance camera feed.

An analytics provider with image compression, motion pre-processing, and limited frame sampling can deliver high-performing analytics at an expected bitrate (somewhere between 20-100 kbps, per camera)—without impacting detection accuracy.

Strengths

• Lower monthly recurring cost, since computing costs are paid for with upfront investment.
• Camera networks can be kept entirely on the LAN, reducing potential cybersecurity risks.

Weaknesses

• High upfront costs for hardware (whether it’s GPU servers or smart cameras)
• Cost inefficient, as it requires upfront investment in hardware that will often sit idle as much as 70-80% of the time.
• More difficult to scale—as doing so will often involve additional hardware installations. Patches and software improvements to the AI algorithms are more difficult to implement.
• Some smart camera brands are completely incompatible with legacy systems (e.g. Verkada) and permanently lock you into their service for the lifecycle of the hardware.

Strengths

• Works on legacy camera infrastructure—no additional hardware costs required.
• Maximizes cost-efficiencies, as cloud service providers can auto-scale pooled resources according to consumption, allowing analytics providers to pass the cost savings onto the customer.
• Easy to scale—without hardware implementation required, a deployment of thousands of cameras can take just a couple of weeks instead of months.
• Patches and model improvements are instantaneously implemented on a regular basis with the stroke of a key, ensuring that the AI models deployed are always the best available technology.
• Compatible with most legacy hardware deployments

Weaknesses

• While a VPN connection is extremely secure, it will never be as secure as an on-premise deployment that is kept completely off of the internet.
• Certain sites with significant bandwidth limitations (e.g. retail outlets on T1 connections) will not be able to support more than a handful of cameras.
• While latency is rarely more than 1-2 seconds, alerts will not come in as fast as on-premise deployments.

Takeaway: Ultimately, compared with on-premise deployments, cloud-based analytics deployments are much more cost-friendly, easier to update, and more scalable. Unless your organization is willing to devote significant resources for a camera infrastructure overhaul, cloud-based deployment is the better solution in most cases.