Keep an Eye on Your Stuff with Frigate NVR, an AI Enabled Video Recorder

436

2023-02-21 | By Nate_Larson

License: See Original Project

Cameras are everywhere nowadays; many people have security cameras in and around their homes, ‎but many of these will record anytime they see motion. While this ensures that any activity is ‎recorded, reviewing these recordings can be tedious. You may have to sort through hours of trees ‎swaying in the wind or cars passing by before you find the footage you are looking for.‎

Frigate NVR is an open-source software that intercepts standard IP camera RTSP feeds, processes ‎them using AI to classify objects within frame based on preferences you define, and creates events ‎based on these detections. This results in recordings that are easier to review, and Frigate can even ‎provide notifications specific to the event type when used with Home Assistant. The list of object ‎models included with Frigate continues to expand as more objects are trained, but the current object ‎list can be found here: https://docs.frigate.video/configuration/objects.‎

This project will walk through the process of installing Frigate NVR in a Docker container running on a ‎LattePanda 3 Delta single board computer (SBC), but the process should be similar for other Linux-‎compatible boards and PCs. For my project, I am also making use of a Google Coral USB Accelerator for ‎faster AI detection. The inclusion of a Google Coral TPU device is optional but recommended.‎

One big advantage to Frigate NVR is that everything is local. All recordings and snapshots are stored on ‎your machine, and the AI is processed locally as well. There is no cloud required for operation, and ‎once set up, the server can run without any internet connection, making this build ideal for those who ‎are concerned about privacy.‎

coral_1

At this time, Frigate doesn’t have the ability to limit recordings based on available disk space. In order ‎to avoid the risk of accidentally overwriting important system files, I have included a second drive that ‎will be used specifically for Frigate’s video recordings.‎

Install Linux

Linux Mint is just one of many Linux distributions available, but this is the operating system I chose for ‎this project since it is easy to install and has all the drivers we need for the LattePanda board included. ‎If you choose a different distro of Linux to install, the Frigate installation should be similar, provided ‎the OS you select is Debian-based. Even within Linux Mint, there are different versions available, ‎which you can find and download here (https://linuxmint.com/download.php), but I went with Xfce ‎Edition since it uses fewer resources, leaving more processing power for the Frigate NVR. ‎

system_2

Flash this disk image to a blank 8GB or larger USB flash drive using Balena Etcher.‎

flash_3

Prepare the LattePanda board for use by attaching the included antennas, installing the standoffs to ‎elevate the board from the work surface, connecting a monitor, and connecting a USB keyboard and ‎mouse. Plug the USB drive that was flashed with Linux into an available USB port and then connect the ‎power supply.‎

prepare_4

Power on the LattePanda board and press the F7 key once when the LattePanda logo appears, then ‎select your USB installation media in the popup window. The Linux Mint desktop will load, but this is ‎just a live preview version of the operating system. On the desktop, you should find an icon to launch ‎the Linux Mint installation process.‎

install_5

You can find more information about the installation process here: https://linuxmint-installation-‎guide.readthedocs.io/en/latest/, but generally, you should be able to just follow the onscreen ‎prompts for a standard installation.‎

Once installation is complete, remove the USB flash drive and reboot the computer. If you have a ‎Google Coral USB Accelerator and/or external hard drive for media, you connect these to the ‎Lattepanda3 Delta now, then login and continue to the next step.‎

Installing Docker

From the desktop, open the Xfce Terminal application.‎

application_6

Update the repository list by entering the following command in the terminal

Copy Code

sudo apt update

Once your repositories are successfully updated, install Docker and all its dependencies:

Copy Code

sudo apt install docker*‎

Then verify the installation was successful:

Copy Code

docker -v

verify_7

Now, verify that Docker runs successfully by running a “hello world” container

Copy Code

sudo docker run hello-world

running_8

Once it is determined that Docker is installed and running, we can begin installing Frigate NVR.

Create Frigate Container

With Docker ready, we can create a folder for the Frigate container file.‎

Copy Code

mkdir Documents/frigate
cd Documents/frigate
nano frigate.yml

Add the following lines to your frigate.yml file and make any changes you may need for your particular ‎setup. Keep in mind that indentation matters for YAML files, so be mindful of the layout. You can find ‎details on setting up your Frigate Docker container here:

‎https://docs.frigate.video/frigate/installation/#docker ‎

Copy Code

version: "3.9"‎
services:‎
‎  frigate:‎
‎    container_name: frigate
‎    privileged: true # this may not be necessary for all setups
‎    restart: unless-stopped
‎    image: blakeblackshear/frigate:stable
‎    shm_size: "64mb" # update for your cameras based on calculation above
‎    devices:‎
‎      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions‎
‎      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here ‎https://coral.ai/docs/m2/get-started/#2a-on-linux
‎      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware‎
‎    volumes:‎
‎      - /etc/localtime:/etc/localtime:ro‎
‎      - /path/to/your/config.yml:/config/config.yml:ro
‎      - /path/to/your/storage:/media/frigate‎
‎      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
‎        target: /tmp/cache
‎        tmpfs:‎
‎          size: 1000000000‎
‎    ports:‎
‎      - "5000:5000"‎
‎      - "1935:1935" # RTMP feeds‎
‎    environment:‎
‎      FRIGATE_RTSP_PASSWORD: "password"‎

Below is my updated frigate.yml file for reference. ‎

Copy Code

version: "3.7"‎
services:‎
‎  frigate:‎
‎    container_name: frigate
‎    privileged: true # this may not be necessary for all setups
‎    restart: unless-stopped
‎    image: blakeblackshear/frigate:stable-amd64‎
‎    shm_size: "64mb" # update for your cameras based on calculation above
‎    devices:‎
‎      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions‎
‎      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here ‎https://coral.ai/docs/m2/get-started/#2a-on-linux
‎      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware‎
‎    volumes:‎
‎      - /etc/localtime:/etc/localtime:ro‎
‎      - /home/nate/Documents/frigate/config/config.yml:/config/config.yml:ro‎
‎      - /media/nate/Media:/media/frigate‎
‎      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
‎        target: /tmp/cache
‎        tmpfs:‎
‎          size: 1000000000‎
‎    ports:‎
‎      - "5000:5000"‎
‎      - "1935:1935" # RTMP feeds‎
‎    environment:‎
‎      FRIGATE_RTSP_PASSWORD: YourPasswordHere

Once your changes are complete, press “CTRL X” followed by “Y” and “Enter” to save the file.‎

Configure Frigate

The last thing we need to do before starting Frigate is to create and set up our Frigate configuration ‎file. ‎

Copy Code

mkdir config
cd config
nano config.yml

Your config file will tell Frigate where to find your camera feeds, what types of events to watch for, ‎how long to store files, and everything else Frigate will need to function. You can find a guide on how ‎to create a minimal configuration file here: https://docs.frigate.video/guides/getting_started, but ‎below you can see the configuration I am using for my setup.‎

Copy Code

mqtt:‎
‎  host: 192.168.10.102‎
‎  user: my_mqtt_user_name
‎  password: my_mqtt_user_password

detectors:‎
‎  # Required: name of the detector‎
‎  coral:‎
‎    # Required: type of the detector‎
‎    # Valid values are 'edgetpu' (requires device property below) and 'cpu'.‎
‎    type: edgetpu
‎    # Optional: device name as defined here: https://coral.ai/docs/edgetpu/multiple-edgetpu/#using-‎the-tensorflow-lite-python-api
‎    device: usb
‎  ‎
cameras:‎
‎  front-yard:‎
‎    ffmpeg:‎
‎      inputs:‎
‎        - path: rtsp://192.168.1.109:7447/unicast
‎          roles:‎
‎            - detect
‎    rtmp:‎
‎      enabled: False # <-- RTMP should be disabled if your stream is not H264‎
‎# Optional: Detect configuration‎
‎# NOTE: Can be overridden at the camera level‎
detect:‎
‎  # Optional: width of the frame for the input with the detect role (default: shown below)‎
‎  width: 3280‎
‎  # Optional: height of the frame for the input with the detect role (default: shown below)‎
‎  height: 2464‎
‎  # Optional: desired fps for your camera for the input with the detect role (default: shown below)‎
‎  # NOTE: Recommended value of 5. Ideally, try and reduce your FPS on the camera.‎
‎  fps: 5‎
‎  # Optional: enables detection for the camera (default: True)‎
‎  # This value can be set via MQTT and will be updated in startup based on retained value‎
‎  enabled: True
‎  # Optional: Number of frames without a detection before frigate considers an object to be gone. ‎‎(default: 5x the frame rate)‎
‎  max_disappeared: 25‎
‎  # Optional: Configuration for stationary object tracking‎
‎  stationary:‎
‎    # Optional: Frequency for confirming stationary objects (default: shown below)‎
‎    # When set to 0, object detection will not confirm stationary objects until movement is detected.‎
‎    # If set to 10, object detection will run to confirm the object still exists on every 10th frame.‎
‎    interval: 0‎
‎    # Optional: Number of frames without a position change for an object to be considered stationary ‎‎(default: 10x the frame rate or 10s)‎
‎    threshold: 50‎
‎    # Optional: Define a maximum number of frames for tracking a stationary object (default: not set, ‎track forever)‎
‎    # This can help with false positives for objects that should only be stationary for a limited amount of ‎time.‎
‎    # It can also be used to disable stationary object tracking. For example, you may want to set a value ‎for person, but leave
‎    # car at the default.‎
‎    # WARNING: Setting these values overrides default behavior and disables stationary object tracking.‎
‎    #          There are very few situations where you would want it disabled. It is NOT recommended to‎
‎    #          copy these values from the example config into your config unless you know they are needed.‎
‎    max_frames:‎
‎      # Optional: Default for all object types (default: not set, track forever)‎
‎      default: 3000‎
‎      # Optional: Object specific values‎
‎      objects:‎
‎        person: 1000‎
‎# Optional: Object configuration‎
‎# NOTE: Can be overridden at the camera level‎
objects:‎
‎  # Optional: list of objects to track from labelmap.txt (default: shown below)‎
‎  track:‎
‎    - person
‎    - car
‎# Optional: Record configuration‎
‎# NOTE: Can be overridden at the camera level‎
record:‎
‎  # Optional: Enable recording (default: shown below)‎
‎  # WARNING: If recording is disabled in the config, turning it on via
‎  #          the UI or MQTT later will have no effect.‎
‎  # WARNING: Frigate does not currently support limiting recordings based‎
‎  #          on available disk space automatically. If using recordings,‎
‎  #          you must specify retention settings for a number of days that‎
‎  #          will fit within the available disk space of your drive or Frigate‎
‎  #          will crash.‎
‎  enabled: True
‎  # Optional: Number of minutes to wait between cleanup runs (default: shown below)‎
‎  # This can be used to reduce the frequency of deleting recording segments from disk if you want to ‎minimize i/o
‎  expire_interval: 60‎
‎  # Optional: Retention settings for recording‎
‎  retain:‎
‎    # Optional: Number of days to retain recordings regardless of events (default: shown below)‎
‎    # NOTE: This should be set to 0 and retention should be defined in events section below‎
‎    #       if you only want to retain recordings of events.‎
‎    days: 10‎
‎    # Optional: Mode for retention. Available options are: all, motion, and active_objects‎
‎    #   all - save all recording segments regardless of activity
‎    #   motion - save all recordings segments with any detected motion
‎    #   active_objects - save all recording segments with active/moving objects
‎    # NOTE: this mode only applies when the days setting above is greater than 0‎
‎    mode: motion
‎  # Optional: Event recording settings‎
‎  events:‎
‎    # Optional: Maximum length of time to retain video during long events. (default: shown below)‎
‎    # NOTE: If an object is being tracked for longer than this amount of time, the retained recordings‎
‎    #       will be the last x seconds of the event unless retain->days under record is > 0.‎
‎    max_seconds: 300‎
‎    # Optional: Number of seconds before the event to include (default: shown below)‎
‎    pre_capture: 5‎
‎    # Optional: Number of seconds after the event to include (default: shown below)‎
‎    post_capture: 5‎
‎    # Optional: Objects to save recordings for. (default: all tracked objects)‎
‎    objects:‎
‎      - person
‎      - car
‎    # Optional: Restrict recordings to objects that entered any of the listed zones (default: no required ‎zones)‎
‎    required_zones: []‎
‎    # Optional: Retention settings for recordings of events‎
‎    retain:‎
‎      # Required: Default retention days (default: shown below)‎
‎      default: 10‎
‎      # Optional: Mode for retention. (default: shown below)‎
‎      #   all - save all recording segments for events regardless of activity
‎      #   motion - save all recordings segments for events with any detected motion
‎      #   active_objects - save all recording segments for event with active/moving objects
‎      #‎
‎      # NOTE: If the retain mode for the camera is more restrictive than the mode configured‎
‎      #       here, the segments will already be gone by the time this mode is applied.‎
‎      #       For example, if the camera retain mode is "motion", the segments without motion are‎
‎      #       never stored, so setting the mode to "all" here won't bring them back.‎
‎      mode: motion
‎      # Optional: Per object retention days
‎      objects:‎
‎        person: 10‎
‎        car: 10‎
ffmpeg:‎
‎  hwaccel_args:‎
       -c:v h264_qsv

Once your config file is complete, press “CTRL X” followed by “Y” and “Enter” to save the file. Now ‎we are finally ready to start our Frigate container.‎

Copy Code

cd ..‎
sudo docker-compose -f frigate.yml up

If you get any errors, you can view the docker log with the following command; otherwise, you can ‎close the terminal window.‎

Copy Code

sudo docker logs frigate

The first time I tried starting my Frigate container, I received an error of “Error starting userland proxy: ‎listen tcp 0.0.0.0:5000: bind: address already in use” which indicates something else was utilizing port ‎‎5000. I had nothing else installed that should have been using that port, so I was able to fix it with the ‎following command before retrying the docker-compose command above.‎

Copy Code

sudo kill -9 `sudo lsof -t -i:5000`‎

Provided there are no errors, you are ready to connect to the Frigate web interface. Start by getting ‎the IP address of your LattePanda by opening a new terminal and typing the following:‎

Copy Code

ip a

The output that follows should contain the computer’s IP address. Now, using a web browser on any ‎computer within the same network, type in the above IP address followed by “:5000” to access ‎Frigate. In my case, I will enter 192.168.1.138:5000 in my browser.‎

browser_9

Provided everything works correctly, you should now see your camera feeds. To verify detection is ‎working, you can click on a camera feed, click “Debug” in the upper right-hand corner, then click “Show ‎Options” where you can switch on “Bounding Box” and observe as the AI detects the objects you ‎specified in the config file in real-time. From here, you can use Frigate standalone, or if you run Home ‎Assistant, you can use the AI object detection to trigger automation.