{"id":1100,"date":"2020-05-06T11:04:35","date_gmt":"2020-05-06T15:04:35","guid":{"rendered":"http:\/\/www.thesmarthomehookup.com\/test_install\/?p=1100"},"modified":"2023-02-08T15:54:48","modified_gmt":"2023-02-08T20:54:48","slug":"the-ultimate-home-surveillance-system-free-local-ai-person-detection","status":"publish","type":"post","link":"https:\/\/www.thesmarthomehookup.com\/test_install\/the-ultimate-home-surveillance-system-free-local-ai-person-detection\/","title":{"rendered":"The Ultimate Home Surveillance System &#8211; Free Local AI Person Detection"},"content":{"rendered":"<p><iframe src=\"https:\/\/www.youtube.com\/embed\/fwoonl5JKgo\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p><strong>Ultimate Home Surveillance System<\/strong><\/p>\n<p>Today on the hookup I\u2019m going to show you the inner workings of my locally controlled, non cloud based surveillance system including free AI person detection and 24\/7 secondary resolution recording. Even if you\u2019re not interested in the ultimate surveillance system, stick around for some crazy home assistant tricks that 99% of users don\u2019t know about.<\/p>\n<p>You\u2019ve got a ton of options when it comes to video surveillance, there\u2019s dedicated NVR packages, IP cameras with on device recording, user friendly wireless cameras, and many more, and every system has it\u2019s merits and use cases, but one of the questions I get asked most often on my channel is \u201cwhat the BEST solution is for video surveillance\u201d, so today, I\u2019m going to show you my system, and then I\u2019m going to show you how you can do it yourself.<\/p>\n<p>Here it is, my PC based Blue Iris surveillance system that records all 9 of my outdoor cameras 24 hours a day, 7 days a week at low 640&#215;480 resolution.&nbsp; Locally processed AI computer vision checks each standard motion event for relevant objects. &nbsp;Personally, I use it to detect people and cars, but you might have a use for some of the other available detection objects like dogs, cats, or even bears.&nbsp; In the event a relevant object is detected it starts recording at full 4K resolution and sends an MQTT message to home assistant to be used in automations.&nbsp; Using this system I can sort through my alerts and not have to deal with a bunch of false motion events like shadows moving and trees blowing\u2026 every clip corresponds to a positive AI detection event which I\u2019ve found to be over 98% accurate, meaning less than 1 out of 50 events is a false trigger.&nbsp; Using this method my 4 terabyte surveillance grade hard drive gives me over 30 days of 24\/7 recording at 640&#215;480 with all relevant motion recorded at 4k.&nbsp; In contrast, recording 24\/7 in 4k would fill up a 4 terabyte hard drive in less than 5 days, and significantly shorten the lifespan of the drive.<\/p>\n<p>One of my favorite parts of a blue iris system is the excellent web based interface that allows you to monitor your cameras and view recorded footage right in chrome without the need for any plugins or special settings.&nbsp; Blue Iris UI3 is light on resources and is hands down the best way to remotely monitor your security system on your PC, I actually prefer using the web interface to the full blue iris program for monitoring and reviewing footage, I only open up the main blue iris program for changing settings or adding cameras.&nbsp; In The blue iris UI3 interface I can view high live views, review all the AI motion events from a specific camera, or quickly scrub through the lower resolution 24\/7 recording.&nbsp; Each motion event generates a thumbnail preview just by hovering over it, making it by far the easiest way to review security footage. I also love the server status system monitor that tells me the CPU and memory usage of my server.<\/p>\n<p>Because the motion alerts are around 98% accurate, I can reliably use them in automations, like sending notifications if there\u2019s a person at the door, or someone is in the back yard when we are not home. &nbsp;I use AI person detection to turn the patio light on and off reliably, and I even have a text to speech event that announces to trespassers that they are being recorded and that I have been notified of their presence if they are detected in the back yard while the house alarm is on.<\/p>\n<p>The best part about all of this, is that all of this is locally controlled and hosted, 100% functional without an internet connection, and doesn\u2019t rely on any cloud services.<\/p>\n<p>Sound interesting? Here\u2019s how it works.<\/p>\n<p>Before we get started I have to address the Elephant in the room: The biggest hurdle for many to overcome with this system is not price or technical ability, but the fact that Blue Iris is a windows based program and cannot run on Linux.&nbsp; I\u2019d like to be able to tell you that a linux version is on it\u2019s way, but it\u2019s just not the case, Blue Iris is, and will remain windows only for the foreseeable future.<\/p>\n<p>I personally run my blue iris system on the same computer that handles my plex server and hosts the virtual machine where I run home assistant, which is a 6<sup>th<\/sup> generation intel core i7 with 32 gigabytes of RAM, 2 4-terabyte spinning hard drives, and a 256 gigabyte solid state boot drive.&nbsp; Mine was a hand me down, but you could build something similar for around $400-500 using ebay or facebook marketplace.<\/p>\n<p>For reference, here\u2019s what the resource usage looks like running two transcoded plex streams, home assistant VM, and AI person detection on all 9 of my cameras.<\/p>\n<p>The other common complaint about blue iris relates to feature creep, which causes the number of settings and options to be completely overwhelming for a new user.&nbsp; I\u2019m not going to cover every setting in this video, but I\u2019ll hit the important ones.&nbsp; To that end, the most important setting, and the one that will make or break your blue iris experience is something called direct to disc recording.&nbsp; This basically means that blue iris takes the exact RTSP stream from your camera and records it without modifying the resolution, bitrate or frame rate, which means your system doesn\u2019t need to waste any processing power re-encoding the video stream.<\/p>\n<p>If you don\u2019t use this setting your computer will absolutely not be able to handle 9 &nbsp;camera streams, but I mentioned that we need access to both a low resolution stream for 24\/7 recording and a high resolution stream for motion events, so how is that possible without re-encoding?<\/p>\n<p>Well, almost all security cameras have the option to output both a main stream, and a second substream, so we\u2019re going to utilize the main stream for our high resolution recordings, and the substream for our 24\/7 recordings.&nbsp; Every camera sets this up a little bit differently, but here are the settings I use on my Annke C800, which is the camera I chose for best all around best 4k camera.<\/p>\n<p>After logging in to the web interface for the camera, click Video\/Audio and select main stream from the stream type dropdown.&nbsp; For this stream I want to use the maximum resolution at 15 frames per second using a variable bitrate and I\u2019m going to use H265+ encoding which will result in much lower file sizes. &nbsp;&nbsp;Go ahead and save that profile, then select sub-stream in the dropdown.&nbsp; For this stream I want 640&#215;480 resolution, 15 frames per second, variable bit rate at only medium quality, and H265 encoding.&nbsp; The low resolution stream will produce very low file sizes which is great for both 24\/7 recording, and AI computer vision, that can sometimes struggle with larger images.<\/p>\n<p>The last thing to do is a bit of googling to find your specific camera manufacturer\u2019s RTSP URLs for the main stream and substream, which are all a bit different.&nbsp; For instance, these are the URLs for Annke cameras.<\/p>\n<p>With our feeds correctly setup we can add them to Blue Iris and set them up for computer vision.<br \/>\nAnd by the way, I\u2019m not taking any credit for this computer vision stuff, the ipcamtalk user GentlePumpkin is the mastermind behind all of this.&nbsp; Link to his awesome how-to post is in the description.<\/p>\n<p>As I mentioned, starting out in blue iris can be overwhelming, so lets cover some of the most basic settings first.&nbsp; Go to main menu, and then general settings. &nbsp;In the clips and archiving tab, at the very least you need to setup a new footage folder to store your video files.&nbsp; You should limit the total amount of space on the drive that will be occupied by footage, and when that amount of space is occupied you can choose to delete clips or archive them in a different location like a NAS drive using the stored folder.&nbsp; For this setup you\u2019ll also need a folder for still images to be used with computer vision, so select Aux 7 and rename it aiinput, and give it a small size limit like 1 gigabyte, since it will only contain low resolution images.<\/p>\n<p>While we\u2019re in setup click over to the web server tab. &nbsp;In this tab you need to decide which port you want to use for your blue iris webserver, I recommend changing it from port 80, which is often used by other services, I personally use port 81.&nbsp; Check the box that says \u201cUse UI3 for non-IE browsers\u201d, which is the awesome web based UI, and make sure the correct ip address is listed for your local internal LAN access.&nbsp; If you want to set up your blue iris to be accessible from outside your network you can also setup remote access, but the most secure way to set it up is without external access and instead use a VPN to simulate local access when you are away.&nbsp; Last, hit advanced and set your authentication to require from non-lan only, meaning if you log into blue iris from within your network you won\u2019t need to enter a username and password.&nbsp; Also deselect \u201cuse secure session keys and login page\u201d and hit okay.<\/p>\n<p>Last, click on users and create a new user with a secure password and admin privileges to use with your computer vision system.<\/p>\n<p>The next step is to add a camera.&nbsp; The first camera we\u2019re going to add is our low resolution 24\/7 recording substream camera. Click main menu, add new camera.&nbsp; On the next screen give the camera a descriptive name, I\u2019m going to call mine annkesd.&nbsp; Select \u201cnetwork IP\u201d as the camera type and check \u201cenable motion detector\u201d and \u201cDirect to disc recording\u201d.&nbsp; Hit okay and the camera configuration window will appear. This is where you\u2019ll put the URL for the rtsp stream of the cameras substream along with the username and password of the camera.&nbsp; I\u2019m not 100% sure what the setting does, but I like to click the box that says \u201cLimit decoding unless required\u201d, because that sounds like a pretty good thing to do!<\/p>\n<p>In the general window you can assign this camera to as many groups as you\u2019d like.&nbsp; I recommend at least adding it to a group called \u201csubstreams\u201d that you can use to comb through any of the 24\/7 recordings that you need to.<\/p>\n<p>In the trigger window you can click the configure button next to motion sensor to increase the sensitivity of detection, because we want to error on the side of too much detection, since the computer vision will be checking them all anyways. &nbsp;Deselect the \u201cCapture an alert list image\u201d box and set the break time to 4 seconds. In the record tab, select continuous from the dropdown and leave the folder as new.&nbsp; Also check the box that says JPEG snapshot each, and change the time period to 5 seconds, and the quality to 100%.&nbsp; Select your aiinput folder and then check the \u201conly when triggered\u201d box.&nbsp; This will take a new still image every 5 seconds when motion is detected and save it into the computer vision folder for analysis.&nbsp; I like to have my continuous recording divided up into 1 hour files, but it\u2019s really up to you.&nbsp; The last thing to do on this page is to deselect the box to include the jpegs in the all clips timeline.<\/p>\n<p>Last, head over to the alerts tab, and select \u201cNever\u201d from the dropdown box, and then you\u2019re done with your 24\/7 recording standard definition stream with jpeg capture on motion.&nbsp; You\u2019ll repeat this exact same process for each of your cameras, luckily, when you create a new camera it allows you to copy the settings of an existing camera, which significantly shortens the process since you\u2019ll only need to update the camera stream URLs and camera name.<\/p>\n<p>To setup the full resolution cameras go to main menu and then add camera, use the same name as before, but hd instead of sd, and this time only select direct to disc, not motion detection.&nbsp; For this camera you\u2019ll enter the URL for your camera\u2019s main high resolution stream and again, and check that mysterious limit decoding unless required box.<\/p>\n<p>Under trigger, the only box you want checked is capture an alert list image, and under record you\u2019ll select \u201cwhen triggered\u201d and then give yourself at least 5 seconds of Pre-trigger video buffer.&nbsp; As far as I can tell, the video pretrigger is stored in RAM, so this shouldn\u2019t be too taxing on your drive, but if you have a low RAM system with 4K cameras you might run into issues, I\u2019ve found that 5 seconds of pretrigger is enough for cameras that will be spotting people, but I use 10 seconds for car detection.<\/p>\n<p>Repeat this process to add each of your high resolution camera streams.<\/p>\n<p>You may have noticed that we set this cameras to record on trigger, but then we disabled motion detection, which is the primary way to trigger a camera, and that\u2019s because the AI program is going to be in charge of triggering the camera, so lets get that set up.<\/p>\n<p>The AI program has two components: The computer vision server that processes images using an API, and a custom built program made by GentlePumpkin that watches the aiinput folder for specific files and sends those images to be processed by the computer vision server.<\/p>\n<p>The computer vision server we\u2019re going to use is called \u201cDeepStack by DeepQuest AI\u201d and it has both paid and free level pricing.&nbsp; We\u2019re going to utilize the free version, but you still need to register on their website to get an activation code\u2026 Even though we need to register to get an activation code, all images are processed locally and no cloud services are used.&nbsp; Once you sign up with DeepStack using the link from the description you can obtain your activation key from the web interface by clicking on dashboard.<\/p>\n<p>You have two main options for installing the deepstack server, either as a windows program, or a docker container.&nbsp; The HUGE downside to using the windows program is that there is no way to auto start the program after a restart because it can\u2019t be run as a service, and you need to click specific buttons to start the server, so to get a consistent and reliable system we need to use the docker container.&nbsp; You could install docker for windows to accomplish this, but if you\u2019re already running home assistant in a virtual machine there is another option: You can install additional docker containers in your home assistant instance, since HassOS uses docker, you just need to access it by installing the portainer addon from the community addons repository.<\/p>\n<p>A quick warning: This gives you way more access than you\u2019re used to having in home assistant, and the likelihood of breaking everything is high if you mess something up, so make sure you have a backup of your virtual machine, and know that this is not for the faint of heart.<\/p>\n<p>After installing portainer, toggle the switch at the top to turn off safe mode, and hit start.&nbsp; After giving it a few seconds to load, hit open webUI to get to the portainer interface.<\/p>\n<p>Click on primary, then containers, and add container.<\/p>\n<p>Name your container something descriptive like deepstack and then for the image, we\u2019re going to pull it from the docker server by entering deepquestai\/deepstack.<\/p>\n<p>We need to send commands to this server, so we need to map a port, hit manual network port publishing.&nbsp; I used port 83, so click publish new network port and map port 83 on the host to port 5000 on the container.<\/p>\n<p>Under advanced container settings, go to volumes and hit map additional volumes, and you\u2019re going to type in \/datastore and click localstorage \u2013 local from the dropdown,<\/p>\n<p>then go to env and add an environmental variable called VISION (DASH) DETECTION in all caps and set it equal to true,<\/p>\n<p>last, click on restart policy and set it to always, then hit deploy container.&nbsp; If everything went well, you should now have a computer vision server running on the ip address of your home assistant virtual machine on port 83 that will automatically start any time your virtual machine is running.<\/p>\n<p>Access your deepstack server by putting that address in a web browser, and you\u2019ll need to put in your activation code from the deepstack website dashboard.&nbsp; The activation code expires after two years, but it\u2019s not clear what will happen at that point\u2026 hopefully you\u2019ll just need to paste in an updated activation code.<\/p>\n<p>Next we need to set up the program that serves your blue iris images to the vision server.&nbsp; Grab the zip file for the latest version of the ai tool from the link in the description and unzip it somewhere permanent on your blue iris computer.&nbsp; You\u2019ll set this up as a windows service later so it automatically starts when your computer reboots, but for now, lets just get it setup.&nbsp; So double click on aitool.exe to get it started.<\/p>\n<p>Under the settings tab, go to input path and browse to the aiinput folder that you set up in blue iris, then for deepstack url you\u2019ll enter the ip address if your virtual machine colon 83, which is the port that the service is running on.&nbsp; Then you\u2019ll setup a camera by clicking add camera.&nbsp; Give it a name and then where it says input file begins with, you\u2019ll put in the name of the low resolution camera we set up in blue iris, so annkesd in this case, if you\u2019re not sure what this will be you can just go look at the aiinput folder to see what prefixes your files have.<\/p>\n<p>Select which objects you\u2019d like to detect and leave your confidence limits at 0 and 100%, we\u2019ll adjust these later when we have some data.&nbsp; I choose not to use the cooldown timer, but you can set this to only analyze one image from that camera in a specific period of time, this would be useful if you have limited system resources and are noticing high CPU usage.&nbsp; The last thing you need is your trigger URL, this uses the blue iris API to trigger a specific camera by feeding it the command \u201ctrigger\u201d with the camera name, username and password as attributes.&nbsp; If you don\u2019t get this URL correct, nothing is going to work, so check your command by putting it in a normal web browser, if it\u2019s correct you should get a response like this and your HD camera should start recording.&nbsp; Hit save and test it out.&nbsp; You should see the Overview page switching between running and processing image.&nbsp; If it seems to be stuck on the processing image step, you likely have an issue with your deepstack server url, or your deepstack server may not be running.<\/p>\n<p>In the history URL you can see the images that are being feed to the AI server, and what objects were detected along with how confident the AI was that it was actually that object type.&nbsp; If you\u2019re noticing too many false positives, you can raise the lower confidence limit, but I\u2019d recommend getting a feel for each camera and the confidence levels that it generates before tweaking those values.<\/p>\n<p>A problem that I personally ran into was detection of objects that were not the cause of the motion, for instance, I wanted to detect cars with my second story front yard camera, but it always detected the cars that were parked in my neighbor\u2019s driveway, resulting in a lot of false motion.&nbsp; The good news is that it\u2019s pretty easy to make a mask for one specific area, which will block triggers from objects in that area.&nbsp; There\u2019s a good tutorial for doing that in GentlePumpkin\u2019s post, but the result is that the computer vision still works, but it doesn\u2019t trigger the camera if the object was in a masked zone.<\/p>\n<p>I recommend checking on your history tab for the first couple of days you are running the server to see what issues the computer vision might have with your specific setup, and when you feel confident with the accuracy of the detection you can set AI tool up to run as a service in the background which will start immediately whenever your computer restarts, again, the exact method for that is outlined in detail in GentlePumpkin\u2019s post.<\/p>\n<p>Now that your motion detection won\u2019t be tricked by shadows, spider webs, rain, or moving trees you can reliably use that data in automations.&nbsp; Blue iris can send and receive MQTT messages, allowing you to get this information in node red, or even set up binary sensors in home assistant.&nbsp; To set up MQTT in blue iris click on main menu, then digital io and iot, then hit the configure button in the MQTT section and put in the credentials for your MQTT broker.&nbsp; Then, to actually send a message, we need to go to the setting of the full resolution camera that we want to monitor, click on alerts, select \u201cfire when this camera is triggered\u201d and then under actions, we\u2019ll go to on alert and add an MQTT topic and payload, then hit okay, and you can test it by hitting the lightning bolt icon.<\/p>\n<p>To set up a motion sensor in home assistant we can add an entry to the configuration file under binary sensor set the device class to motion, put in the MQTT topic for that camera, and since we\u2019re only sending the positive motion events, we need to have an \u201coff delay\u201d for a certain number of seconds.&nbsp; You can tweak this to your liking, but I used 30 seconds without a new motion event to revert back to the \u201cclear\u201d status.<\/p>\n<p>Once in home assistant you can use this binary motion sensor to do any number of things like send actionable notifications, warn intruders, or control outdoor lights only when it\u2019s dark, and a human has been detected, the possibilities are endless.<\/p>\n<p>I\u2019m not sure how many people are going to make it to this point in the video, because setting a system like this can feel overwhelming at times, but in my opinion this is the absolute best system possible for keeping things local, relatively inexpensive and reliable.&nbsp; If you have questions or need help leave a comment or come join the thousands of home automation and security enthusiasts on the hookup home automation facebook group.<\/p>\n<p>If you\u2019d like to help support my channel and the videos that I make, consider becoming a patron or check out the other links in the description.&nbsp; If you enjoyed this video, please hit that thumbs up button and consider subscribing, and as always, thanks for watching the hookup.<\/p>\n<h3>GentlePumpkin&#8217;s AI Write Up: <a href=\"https:\/\/ipcamtalk.com\/threads\/tool-tutorial-free-ai-person-detection-for-blue-iris.37330\/\">https:\/\/ipcamtalk.com\/threads\/tool-tutorial-free-ai-person-detection-for-blue-iris.37330\/<\/a><\/h3>\n<h3>AI Tool Source Code: <a href=\"https:\/\/github.com\/gentlepumpkin\/bi-aidetection\">https:\/\/github.com\/gentlepumpkin\/bi-aidetection<\/a><\/h3>\n<h3>DeepStack AI Server: <a href=\"https:\/\/deepstack.cc\/\">https:\/\/deepstack.cc\/<\/a><\/h3>\n<h3><\/h3>\n<h3>My favorite 4K Camera (Annke C800)<\/h3>\n<h3>Turret: <a href=\"https:\/\/amzn.to\/35yqnVf\">https:\/\/amzn.to\/35yqnVf<\/a><\/h3>\n<h3>Bullet: <a href=\"https:\/\/amzn.to\/35zZu36\">https:\/\/amzn.to\/35zZu36<\/a><\/h3>\n<h3>My favorite 5MP Camera (Reolink RLC4X0)<\/h3>\n<h3>Turret: <a href=\"https:\/\/amzn.to\/3c7WxsV\">https:\/\/amzn.to\/3c7WxsV<\/a><\/h3>\n<h3>&nbsp;Bullet: <a href=\"https:\/\/amzn.to\/2WtJsno\">https:\/\/amzn.to\/2WtJsno<\/a><\/h3>\n<h3>Get help on the Facebook Group: <a href=\"https:\/\/www.facebook.com\/groups\/473812443269387\/\">https:\/\/www.facebook.com\/groups\/473812443269387\/<\/a><\/h3>\n<h3>Follow me on Twitter: @TheHookUp1<\/h3>\n<h3>Support my channel: Patreon: <a href=\"https:\/\/www.patreon.com\/thehookup\">https:\/\/www.patreon.com\/thehookup<\/a><\/h3>\n<h3>Music by www.BenSound.com<\/h3>\n","protected":false},"excerpt":{"rendered":"<p>Ultimate Home Surveillance System Today on the hookup I\u2019m going to show you the inner workings of my locally controlled, non cloud based surveillance system including free AI person detection and 24\/7 secondary resolution recording. Even if you\u2019re not interested in the ultimate surveillance system, stick around for some crazy home assistant tricks that 99% [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":2538,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[4,5],"tags":[],"class_list":["post-1100","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-projects","category-tutorials"],"acf":[],"mb":[],"mfb_rest_fields":["title","gutenberg_elementor_mode"],"_links":{"self":[{"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/posts\/1100","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/comments?post=1100"}],"version-history":[{"count":4,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/posts\/1100\/revisions"}],"predecessor-version":[{"id":2293,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/posts\/1100\/revisions\/2293"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/media\/2538"}],"wp:attachment":[{"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/media?parent=1100"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/categories?post=1100"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.thesmarthomehookup.com\/test_install\/wp-json\/wp\/v2\/tags?post=1100"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}