Ambitious 256 Camera Server

Post Reply
jmmolloy
Posts: 3
Joined: Tue Nov 10, 2020 3:45 pm

Ambitious 256 Camera Server

Post by jmmolloy »

At present my organization uses a virtual machine with 5TB of disk space hosting 49 cameras from 3 satellite locations.

We are now looking to leverage BlueIris to manage 165+ cameras across our 7 locations. I understand the limitation of 64 cameras per instance so here is my plan and I am looking for hardware input/experience/recommendations.

I plan to purchase a dedicated Dell PowerEdge R940xa with 38.4TB of disk space in a 10k RPM RAID 10 array.
Four processors: 2x Intel® Xeon® Gold 5222 3.8G, 4C/8T, 10.4GT/s, 16.5M Cache, Turbo, HT (105W) DDR4-2933
256GB Memory (x4 64GB RDIMM, 3200MT/s, Dual Rank)

I will host 5 Hyper-V guest instances on this Dell server. 4 of these instances will host 64 cameras each, enabling us to connect 256 cameras if necessary. The 5th guest server will be dedicated to serving the “enterprise view” for staff and security to access all 4 instances from one web/client interface (Ken says this feature is forthcoming).

We will only retain 14 days of footage at 720p using Amcrest cameras with wide-viewing angles for maximum coverage of large open spaces.

Does anyone foresee any issues running these 5 instances together, reading and writing to a single 10k RPM RAID 10 array?
Hosting 64 cameras, and with the specs mentioned above, what are the memory and processing recommendations for each instance? I wanted to attach a spec sheet for the server build out but I cannot upload PDFs (the important stuff is listed above anyway).

Any suggestions are greatly appreciated!

Other info:
The server will use a x4, 10G ether-channel (40Gbps throughput) connected to a 10Gbps fiber backbone to our satellite branches.
Matts1984
Posts: 496
Joined: Fri Apr 10, 2020 1:12 pm
Location: Maryland, USA

Re: Ambitious 256 Camera Server

Post by Matts1984 »

This is extremely above my pay grade at this scale but I'm incredibly jealous of your specs!

I'm not running Hyper-V nor multiple instances but I am using ESXi 6.5 with a Server 2019 VM for my BI system and it's not the only VM running. On other forums it's highly discouraged to use VMs but I've seen zero drawback with it - though if you wanted to use Nvidia decoding (you didn't mention it) that could get tricky to serve to the VMs. While not having specs to prove it, a new 10k RAID 10 array should be amazingly fast and sufficient for you. I'm using a much older 7200k RAID 6 array and the performance is similar to using a local SSD. BI does not use a lot of memory, so even with 64 cameras, I think allocating 24GB/instance would be sufficient. That also gives you a LOT of wiggle room if you need or want to increase it.

All this to say, with numbers like these I know you are looking for solid math and proof that things will work when configured like XYZ and I realize I'm not giving you that, so I'm gonna stop filling up forum space.
Blue Iris 5.9.4.x | Server 2022 VM | Xeon E5-2660 v3 @ 2.60GHz - 16 Cores | 24GB RAM | 8TB RAID | Sophos UTM WAF | Mostly various SV3C Cameras
HeneryH
Posts: 721
Joined: Thu Jul 18, 2019 2:50 pm

Re: Ambitious 256 Camera Server

Post by HeneryH »

Can you get intel gpu passthrough with Hyper-v?

Even if you could, I would assume only to one VM.
Matts1984
Posts: 496
Joined: Fri Apr 10, 2020 1:12 pm
Location: Maryland, USA

Re: Ambitious 256 Camera Server

Post by Matts1984 »

Fair point, most likely not. This system is probably going for brute force old school processing power. Yes a lot of optimizations to be had using QSV but with the number of cameras to manage, one physical server is a lot easier to manage and afford than 4 or 5.
Blue Iris 5.9.4.x | Server 2022 VM | Xeon E5-2660 v3 @ 2.60GHz - 16 Cores | 24GB RAM | 8TB RAID | Sophos UTM WAF | Mostly various SV3C Cameras
jmmolloy
Posts: 3
Joined: Tue Nov 10, 2020 3:45 pm

Re: Ambitious 256 Camera Server

Post by jmmolloy »

We don't have GPU passthrough enabled on our current virtual machine and, as Matts1984 had mentioned, brute force processing power has worked for us thus far. Additionally, we have multiple VMs on the same host and I (with only one BL guest) have not experienced any performance issues. My biggest concern was writing all 256, 112-degree cameras to the same raid with quick turn over and high I/O. I realize there's no way of truly knowing the performance limitations until I start digging into it - I just don't want to spend $20K on cameras and $40K on a server to have service degradation during the first year of deployment.

Blue Iris is SO AWESOME I just wish it could support more than 64 cameras per instance. At the price we're willing to pay for hardware, we (and other organizations I partner with) would happily pay $1K+ year for BL to support the number of cameras we need. Thank you for your input guys, any suggestions or feedback is most welcome!
Matts1984
Posts: 496
Joined: Fri Apr 10, 2020 1:12 pm
Location: Maryland, USA

Re: Ambitious 256 Camera Server

Post by Matts1984 »

Yeah, to be honest I started having some heartburn thinking about that this morning (having coffee in kitchen, not even on the forum - man I want things back to normal!). Thats a lot of read/writes. You're building your array about as quick and durable as is commercially available, but TBH I just don't know.

This almost feels like a solution for Verkada (I love BI and have no stake in Verkada - other than I've viewed some webinars that they send you free Yeti's for attending :) ). If you don't already have the cameras, it may be worth at least googling. You pay per camera and they all upload securely to the cloud so no server hardware on prem.
Blue Iris 5.9.4.x | Server 2022 VM | Xeon E5-2660 v3 @ 2.60GHz - 16 Cores | 24GB RAM | 8TB RAID | Sophos UTM WAF | Mostly various SV3C Cameras
Post Reply