Diagnosing Computer Crashes
-
- Posts: 5
- Joined: Wed Aug 19, 2020 3:33 pm
Diagnosing Computer Crashes
Hi, I've been using Blue Iris for a few months now, and it has been working great with one problem... Every few days the computer it is running on just stops working, completely freezing up and requiring that the computer be powered off and back on again. Obviously, this stops the recording and renders the whole system useless until I notice the problem and restart the computer.
The computer is a HP ProDesk with an Intel Core I5-8500T, 16GB of RAM, and a 500GB Samsung 970 EVO NVME drive. I only have 3 cameras at the moment, so the computer is handling the load fine. The system consists of a fresh install of Windows 10 and Blue Iris with nothing else installed.
Trying to figure out my next step. Is there some diagnostic software that I can be running to identify the root cause of the crashes? Additionally, is there some piece of hardware that can power cycle the computer if it goes dark for more than 5 minutes or so?
Doug
The computer is a HP ProDesk with an Intel Core I5-8500T, 16GB of RAM, and a 500GB Samsung 970 EVO NVME drive. I only have 3 cameras at the moment, so the computer is handling the load fine. The system consists of a fresh install of Windows 10 and Blue Iris with nothing else installed.
Trying to figure out my next step. Is there some diagnostic software that I can be running to identify the root cause of the crashes? Additionally, is there some piece of hardware that can power cycle the computer if it goes dark for more than 5 minutes or so?
Doug
Re: Diagnosing Computer Crashes
I'm guessing we're talking about a full computer lockup - not BSOD, but the next closest thing - and only a hard power cycle helps.
I'd look at System logs. My gut says this is a driver though.
As for a hardware system that would power cycle... not that I'm aware of as an out-of-the-box system. You might be able to set up some sort of network based monitor to find when a particular service/port stops responding and then and then has the ability to control a smart plug - but you're probably looking at some custom coding or like a RaspberryPi like project.
I'd look at System logs. My gut says this is a driver though.
As for a hardware system that would power cycle... not that I'm aware of as an out-of-the-box system. You might be able to set up some sort of network based monitor to find when a particular service/port stops responding and then and then has the ability to control a smart plug - but you're probably looking at some custom coding or like a RaspberryPi like project.
Blue Iris 5.9.4.x | Server 2022 VM | Xeon E5-2660 v3 @ 2.60GHz - 16 Cores | 24GB RAM | 8TB RAID | Sophos UTM WAF | Mostly various SV3C Cameras
-
- Posts: 5
- Joined: Wed Aug 19, 2020 3:33 pm
Re: Diagnosing Computer Crashes
Basically, the system just stops what it was doing. If the screen was on at the moment of death, it stays on, but frozen at that last moment. Mouse, keyboard, drive lights, nothing changes. Nothing I've tried can bring the system back out from this other than a manual power cycle.
I don't know what program or log files are available that could help me diagnose this further. Any suggestions?
I don't know what program or log files are available that could help me diagnose this further. Any suggestions?
Re: Diagnosing Computer Crashes
Take a look at "Reliability History". Type it into the search bar. It keeps a record of failures. That was one of my biggest helpers in my recent pc fiasco.
Are you using Acronis True Image ? Reddawg pointed out that the ransomware protection slowed his pc down. It was a major factor on mine too. Ransomware turned off.
I also had a pci-e satellite tuner card with issues. That broke com ports, usb etc as well as just not working properly. Removed from pc and smashed to pieces with a sledgehammer. Very satisfying.
I moved BI5 from my AMD 8350 server due to these issues. It is presently running on an Intel i5-3330 with 16GB RAM and is solid.
1. I have BI5 automatic updates off. Weekend job.
2. I have Win10 updates postponed for two weeks. I will do them manually over the weekend rather than come back to a badly crashed server.
3. Sub-streams. Now I know why people used to think that BI5 was a cpu hog. Cpu down from 18% to 9%.
The BI5 server is showing 10/10 in Reliability now, and it has been up since last weekends Win10 updates. BI5 is running as a service, and all is well.
Let us know if your Rel History shows anything.
Are you using Acronis True Image ? Reddawg pointed out that the ransomware protection slowed his pc down. It was a major factor on mine too. Ransomware turned off.
I also had a pci-e satellite tuner card with issues. That broke com ports, usb etc as well as just not working properly. Removed from pc and smashed to pieces with a sledgehammer. Very satisfying.
I moved BI5 from my AMD 8350 server due to these issues. It is presently running on an Intel i5-3330 with 16GB RAM and is solid.
1. I have BI5 automatic updates off. Weekend job.
2. I have Win10 updates postponed for two weeks. I will do them manually over the weekend rather than come back to a badly crashed server.
3. Sub-streams. Now I know why people used to think that BI5 was a cpu hog. Cpu down from 18% to 9%.
The BI5 server is showing 10/10 in Reliability now, and it has been up since last weekends Win10 updates. BI5 is running as a service, and all is well.
Let us know if your Rel History shows anything.
Forum Moderator.
Problem ? Ask and we will try to assist, but please check the Help file.
Problem ? Ask and we will try to assist, but please check the Help file.
Re: Diagnosing Computer Crashes
Just realised, do you only have one hard drive ?
I keep BI5 storage and database away from the Windows drive. I have a second 250GB SSD just for the BI5 New folder and the db, and a 14TB mech hard drive just for the Storage.
I keep BI5 storage and database away from the Windows drive. I have a second 250GB SSD just for the BI5 New folder and the db, and a 14TB mech hard drive just for the Storage.
Last edited by TimG on Sun Aug 23, 2020 8:47 am, edited 1 time in total.
Forum Moderator.
Problem ? Ask and we will try to assist, but please check the Help file.
Problem ? Ask and we will try to assist, but please check the Help file.
Re: Diagnosing Computer Crashes
Thats pretty cool on the Reliability History - I can confirm it exists in Server 2019 as well (which is essentially Win 10 with some extra features anyway). Thanks for sharing that.
While I'm not sure it's causing the stability issue - yes I'd definitely recommend having a separate hard drive, at least logically, for any database like function, ie massive amounts of video files. There are a ton of read/writes going on that are harder on the drive than they'd typically be used to. If a drive fails, you're only talking about losing recordings. Add a new drive and you're back up in minutes (minus some historical video files, oh well). If it's your system drive, and it fails - well now you're starting over from scratch. Also I've seen too many times where some application fills the drive to max capacity and you lose all capability other than a direct console access to sort it out. I try and keep the minimum on C:
I'd still bet this is some driver related issue (or something like TimG's pci-e card). Hopefully the Reliability History can help point you in the right direction. Looks like it's System Event Logs with a graphical interface and some intelligence to showing you stuff that might actually be relevant. Once you find the issue:
While I'm not sure it's causing the stability issue - yes I'd definitely recommend having a separate hard drive, at least logically, for any database like function, ie massive amounts of video files. There are a ton of read/writes going on that are harder on the drive than they'd typically be used to. If a drive fails, you're only talking about losing recordings. Add a new drive and you're back up in minutes (minus some historical video files, oh well). If it's your system drive, and it fails - well now you're starting over from scratch. Also I've seen too many times where some application fills the drive to max capacity and you lose all capability other than a direct console access to sort it out. I try and keep the minimum on C:
I'd still bet this is some driver related issue (or something like TimG's pci-e card). Hopefully the Reliability History can help point you in the right direction. Looks like it's System Event Logs with a graphical interface and some intelligence to showing you stuff that might actually be relevant. Once you find the issue:
Blue Iris 5.9.4.x | Server 2022 VM | Xeon E5-2660 v3 @ 2.60GHz - 16 Cores | 24GB RAM | 8TB RAID | Sophos UTM WAF | Mostly various SV3C Cameras
-
- Posts: 5
- Joined: Wed Aug 19, 2020 3:33 pm
Re: Diagnosing Computer Crashes
Great info, and I'll start digging into it today and see where it leads...
BTW, unless some sort of product like this already exists, I was thinking that taking a Sonoff switch and reprogramming the ESP8266 inside to listen for a heartbeat and powercycle the computer if it doesn't get it, combined with a program on the Blue Iris computer to send the heartbeat to the Sonoff every few seconds... Obviously, there's a bit of programming involved, but the total hardware cost should be under ten bucks.
BTW, unless some sort of product like this already exists, I was thinking that taking a Sonoff switch and reprogramming the ESP8266 inside to listen for a heartbeat and powercycle the computer if it doesn't get it, combined with a program on the Blue Iris computer to send the heartbeat to the Sonoff every few seconds... Obviously, there's a bit of programming involved, but the total hardware cost should be under ten bucks.
-
- Posts: 5
- Joined: Wed Aug 19, 2020 3:33 pm
Re: Diagnosing Computer Crashes
I ran the system's memory test, and it came back fine. However, that's where the term "decent software" comes into play. I really don't know what to use for a full system diagnostic. If anyone has any suggestions, I'm listening.
I'm running the system and the "New" storage on a ludicrously fast NVME drive (Samsung 970 EVO Plus) in the hopes of overcoming any storage bottleneck issues. I can add a SATA drive, but that really seems counterproductive.
Re: Diagnosing Computer Crashes
Thanks Tim, I did not know about "Reliability History". That's actually very useful. I like that it shows App crashes and Windows updates.
Blue Iris v5.9.4.4 | Win10 x64 version 22H2 | Dahua IPC-HFW2100, Amcrest IP2M-841W, Hikvision MINI PT DS-2CD2F52F-IS, Edimax IC-3030iWn | Intel i5-8700 CPU, 16GB Ram, Samsung 860 EVO 1TB SSD, WD Black 1TB HD.
Re: Diagnosing Computer Crashes
DougLorenz: Does Reliability History give you any info ? You can click on previous days and it gives you the errors from that day. You can then double click on each event to see what it is about.
Other than that, as as been said before, it's likely to be drivers or hardware. You just have to rule them out one by one. For example:
1. If it freezes once a day, and you suspect overheating, take the side off the pc and leave a fan blowing air into it. Did that change the time to the crash ?
2. Are there any exclamation marks in "Device Manager" ?
3. Does this pc have built in diagnostics at BIOS level ? If so, run them all.
Please note that if you think I am trying to teach you to suck eggs, it is only because I am trying to help you, but I don't know how techy you are !
I have used software such as Sisoft Sandra to do burn-in tests on faulty pc's, but if you use that, be aware that it can finish off a failing pc. Reliability History, if it can help, is a very low risk, and easy way to see what is happening to your pc.
As they used to tell Sparky Tim down t'pit "If you can't fix it with a hammer, it's an electrical problem.". Such wise words![Laughing :lol:](https://blueirissoftware.com/forum/images/smilies/icon_lol.gif)
Other than that, as as been said before, it's likely to be drivers or hardware. You just have to rule them out one by one. For example:
1. If it freezes once a day, and you suspect overheating, take the side off the pc and leave a fan blowing air into it. Did that change the time to the crash ?
2. Are there any exclamation marks in "Device Manager" ?
3. Does this pc have built in diagnostics at BIOS level ? If so, run them all.
Please note that if you think I am trying to teach you to suck eggs, it is only because I am trying to help you, but I don't know how techy you are !
I have used software such as Sisoft Sandra to do burn-in tests on faulty pc's, but if you use that, be aware that it can finish off a failing pc. Reliability History, if it can help, is a very low risk, and easy way to see what is happening to your pc.
As they used to tell Sparky Tim down t'pit "If you can't fix it with a hammer, it's an electrical problem.". Such wise words
![Laughing :lol:](https://blueirissoftware.com/forum/images/smilies/icon_lol.gif)
Forum Moderator.
Problem ? Ask and we will try to assist, but please check the Help file.
Problem ? Ask and we will try to assist, but please check the Help file.