How come at 4pm EST does the site go down every day.  Sometimes it is for 5mins and other times it is up to 30 mins or like last night over an hour?

There is an earlier thread on this ...

link to a message in this forum

In it jase says ...



He is researching new suppliers.

Something has been smashing the server at around 20:10 GMT every day, and yesterday it caused the server to run out of memory and pagefile (so it then locked up).  8pm GMT is 4am my time, so I don't see the down alert for a several hours.

Anyway I did a bunch of cleaning up yesterday which seemed to avoid the issue last night, hopefully everyone else saw the same.  I'll also lock down the firewall later today and see if any other spring cleaning can be done.

I just want to thank you for all your work and those of your admins and mods.

Thanks Jase, yes last night was a lot better.

Always a pleasure.

Still need to fix the server (well, get a new one) but it seems more stable at the moment after my spring clean.  Real test will be after about a week of running when resource allocation etc settles down, right now it's still on a high after a nice refreshing reboot.  As mentioned I also tightened up the firewall (about 16 hours ago) to increase security.  That's certainly stopped the attempts to log into the server from exotic locations.

Server is monitored 24x7 so I do get alerts if it's down, but keep in mind I'm GMT+8 so I might be fast asleep if there is another issue.

In reply to jase (msg # 6):

Jase, did you see my other post about SiteGround [link: https://www.siteground.com/ind...873a86f6cf8b1e249076] having Cloud services that automatically scale and move between physical servers to make downtime almost nonexistent?

I've got two really big clients (I'm a web dev) on these and I'm REALLY happy with them.

I did thanks.  It's an interesting option but I've still got concerns around processing power, plus we'd need a bit more disk (currently using 77GB) than most of their options.  Then there's the fact it's fully managed, which could cause issues with our customisations.  Certainly on the list though.

FYI;

I've been keeping a close eye on the server and noticed it suffered poor performance about two days ago.

Through a careful process of elimination in combination of checking the obvious I've tracked down what the issue was -- the backups.  We replicate all of RPoL's data to another location but the 3 million threads on record mean we've got a lot of files to backup, and the backup application was running out of memory trying to keep track of everything (as well as compress it on the fly to minimise bandwidth).

The backups have now been modified to do everything in much smaller parts (10th the size it used to), so hopefully that'll solve the memory utilisation issue.

Anecdotal evidence seems to indicate this (particular) issue has been resolved.  Fingers crossed.

In reply to jase (msg # 9):

Thank you sir!

Holler back at me on rmail about my request to help! :D

Looks like the problem has popped back up again.

Yeah, I've noticed that, too. Seems to have been hitting the same time; around 4pm Eastern (US). Doesn't last long, though. I usually just close the tab, busy myself for about a half hour and it'll come up.

The site seems to be suffering from some sort of hiccup, it's taking ages for pages to come up.  As of 2030 EST.

Any word on fixing the daily site outage?

Its still happening for me too, sometimes I can work around 4 other times its around 6 est for me when it dips.

It was down for about 5-8 hours today. I think the problem is back.

Yeah. I'm used to the 30 minutes to an hour outages every day at 1 PM (PST), but last night was much longer.

I haven't experienced any site outages for the past couple of weeks until today. It was down for about a couple of hours between 2pm PST TO 4pm PST. It was rather frustrating because I just finished a well written post and when I clicked to post it, nothing, after the outage subsided, it was blank and I was devastated,

Are they still going through server changes? That was the last I heard about something in regards to this.

Still a work-in-progress.  Put it on hold for a little while as last time I was looking for a comparable host I was finding the pricing was about 50% higher than our current (if not more).

If the dreaded daily outage starts to happen again I'll obviously have to prioritise a replacement server.