One of our servers (171.64.122.74) had some issues during maintenance which delayed it’s coming back on line. Due to this, 171.64.122.74 was back on line on Saturday (PST) instead of Friday evening, due to an issue found during maintenance. Luckily this maintenance is a once a year thing, so it won’t be needed for a while. It’s back up now.
Due to this server being off line, the backup server (171.64.65.20) took quite a hit. To avoid this, we’re moving to longer WU’s (8-24 hours) and looking to add an additional server so the servers take less of a hit. Longer WU’s alone will make a HUGE difference, since the servers would get hit much less frequently and thus the load would be much smaller. That should greatly improve these issues.
The current situation is that all servers are up and running, but there’s a lot of clients trying to access them. We’ll keep an eye on them throughout the weekend to make sure they’re going well.