It sounds like a Zookeeper issue, which I guess you already suspect.  

Maybe Zookeeper is not functioning properly on Server 5.  Maybe Server 4 cannot talk to Server 5.

https://www.novell.com/documentation/zenworks-2020/zen_sys_servers/data/t4f1y9ouwt65.html

https://www.novell.com/documentation/zenworks-2020/zen_sys_servers/data/t4f1y9ov00nk.html

I Craig,

Thank for your reply

I thought about it at first but when I did the “zman srzlc”.

the “zman srzuc” command was run just in case but we still have the problem.

What I don't understand is that the problem only occurs when servers are switched off.

My degraded mode is designed to allow part of the servers to be online to allow services to continue despite the loss or damage of servers.

Kind Regards,

For reasons I don't know....I suspect Zookeeper is not functioning properly when those servers are off, even though in theory it should be.

Zookeeper handles locks...If locks are not being handled....it could spike the DB.

You may need an Service Request.....They would likely want to extra log debugging as shown in my second link.

It could be something as simple as a FireWall issues between 4 and 5.....or more complex...

For reasons I don't know....I suspect Zookeeper is not functioning properly when those servers are off, even though in theory it should be.

Zookeeper handles locks...If locks are not being handled....it could spike the DB.

You may need an Service Request.....They would likely want to extra log debugging as shown in my second link.

It could be something as simple as a FireWall issues between 4 and 5.....or more complex...

I might also want to test what happens if you only leave 1 and 2 running.....

That would help isolate if it was an issue with losing 2 of 3 Zookeeper nodes or if it was something about Zookeeper on 5.