ovc start as non-root user fails with port 383 unavailable

The OBM environment on Linux RHEL 8.9 uses a non-root user

On nodes configured as couchbase servers, ovc start fails with "(ctrl-7) Error in the target component"

When I look in /var/log/OV/System.txt, I see the error:

ovbbccb (bbc-99) Opening the server port or the lookup for the bind address failed....

ovbbccb (bbc-261) The OV Communication Broker process failed due to a socket bind exception

ovbbccb (bbc-262) Please ensure the OV Communication Broker is started by the user 'root' or ensure the suid flag is set and the port is currently not in use by another application

I verified that port 383 is not in use

netstat -tulp | grep 383

This only occurs on four of over 200 nodes.  Those four happen to be running a couchbase-server process.  However, it doesn't appear to be using port 383

When I install the agent, it first installs with a "root" user runs fine.  Then when I run the /opt/OV/bin/ovswitchuser script to change the OBM user to a non-root user, it will not start.

Any ideas?

  

  • Verified Answer

    +1  

    Hello,

    I think this is best documented here:

    https://docs.microfocus.com/doc/Operations_Agent/12.24/InstallConfiguringAgentUser

    Since the default communication port for the agent is 383 and the non-root user on UNIX doesn't have the permission to access ports lower than 1024, you must perform this step to assign a non-default communication port to the agent.

    If the non-root user on Linux wants to access ports lower than 1024, execute the following command:

    setcap 'cap_sys_chroot,cap_net_bind_service=+ep' /opt/OV/bin/ovbbccb

    There is also an alternative which used to be documented:

    Run ovswitchuser.sh -existinguser <USER> -existinggroup <USERGROUP> including following steps:
    create a user
    # ovc -kill
    # ovswitchuser.sh -existinguser <user> -existinggroup <group>
    # ovconfchg -ns bbc.cb.ports -set PORTS <nodename>:8383
    # chmod 4550 /opt/OV/bin/ovbbccb
    # ovconfchg -ns bbc.cb -set CHROOT_PATH /

    An example (System.txt):
    0: ERR: Wed Nov 16 16:13:09 2016: ovbbccb (11370/1): (bbc-99) Opening the server port or the lookup for the bind address failed: Bind address: '', port: '383', error message: (xpl-352) listen() on '[::
    ]:383' failed.
    (RTL-9) Bad file number.
    0: ERR: Wed Nov 16 16:13:09 2016: ovbbccb (11370/1): (bbc-264) The OV Communication Broker failed due to IOException_t: '(xpl-352) listen() on '[::]:383' failed.
    (RTL-9) Bad file number'
    0: ERR: Wed Nov 16 16:13:09 2016: ovbbccb (11370/1): (bbc-250) OV Communication Broker stopped. Exit code (100).

    As root user:
    # chmod 4550 /opt/OV/bin/ovbbccb
    # ovconfchg -ns bbc.cb -set CHROOT_PATH /


    4550 permissions mean the owner of the file has read-only permission, the group members can read and execute the file, and others (those not the owner or part of the group) can also read and execute the file. The most significant digit is 4 means set the sticky bit. The sticky bit allows access and deletion of files within them.

    # su - opc_op
    $ /opt/OV/bin/ovc -start
    $ /opt/OV/bin/ovc
    coda OV Performance Core COREXT (11338) Running
    opcacta OVO Action Agent AGENT,EA (11413) Running
    opcmsga OVO Message Agent AGENT,EA (11406) Running
    ovbbccb OV Communication Broker CORE (11391) Running
    ovcd OV Control CORE (11296) Running
    ovconfd OV Config and Deploy COREXT (11392) Running

    I was suprised to see there was not a KCS document about this - I'll write one later today.

    I hope this helps.

  • 0 in reply to   

    We solved this by rebuilding the 4 of over 400 VMs that were experiencing the problem.  After re-kicking them the OA Agent installed and is working fine.  I do appreciate the feedback on how to move to a >1000 port for bbcutil.  If needed, I will go that route in the future.