Best practice for NNMi policy ?

Hi Community,

I'm using Opsb and NNMi and I'm wondering if there are any best practice with NNMi SNMP trap policy ? 

When you want to get nnmi incidents on omi, you should deploy a SNMP Interceptor policy on NNMi. This policy is provided by a command on nnmi. 

This command allow you to export all incidents or only incidents for a specific OID : example export only VMWare incident/traps. then you get a file you could import in OMi then deploy on NNMi which contain only rules for VMWare traps.

My question, is it better to have 1policy with all traps inside (Huge file), or is it better to have multiple policy deployed NNMi (One per Vendor for example) ? 

There is limitation or impact if you deploy 20 Snmp policies on a single agent  ? 

Regards 

Tags:

  • 0  

    Hello,

    https://docs.microfocus.com/doc/Operations_Bridge_Manager/24.2/IntegrateNNMi and https://docs.microfocus.com/doc/Network_Node_Manager_i/24.3/NOMOBMIntegration describes how you can integrate OBM with NNMi.  The general idea is that NNMi will manage the events, forwarding only what NNMi needs to forward - so it acts as a filter.  If you start forwarding all events (which you can do), then you may end up with lot of events.  Each one of those events needs to have a business reason why it's there, unless OBM is going to process the events automatically (e.g. SBEC/TBEC).  So, I recommend only forwarding what you really need.  There are also some configuration variables that help: docs.microfocus.com/.../ConfigurationVariablesNNMi

    When generating policies from NNMi they can get quite large.  In performance terms, there are yet more configuration options like OPC_SEND_MAX_ONE_MSG_PER_TRAP (see https://docs.microfocus.com/doc/Operations_Agent/12.04/ref_guide/rg_confvar2) that can also help.  OPC_SEND_MAX_ONE_MSG_PER_TRAP controls processing of additional trap templates, if a message was already generated from a template.  This speeds up processing, but may prevent some templates from even 'looking' at the trap.  As a result, less messages might be sent to the server and the behavior of duplicate suppression for 'Suppress Messages Matching Condition' may change.  Default is false so all polices are evaluated. It is not possible to specify which order policies are processed so use with care.  

    I think it's difficult to answer if it's better to have one policy with all traps inside (beung a large file), or is it better to have multiple policy deployed NNMi (one per vendor).  In some ways, having multiple files means you can split management tasks.  I'm not sure I recommend 10MB+ policy files simply because it can take some time to edit the file.  So, I think it's almost impossible to answer without lots of information.

    I think knowing how opctrapi works can help a little - every incoming trap to the server will be processed by opctrapi process trying to match against all the rules/conditions inside every trapi type policy. As opctrapi is a single-threaded process it will sequentially go through all the rules in all the policies one after one until a condition is matched thereby taking time, unless OPC_SEND_MAX_ONE_MSG_PER_TRAP is being used. As the number of unmatched or unwanted traps increases or when the most matched conditions are at the bottom of the policies, the more time opctrapi takes to process them as it checks against all the rules inside the policies and eventually resulting in the spike of CPU usage.

    Recommendations: move frequently matched conditions/rules to the top of the condition list inside a policy.
    - Move the Suppression rules to the top of the policy. (so that unwanted traps do not go through the rest of the rules).
    - Check if the unmatched conditions are rightly defined in the policies.
    - Reduce the usage of Complex Regular Expressions in the policies as much as possible.
    - Increase the value of HOSTNAME_CACHE_TIMEOUT to 86400 or more as it helps to cache the hostnames of resolved IP addresses in the traps, for a longer duration thereby saving the time to query the DNS server.

    I hope this helps and sorry it's a little vague.  Hope you get a better answer.

  • Verified Answer

    +1   in reply to   

    Hello again,

    I should have also mentioned the SNMP receive buffer size. This can help during high SNMP loading and (potentially) slow event processing as it means events are stored on the receive buffer used by the socket.  This means that event processing speed is not as importaint and events should not get lost, but this isn't meaning you should not be careful with ensuring all the event processing rules are efficient (as described above). It's based around the eaagt configuration variable SNMP_SERVER_RECV_BUF which is documented here: docs.microfocus.com/.../ConfiguringSNMPTrapInterceptor.

    I think this may be useful:

    SNMP_TRAP_QUEUE_SIZE: Use this property to specify the number of traps that can be held in opctrapi application memory. You can set a minimum trap queue size 0, and a maximum trap queue size of 112,500. The maximum value supports a trap burst rate of 1250 traps per second for 90 seconds. The variable is available only on the Linux platform.

    The SNMP_SERVER_RECV_BUF under [eaagt] namespace has options like:
    SNMP_SERVER_RECV_BUF=OS_DEFAULT (default value, value from kernel net.core.rmem_default).
    SNMP_SERVER_RECV_BUF=OS_MAX (default value, value from kernel net.core.rmem_max).
    SNMP_SERVER_RECV_BUF=NETSNMP_DEFAULT (netsnmp decides the value).
    SNMP_SERVER_RECV_BUF=<Custom_Value> (net.core.rmem_max may not be enough).

    You will need to use a some custom value – for example:
    # ovconfchg -ns eaagt -set SNMP_SERVER_RECV_BUFFER 262144


    SNMP_SERVER_RECV_BUF: Use this property to specify the amount of data read from the UDP buffer through the NETSNMP library (this is what opctrapi uses to format SNMP events) during each iteration. When SNMP trap requests are sent to the SNMP agent over UDP, the OS's networking stack buffers the incoming packets before they reach the SNMP trap interceptor. If the buffer is too small and the request rate is high, the kernel packets may be dropped, leading to lost SNMP trap requests or responses. To optimize NETSNMP and the UDP kernel parameters in high-performance or high-traffic environments, adjust the SNMP trap interceptor’s settings and the underlying kernel UDP’s buffer settings.

    All the best.