NA 2023.05 Linux lost of ssh connection when trying to pull device configuration

I'm having an issue with a new deployment of NA 2023.05 on RHEL 8 VMs. Error message:

NA 2023.05 receives error: Attempt to retrieve data from device failed: Task thread was interrupted. When the snapshot starts it connect using the svc_na account using SSHv2 port 22. The account logs in and connects with the password. The device can be accessed and it's building the configuration and it fails when retrieving the configuration. We are also having problems finding on the server where the session logs are located to analyze more information. The driver packs have been updated to the latest version.

 Thanks,  Jim

  • 0   in reply to   

    Had tried to message you but no luck - didn't seem to work.  If you can, see if you can send me a message, I have some other suggestions.  

  • 0 in reply to   

    I'll run through this. We are also seeing on some of the snapshot fails that the Task did not complete, Task thread was interrupted

  • 0   in reply to 

    OK, hopefully you will find the missing piece.  

    One more question - are the new VMs using the same IPs / hostnames?  

    Good luck!

  • 0 in reply to   

    Each VM has it's own IP address and sits in the same subnet as the local network devices. any way you can contact me outside this chat?

  • 0   in reply to 

    Yes, email me at bob_lithe.0v@icloud.com

  • 0 in reply to   

    I can be reached at james.stocks@invictusic.com

  • 0 in reply to 

    Not sure exactly what the issue may be… as certain makes and models of devices are taking successful snaps and some are not.  So I don’t know if this is a database issue or not.  I am noticing that all Juniper (varying model and software versions), all Cisco ASR’s (varying model and software versions), and all Cisco Nexus’s (varying models and software versions) are having this issue.

     

    Device models such as Cisco 3560’s, 3750’s, 3850’s, 7606’s, etc. are working fine.

    Thanks

    P.S. I wonder if these devices need a new iOS image update?

  • 0 in reply to 

    Latest update: I had to build out an emergency NA server with embedded Postgres to get operational. Still working on the current server issue. One thing viewed in the logging during the ssh connection is NA connects, builds the configuration and dies during the "get" command while trying to pull the configuration back. Wondering if the get command dying in the ssh session could be something in the device configuration? Other device snapshot connections work pulling the configuration with the get command. Still seeing this issue with the Nexus and ASR devices. 

  • 0 in reply to 

    Issue continues with get command failing during snapshots. 

    Here's the issue from a session log showing the error:
     <h4> Retrieve running configuration via SNMP/SCP</h4>
     Error while retrieving running configuration : SCP server may be down or unreachable.
     <h4> Retrieve running configuration via SNMP /TFTP</h4>
     Error while retrieving running configuration:  TFTP server may be down or unreachable.
    <b> Error: </b> Failed to retrieve configuration.
    &nbsp;&nbsp;&nbsp;<b>SCP</b> failed Failure executing SCP command 'get': com.maverick.ssh.SshException: java.io.IOException]
    &nbsp;&nbsp;&nbsp;<b>SCP</b> failed [Error opening system:/running-config (No such file or directory)]
    &nbsp;&nbsp;&nbsp;<b>FTP</b> failed [Error opening system:/running-config (No such file or directory)]
    &nbsp;&nbsp;&nbsp;<b>TFTP</b> failed [Error opening system:/running-config (No such file or directory)]
    &nbsp;&nbsp;&nbsp;<b>CLI</b> failed [java.lang.UnsatisfiedLinkError: /opt/NA/server/ext/wrapper/lib/libjavaperl.so: libnsl.so.1 wrong EFL class:
    ELFCLASS32
    &nbsp;&nbsp;&nbsp;<b>SCP/SNMP</b> failed [An error occured while using the SNMP /SCP access protocol.]
    &nbsp;&nbsp;&nbsp;<b>TFTP /SNMP</b> failed [An error occured while using the SNMP /TFTP access protocol.]
    <b>Send:</b> exit\r
    </span>
      
    Thanks,