Decrease in backup speed after change from HP DP 09.09 to DP 24.1

Hi,

We have made the change from HP DP 09.09 to DP 24.1. File archiving is carried out in several branches controlled from headquarters. 


In one of the branches there is an LTO7 autoloader - Quantum Super Loader 3, connected to barmetal server with Windows server 2019.
In the old version of HP DP was in the same configuration of backup and Autoloader, data backup of about 4.3 TB took about 8 hours. Now after changing to the new version of HP DP version 24.10, the same amount of data takes about 80 hours. This is 10 times slower.

We did update of firmware of Autoloder to version 106, but IBM tape drive is still in version M571 becouse update still give error: Bad Code Update Image Data: Comunication error (code: 0x71)

We upgrade drivers for windows server 2019 for drive and autoloader.  

After it timing went to 64 hours. At other branches but with diffirent Autoloader has no problems. 

 

What could be the reason? What else can I check? We have not changed anything in the network configuration, etc. this situation occurs immediately after the change. 

  • 0  

    So the problem is seen in one of the branches only. Something which is maybe worth testing is to run the same backup to a null device. Create a null device on the same Windows media agent, make a copy of the backup specification, replace the devices by the null device and run a test backup. This may give some idea about the location of the bottleneck. The test backup to the null device should be much faster. If that is not the case then the bottleneck most likely isn't located at the devie itself.

    Is there any (software) compression or encryption involved at the disk agent side? This can be seen in the Filesystem options for a file system backup.

    And to state the obvious: if this is only seen with one of the branches then what else can be different specifically with this one compared to the others. I understand nothing was changed, but somehow something has to be different. 

  • 0 in reply to   

    Hi Koen, 

    Thank You for quick answer ans suggestions. 

    I made test with null and it is successfull. So fast.  5 minutes and 190 GB. 

    Responding to another of your suggestions/responses: 


    "Is there any (software) compression or encryption involved at the disk agent side? This can be seen in the Filesystem options for a file system backup."

    No. it is unchecked. We backup via DP, files from baremetal server to which are connected Autoloader and some from slected VMs.

    I have made a notification to Quantum regarding problems with the IBM Tape drive firmware update to rule out this problem.

    Although the settings are the same as in the old HP DP it puzzled me that as I was creating the null device the segment size settings are different than we had  in HP DP and have set for actual DP the Tape Drive.
    We have a Segment size of 2000 MB for the Tape Drive, in the test for the null it is automatically set to 10000 MB. Could this have an impact on our problem?

    Block size is identical 256 kB and Agent buffers 8.

  • 0   in reply to   

    You could run the same null device test with a segment size of 2000, but I doubt this will have any visible impact.

    So overall this test is confirming that you realy have to take a look at the device side ... the device itself, driver, interface, cabling, .... Strange that this is happening right after the upgrade. Has the hardware possibly been touched at the same time in any possible sense?

    As blocksize and buffers are identical and you have proven that the complete infrastructure is handling the load well with the tape library excluded, I have no immediate ideas currently.

  • 0 in reply to   

    Let me add one more thing. This device is also used by Veeam, whose service is installed on the server to which the autoloader is connected. On it, it performs archiving to tape awithout any problems and the time has not increased.

    Does DP have any additional configuration options in the ini cfg files or in the registry? I'm asking because I'm wondering if anything else can be set on the server to which the autoloader is connected besides the GUI.

  • 0   in reply to   

    Nothing which could have any influence as far as I'm aware. You can check registry keys in HKEY_LOCAL_MACHINE\SOFTWARE\Hewlett-Packard\OpenView\OmniBackII.

  • 0   in reply to   

    Is the path between the Veeam service and our media agent identical? Host, drivers, interface, FC infrastructure, ..... is it 100% identical or are there any differences?

  • 0   in reply to   

    Besides all this it may be interesting to see some older (9.09) and newer (24.1) DP session reports. Also the library and device configuration details (use omnidownload command) and maybe also a Veeam report, so we can compare everything and possibly see any differences. Oh ... a registry dump may also be good so we can dig into that if needed.

    I guess it would be best to provide all this through the FTS in the case you have (I was made aware of that case) and give me a sign when that has been uploaded.