File permissions and rights

Hello, we are trying to use a program called duplicati to backup netware volumes. Duplicati seems to have a problem walking the directory trees of netware volumes. We have been unable to determine why but it seems that the OES client hands back extended file status which breaks Duplicati. This is on a netware 6.5 SP8 server. 

I am hoping that an old NETWARE guru may have the answer to the following.

1) What is netware doing that is messing up tree walking and/or is there an issue where volumes span several segments or drives?

2) Presuming we can't fix this, what backup programs exist that allow us to completely backup this server?

3) If we upgrade to Open Enterprise Server, how can we migrate data from the old netware server to an OES box given that it is OES on top of Linux?

This server is Netware 6.5 SP8. The client is Client for Open Enterprise Server 2 SP7 (IR3), Duplicati is located here: www.duplicati.com and they have entered a thread on this topic in their support forum.

Thank you! 

Rich D.

  • 0  

    On the machine hosting Duplicati, with the same logged in credentials, can you readily navigate to through those directory trees with Windows Explorer?
    There can be issues where the path can become too long to be navigated by Windows with its 256 character full file name limit you might be hitting.  Something many have hit many times in the past, with the problem being on the Windows side that is being worked on, but there is still current software that trips over it even when the rest of the system is up-to-date. 

    What version of Windows is hosting Duplicati?  There are much newer OES clients available.

    Is NetWare directly on physical server, or has it been virtualized yet? 

    How big is the data being backed up?  Both in file size and number of files. It appears that Duplicati has some issues on scaling on that front.

    Current OES doesn't have the migration tools that actually work any more given how far out of support NetWare is (at least you are on the last version of it), but there are methods available that others have written about here, and we can help when going that direction.

    I have used rsync as a method to effectively sync data off of NetWare and beyond. Given how long that NetWare has been out of support, I don't know if there is anything left with direct support.  rsyncing to an intermediate storage and other backup system to pull off of that is an option.  Doing that would be another test for Duplicati if you are hitting its limits.

    Spanning segments and drives on NetWare is invisible to the client, with one possible exception, of performance.

    How are you handling Rights/permissions backup?  

    ________________________

    Andy of KonecnyConsulting.ca in Toronto
    Please use the "Like" and/or "Verified Answers" as appropriate as that helps us all.

  • 0 in reply to   

    Thanks Andy.

    1) Yes, we can traverse the directory tree with issue though the client.  The duplicati team uses .net libraries to access the tree and apparently read the data and then determine if it is a file or folder. Seems backwards to me but that is what they do and I think there is extended status that comes back and gums up the works but I can't confirm that. I do see that the file read takes place. I have logs.

    2) Yes, we have tried the OES client 24.4 and no change.

    3) Should I use teracopy or similar to copy files? Is Rsync part of netware 6.5?

    4) What about current OES? You mention netware it out of support but OES 25 seems to be in current support and it supports NSS volumes so I would install a LW SCSI controller and the drives and migrate data from them to new drives in the new server. I guess this is the old keep it on netware versus move to MS Server with the File Server package installed. I like the reliability of netware.

    5) Rights/permissions, we have supervisory for all for about 300GB of data. All users have the same so I doesn't matter about rights and permissions from our side. I can keep getting new LW SCSI  drives and can keep on this hardware forever but the lack of good backup is making me crazy!

    Thanks!!

  • 0   in reply to 

    1) how old are the .net libraries used? Which version of duplicati have you tried? and on what Windows version. 
         Could you backup just a small part of the volume, such as just one folder with a couple others inside?

    2)  but what Windows versions?  The support for greater than 256 has been released in parts, starting part way in Win10, and still progressing in Win11

    3)  I'm not familiar with teracopy, so can't say how it would work. I found robocopy keep failing on anything of a server scale.  certainly options to try to get to a point something like duplicati could use to manage having sufficient copies in different location.(3-2-1 rule of backup)
    An older version of rsync.nlm may be in SYS:\SYSTEM and you would need to setup an rsync target. built into Linux, and even many NAS. Windows options are readily available.   Then newest/last rysnc for NetWare (2.6.3) is available online in a few places, and I have a copy if needed. 

    4) Yes, NSS very much supported. As for just reattaching the drives, that's a risky move. I would set up the new server in parallel in the same tree, then rsync the data over.   Then a bunch of fiddly bits to take care of to make sure everyone can use them on the new location on a switch over point in time, where you dismount the drives on the old box and everyone just as to restart for a clean reconnect.

    5) small shop openness, relaxing in someways, but given the increasingly hostile environment (many bad actors with automated tools hit Everything/Everywhere) looking at ways to tighten things up.  All staff Security Awareness training covers so many of the common vectors aimed at social engineering.

    ________________________

    Andy of KonecnyConsulting.ca in Toronto
    Please use the "Like" and/or "Verified Answers" as appropriate as that helps us all.

  • 0 in reply to   

    Thanks Andy.

    1) latest .net, windows 10 H2 pro. Have tried smaller folders and have still not isolated what happens. This is the note from the developer.

    Hi, thanks for the logs. I have been examining the code base+logs and from what I can see, all the errors are handled as gracefully as possible. The only issue is the setup where the code calls EnumerateFileSystemEntries once, so we get back both files and folders.

    I think this is the optimal way to perform the call, but it has the downside that we need to obtain attributes on each entry afterwards to figure out if it is a file or folder. In this situation, the failed attribute call has the effect that we do not traverse the folder as it is misclassified.

    I still do not understand why the attributes fail to be read, but perhaps there is no good way to solve that.

    I am currently putting the finishing touches on a new feature that allows sources other than local files. With this feature, it will be possible to re-purpose the CIFS backend as a source-provider and read files directly via the CIFS protocol, which has more features than what the file-system driver allows.

    Because of the issue with attribute failures, I am also isolating the choice for how the enumeration is done into a single place so we can easily choose which to use.

    tl;dr: The missing folders problem should be fixed shortly.

    .

    duplicati-2.1.0.4_stable_2025-01-31-win-x64-gui.msi

    Regarding 5: We put it behind our netware server behind a firewall and do not let it live on the web..ever! :-) that said thanks for giving me another sleepless night! :-)

  • 0   in reply to 

    Interesting.  Hopefully this means a working solution soon.  Though, the assumption of a CIFS back end is likely not accurate if you are using the OES client, because it uses NCP as the native NetWare/OES file access protocol, and adding the CIFS ability to NetWare/OES is an extra range of steps.   Hopefully one of the OES Client 2nd+ tier support or devs sees this and can comment.

    Firewalls are a good thing, but not the only part of it. 
    If you want to challenge those sleepless nights, attend some InfoSec gatherings. It will hit you both ways with the ways into an environment and how to shield them.  If you were in/near Toronto, I'd say join us at TASK.to, though for you CharmSec is likely easier to get to.

    ________________________

    Andy of KonecnyConsulting.ca in Toronto
    Please use the "Like" and/or "Verified Answers" as appropriate as that helps us all.

  • 0  

    >> 1) Is there an issue where volumes span several segments or drives?

    Did you mean a pool spanning several segments/drives, or do you have a DFS junction that helps you span a volume across multiple pools?

  • 0 in reply to   

    No DFS here. Just a NSS pools spaning multiple drives. Thanks.

  • 0 in reply to   

    Thanks. Yes, we look at multiple attack vectors. Thanks.

    I have an followup question. We have one file that is corrupt on a drive. It is an NSS volume so Vrepair does not work. What tools work on NSS volumes? Thanks!

  • 0 in reply to   

    Many weeks back I tried the CIFS access to the server and same errors with duplicati!  I thought I had found a solution but it is a problem with their code!

  • Verified Answer

    0   in reply to 

    You can use "nss /poolverify=<poolName>" to check if there is a corruption in the file system (see: https://www.novell.com/documentation/nw65/os_util_nw/data/bw7ki1s.html). If verify reports corruption then you can use 'nss /poolrebuild=<poolName>' (see: https://www.novell.com/documentation/nw65/os_util_nw/data/h4uk0rz1.html).

    How do you know that you have a corrupt file? Is NSS deactivating the pool when you read that file?

    Note that the "rebuild" may not be able to fix a corrupt file. In that case, it may simply throw away the corrupt file to keep the rest of the pool healthy. Further, NSS can only fix the metadata, not the data. If the file user data (file contents) is corrupted then that cannot be fixed as NSS does not know anything about user data.