Hm ok.

Im having some trouble wrapping my head around how the driverconfiguration actually works..

Im on the publisher channel in "poll"-mode so data flows into the IDM i get a response body from the pull wich i work on and thats all fine, but when should the driverconfiguration regarding paging actually do its job? seems to me that it should "work on its own" without me tampering with the xml dokument (other than getting my data out). 

And if i were to create a query in my own code, disregarding the driverconfiguration, how would it travel alle the way around to the subscriber channel and make a new poll/query?

Does anyone have a document with either the driverconfiguration working or a modified document with a query, that I can replicate in my driver?

Not for the publisher, only subscriber, but I have shared a tiny bit of how I did it: https://github.com/casperghst42/idmsnippets/tree/main/RESTpagination - my way of doing it probably just one out of hundreds (and the the most complex one).

Using the publisher pull, should also be possible. Why not set the trace level as high as possible (99), let the driver do a pull and see how the http headers look like - that might help you in figuring out how to configure the pagination part.

ChrisKnudsen said:

When does the next page "happen"?

In your case, I believe this happens in the driver shim itself. So you cannot intercept and manipulate it in policy.

Your driver-parameters look to be correct given your example.

Possibly stating the obvious, but applying the driver package is only half of the picture here.

You also need to update the driver shim to a version that knows how to deal with the pagination parameters. (preferably use the absolute latest shim available). For easier testing and debugging suggest you use a dedicated remote loader instance for this shim.

If you can, post a more detailed driver-side trace that includes driver startup and first poll. Set driver trace level to 99.

Geoffrey Carman said:

THINK the shim is expecting your JSON->XML to spit out an <instance> doc and probably with a <query-token> node specifying the next node

Polling functionality is all self-contained within the shim from what I understand.

The JSON->XML translation always happens on the engine side, not in the shim.

What is confusing, is that like the AD driver, the default engine side policies call out to a java class that is normally bundled with driver shim updates. When per best practice, one runs the driver shim on a separate remote loader, it is far too easy to forget to update/patch/install this java class on the engine side, which can cause engine side policy evaluation to fail.

Now don't get me started on how broken and poorly written the built-in JSON->XML implementation that ships with the REST driver is. One of the first things I do when working on a REST driver is to replace this with a less broken and more flexible approach (several options exist).

Yes that is the case, thanks for pointing that out.

Unfortunately the customer is running IDM 4.7.3 - and the shim does not seem to be compadable?

Well current maintenance for 4.7 ended in 2020.

You might be lucky and get it to work by running the REST driver in a 4.9 remote loader.