I'm indexing a set of assorted documents from a database using the ODBCConnector, but somewhere in the process it is generating additional documents when the document contains HTML.
How do I prevent this happening?
Cybersecurity
DevOps Cloud
IT Operations Cloud
If an answer to your question is correct, click on "Verify Answer" under the "More" button. The answer will now appear with a checkmark. Please be sure to always mark answers that resolve your issue as verified. Your fellow Community members will appreciate it! Learn more
I think I found the source of the problem, and it was user error ( i'm new at this and it was a painful experience! ).
There was a "post" lua script that was parsing the document content and removing URLs, but it always just ended, never returning true or false.
If the document was empty I added a return false and that seems to have prevented the un-necessary documents being created.