Sentinel Reindex finished ok, but no new data is searchable, reindex needed though

Hi,

we have Sentinel 8.6.1.1 and we brought lots of logdata into it when installation was finished. Most of the indexes are just fine and data is searchable. We still have several thousand indexes/partitions that we haven't been able to get  searchable, Sentinel finishes them ok, no errors, Outcome: COMPLETED, total number of indexes same as total number of succesfull partitions. That imported data is same as the ones that succeeded but just don't end up searchable and after a while Sentinel announces that "X Event Data partitions are currently now searchable. Click Event Partition Administration to resolta." I do just that and yet, once again same, no new searchable data, but once again, same announcement.

What is wrong when Sentinel can't finish those indexes/partitions as searchable? I have tried both offline and qui based reindexin tools. No difference in outcome. I have tried to find a file or something that would hold the info about successfully integrated partitions, but now luck. Even PostgreSQL doesn't hold much more info that reindex0.0.log. 

I really appreciate any ideas and help that might get us through this cointinuum - we have fought with this already for several months now. Thank you!

Kind Regards,

Timo Salmi / Finland

  • 0

    To give more information: 

    After we have imported our data to our Secondary storage, Sentinel detects them and suggests that we reindex them. And that's what I do. After reindexing I need to go to Storage -> Events and  Find Data. I manage to find all the newly indexed partition which I select and Restore.  I get the following events into the server0.log. Indexes/partitions are located on our secondary storage:

    Tue Nov 12 09:45:02 EET 2024|INFO|ExportEventAssociations-Partition-127289|esecurity.ccs.comp.event.indexedlog.ExportImportEvtAssocThread.run
    Starting ExportEventAssociations for partition 127289
    Tue Nov 12 09:45:02 EET 2024|INFO|ExportEventAssociations-Partition-127289|esecurity.ccs.comp.event.indexedlog.ExportImportEvtAssocThread.importData
    Import directory /var/opt/novell/sentinel/data/eventdata/exported_associations/20230903_D43B9039-ED96-103C-AFF2-005056B90CF4 for partition 20230903_D43B9039-ED96-103C-AFF2-005056B90CF4 (ID=127289) does not exist. Could NOT import event association data.
    Tue Nov 12 09:45:02 EET 2024|INFO|ExportEventAssociations-Partition-127289|esecurity.ccs.comp.event.indexedlog.ExportImportEvtAssocThread.run
    Completed ExportEventAssociations for partition 127289. Took 8 ms

    After restore I get a screen via what I could Apply those newly reindexed and found partitions. Restore data windows tells me that there's Error Importing and info says (this is an example):

    Partition Name: 20230711_D43B9039-ED96-103C-B035-005056B90CF4_20240416
    Partition Date: 2023 July 11
    Event Count: 8
    Byte Count: 14.29 KB
    Event Association Data Action: Importing event assocation records into database
    Action Start Time: 2024 August 27 14:52:20 UTC+3
    Action Running Time: 00:04:51.585
    Event Association Data in DB: No Records Restored
    Event Association Data Exported:YES
    Incidents Events Exported: 0
    Correlated Events Exported: 0
    Incidents Events Imported: 0
    Correlated Events Imported: 0
    Action End Cause: Import files not found on file system.

    After applying selected partitions, they disappear of course, but are not searchable. I don't know if it matters, but reindexed partitions are still mounted during this restore process even though they would have been indexed day or two earlier. 

    Regards, Timo

  • Verified Answer

    +1 in reply to 

    Hi, answering to my self again: problem solvet:

    Sentinel says this about Data restoration (storage -> Events after Retention section):

    Data Restoration

    To restore data, copy the event data directories to be restored into the primary event data directory (/var/opt/novell/sentinel/data/eventdata/events) or the secondary storage directory (Configured above), then click Find Data to get a listing of all restorable data. Event data directories must be re-indexed for it to be restorable. Kindly navigate to Storage >> Event Partition Administration for more information.

    We have made a massive work, built new systems, installed Sentinel and imported many many Teras of log data. That note worked partly during the process. Meaning: only some most recent data (and we upgraded from 8.6) were found by that way. Majority of out logs just were "found" by Reindegin tool (GUI). That was just fine and after reindexin data was ok. I had to add some more logdata after that massive import and this was totally different with info text above. I now know, so one has to import logdata into primary or to secondary, Reindex all the data first and then come here to click Find Data -> it will propably give an error ( Action End Cause: Import files not found on file system.) but nevertheless just Apply and data newly reindexed data is searchable. Note to Opentext: this propably need to be fixed or otherwise informed. I would have save 2 months, but the most important things is that finally found the working process.