Using sqlserver and importing very large FPR's into SSC (~1.4Gb) it's been noted that processing time can take, on average, around 280 minutes. After investigating it was discovered that the primary reason for the delay was some extremely high forward fetch counts on the `sourcefilemap` table due to improper indexing. It was noted in our production environment the forward fetch count of over 63 million on the sourcefilemap table, when the heap size was nearly over 200Gb.
This seemingly can be resolved by
- Dropping the SOURCEFILEMAP_CHECKSUM_IDX as it's the source of the extremely large heap, and in our production environment has about 1 reads to every 17 million writes (meaning it wasn't used and did nothing but add overhead)
- Making the `SourceFileMapScanIdFileNameInd` index clustered
For us, this reduced FPR processing time from ~280 minutes to ~17 minutes.
The question is, why does said checksum index exist.