This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

How do you compute standard deviation from the load test raw data?

Dear LoadRunner Cloud community, 

I am trying to create a report identical to the one generated after a load test in the REPORT pane beside the DASHBOARD. I have managed to identify how the average/min/max trt are computed, as well as failed and passed transactions. 

However, I am still trying to crack how the standard deviation is computed.

Would anyone know how to aggregate the data for Standard Deviation?

EDIT
these are what I have tried so far (per transaction):

  • calculating   for the whole timeseries where
    • w = weight
    • xi = average trt value
    • x-bar = weighted mean
    • M = the number of non-zero weights
  • calculating the same as above per time_stamp, and then getting the mean of calculated values throughout the timeseries
  • getting the weighted covariance of the average trt values, and then getting its square root (just using library functions in Python)

 

Regards,

Leif 

PS If anyone is curious about the other metrics:

  • the average trt for a transaction through the whole timeseries is a weighted average of all the the transaction average trt values--the weights being total_duration/val
  • min trt and max trt come respectively from the minimum and maximum values of the min trt and max trt metrics in the val column
  • passed transactions and failed transactions are cumulative sums over the timeseries 
Parents
  • Verified Answer

    +1

    UPDATE

    Every average TRT value in the csv file (and the dashboard) is an aggregation of the transaction response times of multiple Vusers (given then you are running multiple Vusers in the first place). I suspect that the standard deviation in the report is computed alongside the TRT AVG. Since these individual Vuser response times are not available in the csv, it is not possible to compute the standard deviation thence.

    To elaborate

    If we did have data and graph of individual TRTs for each Vuser, the graph would look something like this (hypothetical):

     

    In the dashboard, the TRT MAX and TRT MIN graph will come from the max and min values recorded from individual Vusers before they are aggregated per timestamp, i.e. the blue (max) and green (min) data points. 

     

    In the transactions report, the MAX value is the maximum value recorded among TRT MAX, and the min value recorded among TRT MIN. Each timestamp will then have an aggregated TRT AVG, the average of which is the AVG in the transactions report. Each TRT AVG will also have a corresponding standard deviation, show as vertical bars in the illustration below:

     

    MIN, AVG, and MAX in the report would be 2.029, 2.48, and 2.971 respectively. I suspect the standard deviation in the transactions report is also an average, which for this example = 0.289.

     

    So in the csv, for response times, only MAX TRT, MIN TRT, and AVG TRT is given per timestamp. Since the AVG TRT is already an aggregated number (from individual Vusers), it is not possible to calculate the standard deviation as shown in the report using the csv. 

Reply
  • Verified Answer

    +1

    UPDATE

    Every average TRT value in the csv file (and the dashboard) is an aggregation of the transaction response times of multiple Vusers (given then you are running multiple Vusers in the first place). I suspect that the standard deviation in the report is computed alongside the TRT AVG. Since these individual Vuser response times are not available in the csv, it is not possible to compute the standard deviation thence.

    To elaborate

    If we did have data and graph of individual TRTs for each Vuser, the graph would look something like this (hypothetical):

     

    In the dashboard, the TRT MAX and TRT MIN graph will come from the max and min values recorded from individual Vusers before they are aggregated per timestamp, i.e. the blue (max) and green (min) data points. 

     

    In the transactions report, the MAX value is the maximum value recorded among TRT MAX, and the min value recorded among TRT MIN. Each timestamp will then have an aggregated TRT AVG, the average of which is the AVG in the transactions report. Each TRT AVG will also have a corresponding standard deviation, show as vertical bars in the illustration below:

     

    MIN, AVG, and MAX in the report would be 2.029, 2.48, and 2.971 respectively. I suspect the standard deviation in the transactions report is also an average, which for this example = 0.289.

     

    So in the csv, for response times, only MAX TRT, MIN TRT, and AVG TRT is given per timestamp. Since the AVG TRT is already an aggregated number (from individual Vusers), it is not possible to calculate the standard deviation as shown in the report using the csv. 

Children
No Data