Using Error Analysis

Select the component whose approximation you want to visualize.
- From the Design Gateway,
  - Select a component on the Sim-flow tab or in the model explorer, and click the Approximations button on the component title bar.
  - Right-click the component on the Sim-flow tab or in the model explorer, and select Approximations.
  The Approximations dialog box appears.
  1. In the Activated column, select the approximation that you want to analyze, and click Error.
    The Approximation Viewer dialog box appears.
    Tip: You can also access this interface from the Approximation Wizard after initialization by clicking Error. For more information, see Initializing Approximations and Viewing the Data.
- From the Runtime Gateway,
  1. Select a component on the Sim-flow tab or in the model explorer.
  2. Click the Visual Design tab, and select the approximation from the list next to the component title bar.
  3. Click the Error Analysis subtab.

From the Error Type list, select the desired error type.

Option	Description
Average	The differences between the actual (simulation process flow execution) and predicted (approximation model execution) values for all error samples are averaged and then normalized by the range of the actual values for each response. Therefore, the value is a fraction of the response data range for the error sample points. Normalizing the error value allows the error level of different responses with different magnitudes to be compared with respect to the quality of predictions in the approximation model. The average error is calculated as follows: $A v e r a g e_E r r o r_N o r m a l i z e d = \frac{A v e r a g e [A B S (A c t u a l - P r e d i c t e d)]}{M a x_A c t u a l - M i n_A c t u a l}$ where $A c t u a l$ is the observed value, $P r e d i c t e d$ is the approximated values, $A B S$ is the absolute value, and $A v e r a g e$ is the average value.
Maximum	The maximum difference between the actual (simulation process flow execution) and predicted (approximation model execution) values for all error samples is taken and then normalized by the range of the actual values for each response. Therefore, the value is a fraction of the response data range for the error sample points. Normalizing the error value allows the error level of different responses with different magnitudes to be compared with respect to the quality of predictions in the approximation model. The maximum error is calculated as follows: $M a x i m u m_E r r o r_N o r m a l i z e d = \frac{M a x [A B S (A c t u a l - P r e d i c t e d)]}{M a x_A c t u a l - M i n_A c t u a l}$ where $A c t u a l$ is the observed value, $P r e d i c t e d$ is the approximated values, $A B S$ is the absolute value, and $M a x$ is the maximum value.
Root Mean Square	The squared differences between the actual (simulation process flow execution) and predicted (approximation model execution) values for all error samples are averaged. The square root is taken, and the result is normalized by the range of the actual values for each response. Therefore, the value is a fraction of the response data range for the error sample points. Normalizing the error value allows the error level of different responses with different magnitudes to be compared with respect to the quality of predictions in the approximation model. The root mean square error is calculated as follows: $R o o t_m e a n_s q u a r e_E r r o r_N o r m a l i z e d = \frac{R M S D}{M a x_A c t u a l - M i n_A c t u a l}$ where $R M S D = \sqrt{\frac{Σ_{k = 1}^{n} (A c t u a l - P r e d i c t e d)^{2}}{n}}$ where $n$ is the number of points, and $R M S D$ is the root_mean_square deviation of the error.
R-Squared	The coefficient of determination is calculated based on the error samples. The coefficient of determination always ranges between 0 and 1, where 1 represents a perfect fit (or no prediction error). The R-squared error is calculated as follows: $R^{2} = 1 - \frac{S S R}{S S T O T A L}$ where $S S R = Σ_{k = 1}^{n} (A c t u a l - P r e d i c t e d)^{2}$ and $S S T O T A L = Σ_{k = 1}^{n} (A c t u a l - A v e r a g e (A c t u a l))^{2}$ where $n$ is the number of points, $S S R$ is the sum of the squared residus, and $S S T O T A L$ is the total sum of squares.

Error analysis types are implemented as plug-ins. Therefore, they are extendable by creating new plug-ins for new error analysis techniques. For more information on creating plug-ins, see Creating a Plug-In in the Isight Development Guide.

In the Acceptance Level text box, enter the acceptance level for the selected error type.

The acceptance level defines the cutoff value for the error type that distinguishes a response with acceptable fit (acceptable approximation quality) from a response with unacceptable fit (unacceptable approximation quality).

For the Average, Maximum, and Root Mean Square error types, low values are desired. The Acceptance Level is an upper limit. Reported error values greater than the acceptance level are flagged in red in the table of responses and in the plots as unacceptable approximation quality. The default Acceptance Level is 0.2 for the Average and Root Mean Square error types and 0.3 for the Maximum error type.

For the R-Squared error type, high values are desired; therefore, the Acceptance Level is a lower limit. Reported error values less than the acceptance level are flagged in red in the table of responses and in the plots as unacceptable approximation quality. The default Acceptance Level for the R-Squared error type is 0.9.

Select any of the tabs in the upper right to view the plots, as desired:

Option	Description
Response Fit	Select Response Fit to view actual versus predicted response values for each response.
Residual	Select Residual to view the difference between the actual and predicted values for all error sample points for each response.
Residual Frequency	Select Residual Frequency to view the residuals (the difference between the actual and predicted values for all error sample points for each response) as a frequency of occurrence, from 0 to the maximum residual.
Total Error	Select Total Error to view the total error for all responses in a bar chart.

To view one response residual plot in greater detail, do one of the following:
- From the Show list, select the response that you want to view full size. The graph appears in full size on the tab. To return to the thumbnail display of all the responses, select All Responses from the Show list.
- Double-click the response plot that you want to view full size. The graph appears in full size on the tab. To return to the thumbnail display of all the response plots, double-click the plot again.
For more information about how to interpret the displayed results, see Error Result Interpretation.
If you are working in the Design Gateway, click Close to exit the Approximation Error Analysis dialog box and to return to the Approximations dialog box.