Using Product Evaluation to Monitor an Asphalt Mix

In previous blogs we have discussed the benefits of the Stonemont KPI and the Mix Risk analysis tool.  Mix Risk uses the Stonemont KPI to provide an evaluation of potential future performance of a mix design prior to or during production of the mix.  However, the Stonemont KPI also can be used to analyze production results directly and this is done using Product Evaluation.  Remember that the original mix design meets the specifications and is generally close to the target values.  For the purposes of this example, we are assuming that the design was put into production without the benefit of prior information provided by the Mix Risk analysis.
Product Evaluation
Figure 1
Figure 1 shows the results of a Product Evaluation for a portion of the production run for this mix.  Notice that this evaluation considers properties that are deemed important for this particular asphalt superpave product (AC Content, Air Voids, and VMA).  If this were a concreteproduct these properties would differ than those shown.  These different properties can easily be assigned to each product as critical parameters in our software. 
One of the most important aspects of the Product Evaluation tool is that it not only provides you with the statistics necessary to make an informed decision regarding the performance of your products but it includes a textural definition of issues that the statistics are revealing.  It is clear from this Product Evaluation that we have issues regarding specifications being inside two standard deviations.  However, the Product Evaluation will also identify possible trends or changes in mean,  distributions that are very off center, large discrepancies between measured and predicted failures, last values out of specifications or limits and rounding issues possibly indicative of data falsification.  These textural definitions help you quickly identify and understand potential issues.  This is in contrast to having to manually scan the data and understand each statistic well enough to identify potential issues.
Figure 2
Figure 2
Let’s consider air voids (Va@Ndes) in more detail because it has some interesting characteristics.  The run chart shown in Figure 2 is the same data set of air voids that was used for the Product Evaluation.  Notice that run chart shows not no obvious trend and no failures (points out-of-specification) for air voids, which is consistent with the value of 100% for CTS (Conformance To Specification) shown in the Product Evaluation.  The Product Evaluation shows a PWS (Percent Within Specification) value of 95.2%.  A PWS value above 95% is generally regarded as sufficiently high enough probability of future conformance.  The Ppk value of 0.56 indicates that the specifications are inside two standard deviations, which is generally a situation to avoid if trying to meet 95% compliance.  The Ppk/Pp value of 0.62 indicates that the mean of the data is quite off-center relative to the specifications.
These four values are used in the calculation of the Stonemont KPI, which was 93%.  A KPI value of 93% (below 95%) does provide an indication of future risk of non-conformance to specifications and could be regarded as an early warning of potential future problems.  Furthermore, this is an example where simply looking at individual sample results regarding pass/fail would not have provided a clear understanding of the risk since no air voids samples failed relative to specifications. 
In Figure 3, we have added more samples to the Product Evaluation and we can see that PWS and KPI have lowered to 93.2% and 90%, respectively.
Figure 3.
By the end of the production run (Figure 4), the air voids PWS and KPI have dropped to 91.5 and 88, respectively.  The run chart of air voids (Figure 5) shows that we had 5 failures following our initial Product Evaluation. This example demonstrates that although we didn’t have any previous air void failures that the risk or probability of non-conformance identified by the Product Evaluation was real and should have been addressed.  The Product Evaluation results also shows that other critical parameters, including the ½” sieve, are sources of potential risk.  Recall that the Mix Risk analysis we performed identified an increased risk of producing non-conforming material due to the ½” sieve.  


Figure 4.
Figure 5.
Early Warning System
A very powerful feature of Stonemont Software is the ability to have Product Evaluations run automatically (Enterprise/Hosted Editions) so that they can serve as an “early warning system.”  This can be done either on a product basis using the Auto Analysis capabilities or on a plant basis using Auto Evaluation.   They can be run daily, weekly, or monthly (or hourly in Version 7) and automatically emailed to the appropriate personnel.  
The Auto Evaluation can take advantage of user-defined triggers as to whether or not a potential issue exists.  For example, a trigger of 95% can be placed on KPI, meaning that if any of the parameters being included in the analysis for that product are below 95% then the product will be included in the auto analysis report.   In our example using air voids, the KPI fell below 95% on June 8th so we could have been alerted to the potential issue prior to any actual failures.  Although the auto evaluation can be run for every product at every plant it can be setup so that only those products that indicate a potential issue will be reported.  This reduces the amount of data that a quality manager must filter through to identify those products that require their attention. 
Hopefully we have shown the importance of using both Mix Risk and Product Evaluations that incorporate the Stonemont KPI .  Mix Risk will help identify potential issues in the mix design stage prior to them becoming problems during production.  Product Evaluations will monitor the product performance during production hopefully prior to them becoming customer issues.  
James Beal
Adrian Field
Stonemont Solutions, Inc.