The discussions and examples regarding how to analyze data and develop acceptance plans that are presented in this and the following chapters will help an agency to decide how much sampling and testing it believes is economically justified for its particular situation. As part of the acceptance procedures and requirements, one of the first decisions that must be made is "Who is going to perform the acceptance tests?
The agency may decide to do the acceptance testing, may assign the testing to the contractor, may have a combination of agency and contractor acceptance testing, or may require a third party to do the testing. The decision as to who does the testing usually emanates from the agency's personnel assessment, particularly in days of agency downsizing.
However, the lack of personnel availability by the agency should not be the sole reason to decide to use contractor acceptance testing, even though this has often been the case. In fact, agencies have sometimes found no significant decrease in agency personnel resulting from the use of contractor acceptance testing.
Also, if an agency adopts contractor acceptance testing solely to reduce the agency's staffing needs, then the agency is less likely to follow all the steps, such as developing appropriate validation procedures and conducting pre-implementation training, necessary to successfully implement the QA specification. Furthermore, contractors should never be assigned the responsibility for acceptance testing without being given sufficient preparation time to assume this task, especially in terms of personnel and facilities.
If the agency does the acceptance testing, "business as usual" will be the predominate theme and the next step is to determine what quality characteristics to measure. If the agency does not do the acceptance testing, then it must decide who will perform that function prior to determining what quality characteristics to measure. Many agencies are requiring the contractor or a third party to do the acceptance testing.
Using PLS path modeling in new technology research: updated guidelines
As mentioned, this has often come about, at least partially, because of agency staff reductions. What has often evolved is that the contractor is required to perform both QC and acceptance testing. This is one reason that these two functions can become intermingled if care is not taken to assure their separation. If both functions are assigned to the contractor, it is imperative that the difference between the two functions and the purpose for each is thoroughly explained to both contractor and agency personnel.
Additionally, if the contractor is assigned the acceptance function, the contractor's acceptance tests must be verified by the agency. Statistically sound verification procedures must be developed that require a separate verification program. There are several forms of verification procedures and some forms are more efficient than others. To avoid conflicts, it is in the best interest of both parties to make the verification process as effective and efficient as practicable.
If the contractor or a third party acting on behalf of the contractor, such as a consultant, is required to do the acceptance testing, the agency must have a verification procedure to confirm or refute the acceptance test results. Quality control sampling and testing results may be used as part of the acceptance decision provided that:. The essence of this requirement is a valid and reasonable way to protect taxpayer's interests. That is, that QC and acceptance are separate functions and should not be commingled.
The reasons for this are presented and discussed in detail in chapters 3 and 4 of this manual.
Statistical hypothesis testing
In this manual, contractor tests that are used in the acceptance decision are referred to as acceptance, rather than QC, tests. QC tests are those used by the contractor for the purpose of process control. While it is true that contractors will definitely relate to their processes the results of the acceptance tests, truly beneficial QC tests are those for which results can be obtained during the process so that adjustments can be made to help ensure that the subsequent acceptance tests will meet the requirements of the acceptance plan.
The definitions used here are intended to assure that QC sampling and testing is a separate function from acceptance sampling and testing. However, the need for verification procedures is the same for both sets of definitions.
All books of the series Statistics for Industry and Technology
FHWA 23 CFR B 6 uses the term "verification sampling and testing" and defines it as "Sampling and testing performed to validate the quality of the product. The need for the use of independent samples as opposed to split samples has been questioned by some agencies. To understand the difference in the information provided by the two sampling procedures, i. Variability can come from many different sources. Statisticians sometimes refer to these variabilities as "errors"-sampling error, testing error, etc. These terms mean sampling variability and testing variability, not mistakes.
The sources of variability are combined by the use of the basic measure of variability, called the variance, denoted as s 2. The sources of variability are combined by adding the variances not the standard deviations, denoted as s. The sources of variability are important when deciding whether to use independent or split samples. The decision depends upon what the agency wants to verify. Independent samples, i. Split samples contain only testing method variability. These variability components are illustrated in figures 8 and 9.
Goodness-of-Fit Tests and Model Validity - Google книги
There has been a considerable amount of confusion between the uses of independent versus split sampling procedures. In an attempt to reduce this confusion, in this manual, the term test method verification refers to the case where split samples are used, while the term process verification refers to the case where independent samples are used. The statistical implications of these terms extend further than mere definitions.
If independent samples are used to verify that two sets of data are statistically similar, then the agency could consider combining the two data sets to make the acceptance decision. Variability issues must be considered when making a decision whether or not to combine the two sets of data. The fact that the data are not shown to be different does not necessarily mean that they are the same. It simply means that they could not be proven to be different given the sample sizes that were involved.
Therefore, it is possible that combining the two sets of data could lead to increased variability. On the other hand, the increased number of values in the combined data set might offset a possible increase in variability. In general, it is probably best to use the agency's verification tests simply for verification, and to use only the contractor's acceptance tests if they compare with the agency's tests.
However, if split samples are used to verify two sets of data, these data should not be combined to make the acceptance decision, even when they were determined to be statistically similar. This is because the two split-sample test results represent essentially the same material, and therefore there is little to no additional information provided by using both results. In fact, using both split-sample test results simply represents a double counting of this particular sample location.
There are no universally accepted verification sampling frequencies. However, like any statistical procedure, the ability of the comparison procedure to identify differences between two sets of results depends on several factors. One of these is the number of tests that are being compared-the greater the number of tests, the greater the ability of the procedure to identify statistically valid differences. A minimum agency rate of 10 percent of the testing rate of the contractor or third party has been used as a rule of thumb.
In practice, the verification testing frequency is usually an economic, rather than statistical, decision. The statistics of the issue will generally call for as many, or more, tests as the agency has the resources to perform. A detailed discussion of the effects of verification testing frequency is presented in the technical report for this project. Hypothesis Testing and Levels of Significance. Before discussing the various procedures that can be used for test method verification or process verification, two concepts must be understood: hypothesis testing and level of significance.
When it is necessary to test whether or not it is reasonable to accept an assumption about a set of data, statistical tests, called hypothesis tests, are conducted. Strictly speaking, a statistical test neither proves nor disproves a hypothesis. What it does is prescribe a formal manner in which evidence is to be examined to make a decision regarding whether or not the hypothesis is correct. To perform a hypothesis test, it is first necessary to define an assumed set of conditions known as the null hypothesis, H o.
Subscribe to our newsletter
Additionally, an alternative hypothesis, H a , is, as the name implies, an alternative set of conditions that will be assumed to exist if the null hypothesis is rejected. The statistical procedure consists of assuming that the null hypothesis is true, and then examining the data to see if there is sufficient evidence that it should be rejected. H o cannot actually be proved, only disproved. If the null hypothesis cannot be disproved or, to be statistically correct, rejected it should be stated that we "fail to reject," rather than "prove" or "accept," the hypothesis.
In practice, some people use "accept" rather than "fail to reject," although this is not exactly statistically correct. Consider concrete compressive test cylinders as an example. The null hypothesis might be that the average strength of a concrete bridge deck is 35, kilopascals kPa , while the alternative hypothesis might be that the average strength is less than 35, kPa.
If three tests are performed-and the test results are 30,, 31,, and 31, kPa-this would seem to be ample evidence in this simple example that the average strength is not 35, kPa, so the null hypothesis would be rejected. The alternative hypothesis, that the average strength is less than 35, kPa, would therefore be assumed true. Hypothesis tests are conducted at a selected level of significance, a, wherea is the probability of incorrectly rejecting the H o when it is actually true.
The value ofa is typically selected as 0. Test Method Verification Procedures. The two procedures used most often for test method verification are the D2S limits and the paired t- test. D2S Limits. This is the simplest procedure that can be used for verification, although it is the least powerful.
- Goodness-of-Fit Tests and Model Validity!
- Statistics for Industry and Technology: Goodness-of-Fit Tests and Model Validity (2002, Hardcover).
- Rethinking Security Governance: The Problem of Unintended Consequences (Contemporary Security Studies);
- Winning Poker Tournaments One Hand at a Time Volume II.
- Fungal Viruses: XIIth International Congress of Microbiology, Mycology Section, Munich, 3–8 September, 1978!
- Treatment of Contaminated Soil: Fundamentals, Analysis, Applications.
Because the procedure uses only two test results it cannot detect real differences unless the results are far apart. The D2S limit indicates the maximum acceptable difference between two results obtained on test portions of the same material and thus, applies to only split samples , and is provided for single and multilaboratory situations. It represents the difference between two individual test results that has approximately a 5 percent chance of being exceeded if the tests are actually from the same population.
When this procedure is used for test method verification, a sample is split into two portions and the contractor tests one split-sample portion while the agency tests the other split-sample portion. The difference between the contractor and agency test results is then compared to the D2S limits.
If the test difference is less than the D2S limit, the two tests are considered verified. If the test difference exceeds the D2S limit, then the contractor's test result is not verified, and the source of the difference is investigated. Suppose that an agency wished to use the D2S limits for test method verification of a contractor's asphalt content determination using the ignition method. So, for a split sample, if the difference between the contractor and agency results is 0.
If the difference is greater than 0. Paired t- test. For the case in which it is desirable to compare more than one pair of split-sample test results, the t -test for paired measurements can be used.