Title 7
SECTION 275.11
275.11 Sampling.
§ 275.11 Sampling.(a) Sampling plan. Each State agency shall develop a quality control sampling plan which demonstrates the integrity of its sampling procedures.
(1) Content. The sampling plan shall include a complete description of the frame, the method of sample selection, and methods for estimating characteristics of the population and their sampling errors. The description of the sample frames shall include: source, availability, accuracy, completeness, components, location, form, frequency of updates, deletion of cases not subject to review, and structure. The description of the methods of sample selection shall include procedures for: estimating caseload size, overpull, computation of sampling intervals and random starts (if any), stratification or clustering (if any), identifying sample cases, correcting over-or undersampling, and monitoring sample selection and assignment. A time schedule for each step in the sampling procedures shall be included.
(2) Criteria. Sampling plans proposing non-proportional or other alternative designs shall document compliance with the approval criteria in paragraph (b)(4) of this section. All sampling plans shall:
(i) Conform to principles of probability sampling;
(ii) Specify and explain the basis for the sample sizes chosen by the State agency;
(iii) If the State agency has chosen an active sample size as specified in paragraph (b)(1)(iii) of this section, include a statement that, whether or not the sample size is increased to reflect an increase in participation as discussed in paragraph (b)(3) of this section, the State agency will not use the size of the sample chosen as a basis for challenging the resulting error rates.
(iv) If the State agency has chosen a negative sample size as specified in paragraph (b)(2)(ii) of this section, include a statement that, whether or not the sample size is increased to reflect an increase in negative actions as discussed in paragraph (b)(3) of this section, the State agency will not use the size of the sample chosen as a basis for challenging the resulting error rates.
(3) Design. FNS generally recommends a systematic sample design for both active and negative samples because of its relative ease to administer, its validity, and because it yields a sample proportional to variations in the caseload over the course of the annual review period. (To obtain a systematic sample, a State agency would select every kth case after a random start between 1 and k. The value of k is dependent upon the estimated size of the universe and the sample size.) A State agency may, however, develop an alternative sampling design better suited for its particular situation. Whatever the design, it must conform to commonly acceptable statistical theory and application (see paragraph (b)(4) of this section).
(4) FNS review and approval. The State agency shall submit its sampling plan to FNS for approval as a part of its State Plan of Operation in accordance with § 272.2(e)(4). In addition, all sampling procedures used by the State agency, including frame composition, construction, and content shall be fully documented and available for review by FNS.
(b) Sample size. There are two samples for the SNAP quality control review process, an active case sample and a negative case sample. The size of both these samples is based on the State agency's average monthly caseload during the annual review period. Costs associated with a State agency's sample sizes are reimbursable as specified in § 277.4.
(1) Active cases. (i) All active cases shall be selected in accordance with standard procedures, and the review findings shall be included in the calculation of the State agency's payment error rate.
(ii) Unless a State agency chooses to select and review a number of active cases determined by the formulas provided in paragraph (b)(1)(iii) of this section and has included in its sampling plan the reliability certification required by paragraph (a)(2)(iii) of this section, the minimum number of active cases to be selected and reviewed by a State agency during each annual review period shall be determined as follows:
| Average monthly reviewable caseload (N) | Minimum annual sample size (n) |
|---|---|
| 60,000 and over | n = 2400 |
| 10,000 to 59,999 | n = 300 + [0.042(N−10,000)] |
| Under 10,000 | n = 300 |
(iii) A State agency which includes in its sampling plan the statement required by paragraph (a)(2)(iii) of this section may determine the minimum number of active cases to be selected and reviewed during each annual review period as follows:
| Average monthly reviewable caseload (N) | Minimum annual sample size (n) |
|---|---|
| 60,000 and over | n = 1020 |
| 12,942 to 59,999 | n = 300 + [0.0153(N−12,941)] |
| Under 12,942 | n = 300 |
(iv) In the formulas in paragraphs (b)(1)(ii) and (iii) of this section n is the required active case sample size. This is the minimum number of active cases subject to review which must be selected each review period. Also in the formulas, N is the average monthly participating caseload subject to quality control review (i.e., households which are included in the active universe defined in paragraph (e)(1) of this section) during the annual review period.
(2) Negative cases. (i) Unless a State agency chooses to select and review a number of negative cases determined by the formulas provided in paragraph (b)(2)(ii) of this section and has included in its sampling plan the reliability certification required by paragraph (a)(2)(iv) of this section, the minimum number of negative cases to be selected and reviewed by a State agency during each annual review period shall be determined as follows:
| Average monthly reviewable negative caseload (N) | Minimum annual sample size (n) |
|---|---|
| 5,000 and over | n = 800 |
| 500 to 4,999 | n = 150 + [0.144(N−500)] |
| Under 500 | n = 150 |
(ii) A State agency which includes in its sampling plan the statement required by paragraph (a)(2)(iv) of this section may determine the minimum number of negative cases to be selected and reviewed during each annual review period as follows:
| Average monthly reviewable negative caseload (N) | Minimum annual sample size (n) |
|---|---|
| 5,000 and over | n = 680 |
| 684 to 4,999 | n = 150 + [ 0.1224(N−683)] |
| Under 684 | n = 150 |
(iii) In the formulas in this paragraph (b)(2), n is the required negative sample size. This is the minimum number of negative cases subject to review which must be selected each review period.
(iv) In the formulas in this paragraph (b)(2), N is the average monthly number of negative cases which are subject to quality control review (i.e., households which are part of the negative universe defined in paragraph (e)(2) of this section) during the annual review period.
(3) Unanticipated changes. Since the average monthly caseloads (both active and negative) must be estimated at the beginning of each annual review period, unanticipated changes can result in the need for adjustments to the sample size. FNS shall not penalize a State agency that does not adjust its sample size if the actual caseload during a review period is less than 20 percent larger than the estimated caseload initially used to determine sample size. If the actual caseload is more than 20 percent larger than the estimated caseload, the larger sample size appropriate for the actual caseload will be used in computing the sample completion rate.
(4) Alternative designs. The active and negative sample size determinations assume that State agencies will use a systematic or simple random sample design. State agencies able to obtain results of equivalent reliability with smaller samples and appropriate design may use an alternative design with FNS approval. To receive FNS approval, proposals for any type of alternative design must:
(i) Demonstrate that the alternative design provides payment error rate estimates with equal-or-better predicted precision than would be obtained had the State agency reviewed simple random samples of the sizes specified in paragraphs (b)(1) and (b)(2) of this section.
(ii) Describe all weighting, and estimation procedures if the sample design is non-self-weighted, or uses a sampling technique other than systematic sampling.
(iii) Demonstrate that self-weighting is actually achieved in sample designs claimed to be self-weighting.
(c) Sample selection. The selection of cases for quality control review shall be made separately for active and negative cases each month during the annual review period. Each month each State agency shall select for review approximately one-twelfth of its required sample, unless FNS has approved other numbers of cases specified in the sampling plan.
(1) Substitutions. Once a household has been identified for inclusion in the sample by a predesigned sampling procedure, substitutions are not acceptable. An active case must be reviewed each time it is selected for the sample. If a household is selected more than once for the negative sample as the result of separate and distinct instances of denial, suspension or termination, it shall be reviewed each time.
(2) Corrections. Excessive undersampling must be corrected during the annual review period. Excessive oversampling may be corrected at the State agency's option. Cases which are dropped to compensate for oversampling shall be reported as not subject to review. Because corrections must not bias the sample results, cases which are dropped to compensate for oversampling must comprise a random subsample of all cases selected (including those completed, not completed, and not subject to review). Cases which are added to the sample to compensate for undersampling must be randomly selected from the entire frame in accordance with the procedures specified in paragraphs (b), (c)(1), and (e) of this section. All sample adjustments must be fully documented and available for review by FNS.
(d) Required sample size. A State agency's required sample size is the larger of either the number of cases selected which are subject to review or the number of cases chosen for selection and review according to paragraph (b) of this section.
(e) Sample frame. The State agency shall select cases for quality control review from a sample frame. The choice of a sampling frame shall depend upon the criteria of timeliness, completeness, accuracy, and administrative burden. Complete coverage of the sample universes, as defined in paragraph (f) of this section, must be assured so that every household subject to quality control review has an equal or known chance of being selected in the sample. Since the SNAP quality control review process requires an active and negative sample, two corresponding sample frames are also required.
(1) Active cases. The frame for active cases shall list all households which were: (i) Certified prior to, or during, the sample month; and (ii) issued benefits for the sample month, except for those households excluded from the universe in paragraph (f)(1) of this section. State agencies may elect to use either a list of certified eligible households or a list of households issued an allotment. If the State agency uses a list of certified eligible households, those households which are issued benefits for the sample month after the frame has been compiled shall be included in a supplemental list. If the State agency uses an issuance list, the State agency shall ensure that the list includes those households which do not actually receive an allotment because the entire amount is recovered for repayment of an overissuance in accordance with the allotment reduction procedures in § 273.18.
(2) Negative cases. The frame for negative cases shall list:
(i) All actions to deny an application in the sample month except those excluded from the universe in paragraph (f)(2) of this section. If a household is subject to more than one denial action in a single sample month, each action shall be listed separately in the sample frame; and
(ii) All actions to suspend or terminate a household in the sample month except those excluded from the universe in paragraph (f)(2) of this section. Each action to suspend or terminate a household in the sample month shall be listed separately in the sample frame.
(3) Unwanted cases. A frame may include cases for which information is not desired (e.g., households which have been certified but did not actually participate during the sample month). When such cases cannot be eliminated from the frame beforehand and are selected for the sample, they must be accounted for and reported as being not subject to review in accordance with the provisions in §§ 275.12(g) and 275.13(e).
(f) Sample universe. The State agency shall ensure that its active and negative case frames accurately reflect their sample universes. There are two sample universes for the SNAP quality control review process, an active case universe and a negative case universe. The exceptions noted below for both universes are households not usually amenable to quality control review.
(1) Active cases. The universe for active cases shall include all households certified prior to, or during, the sample month and receiving SNAP benefits for the sample month, except for the following:
(i) A household in which all the members had died or had moved out of the State before the review could be undertaken or completed;
(ii) A household receiving SNAP benefits under a disaster certification authorized by FNS;
(iii) A household which is under investigation for intentional Program violation, including a household with a pending administrative disqualification hearing;
(iv) A household appealing an adverse action when the review date falls within the time period covered by continued participation pending the hearing; or
(v) A household receiving restored benefits in accordance with § 273.17 but not participating based upon an approved application. Other households excluded from the active case universe during the review process are identified in § 275.12(g).
(2) Negative cases. The universe for negative cases shall include all actions taken to deny, suspend, or terminate a household in the sample month except the following:
(i) A household which had its case closed due to expiration of the certification period;
(ii) A household denied SNAP benefits under a disaster certification authorized by FNS;
(iii) A household which withdrew an application prior to the agency's determination;
(iv) A household which is under active investigation for Intentional Program Violation;
(v) A household which has been sent a notice of pending status but which was not actually denied participation;
(vi) A household which was terminated for failure to file a complete monthly report by the extended filing date, but reinstated when it subsequently filed the complete report before the end of the issuance month;
(vii) Other households excluded from the negative case universe during the review process as identified in § 275.13(e).
(g) Demonstration projects/SSA processing. Households correctly classified for participation under the rules of an FNS-authorized demonstration project which FNS determines to significantly modify the rules for determining households' eligibility or allotment level, and households participating based upon an application processed by Social Security Administration personnel shall be included in the selection and review process. They shall be included in the universe for calculating sample sizes and included in the sample frames for sample selection as specified in paragraphs (b) through (e) of this section. In addition, they shall be included in the quality control review reports as specified in § 275.21(e) and included in the calculation of a State agency's completion rate as specified in § 275.23(b)(2). The review of these cases shall be conducted in accordance with the provisions specified in §§ 275.12(h) and 275.13(f). FNS shall establish on an individual demonstration project basis whether the results of the reviews of active and negative demonstration project cases shall be included or excluded from the determination of State agencies' error rates as described in § 275.23(b). Cases processed by SSA in accordance with § 273.2(k) of this chapter, except demonstration project cases, shall be excluded from the determination of State agencies' error rates. FNS shall establish on an individual project basis whether demonstration project cases processed by SSA shall be included or excluded from the determination of State agencies' error rates.
[Amdt. 260, 49 FR 6304, Feb. 17, 1984; 49 FR 14495, Apr. 12, 1984, as amended by Amdt. 262, 49 FR 50598, Dec. 31, 1984; Amdt. 266, 52 FR 3409, Feb. 4, 1987; Amdt. 328, 56 FR 60051, Nov. 27, 1991; Amdt. 366, 62 FR 29658, June 2, 1997; Amdt. 373, 64 FR 38295, July 16, 1999; 68 FR 59523, Oct. 16, 2003; 75 FR 33436, June 11, 2010]