Security Content Automation Protocol Validation Program SCAPVP

To enable the goals set forth in OMB Memorandum M-08-22, it is necessary to have security configuration scanning tools that can use official SCAP content. In response, NIST established the SCAP validation program. Implemented through the NIST National Voluntary Laboratory Accreditation Program (NVLAP), independent laboratories can be accredited to perform the testing necessary to validate that security tools can accurately parse the SCAP content required for their specific functionality. Additional details on SCAP validation are available at Security Content Automation Protocol Validation Program. More information about USGCB may be found at https://usgcb.nist.gov

SCAP 1.2 product validations are awarded to products that meet all requirements defined in NIST IR 7511 revision that is active at the time when the product is validated. The current version of IR 7511 is Revision 4. Vendors may choose the SCAP capabilities and platforms supported. Consumers and acquisition professionals should select products based on the needs of their respective organization. The SCAP Validation Program encourages consumers to note the following differences between validated products:

• Operating systems supported
• SCAP capabilities such as Authenticated Configuration Scanner, CVE, or OCIL
• SCAP version of validation

NOTE: Consumers should coordinate with the vendor to ensure the validated product supports operating systems relevant to their environment prior to purchase.

The SCAP validated products that are listed on the SCAP Validated Products page at Validated Products and Modules are those that have met all requirements defined in NIST IR 7511. Consumers of these products may download SCAP expressed checklists from https://checklists.nist.gov, import the SCAP expressed checklist into the SCAP validated product, and scan computer systems assessing the configuration compliance with the checklist. SCAP expressed checklists are also referred to as SCAP Content and are listed as Tier III and Tier IV on the https://checklists.nist.gov site. SCAP enabled products should be capable of importing and processing SCAP content available in the National Checklist Program Repository at https://checklists.nist.gov as well as any well-formed SCAP content from other sources. Consumers should refer to NIST IR 7511 for a complete list of test requirements that must be passed in order for a product to be awarded an SCAP validation and listed on the SCAP Validated Products web page. NIST validates products that meet the requirements defined in NIST IR 7511. NIST does not provide guarantee concerning product performance.

An SCAP module is a software component that may be embedded into another product or application.

No, the products that embed an SCAP validated module have not undergone testing by an accredited laboratory and are not validated by NIST. Only the products listed in SCAP 1.2 Validated Products and Modules section on the NIST website were validated. Products that embed an SCAP Validated Module may use the "SCAP 1.2 Inside" phrase and logo only after permission to use them has been granted by NIST.

ONLY the products or modules listed on Validated Products and Modules are validated. Purchasers of SCAP validated products should make sure that the vendor does not claim validation based on expired versions of the program and should periodically visit the above link to view the current list of valid SCAP 1.2 products. Although some vendors may claim support for individual SCAP component specifications (i.e. OVAL, XCCDF, CVE, etc.) it does not mean that their products are SCAP validated. Only products that have been rigorously tested by an independent NVLAP laboratory and validated by NIST are SCAP Validated products.

If the SCAP module is a component of a larger product or application, one should contact the product or application vendor in order to determine what products use an embedded validated SCAP module. There may be a larger number of security products available which use a validated SCAP module, than the number of modules which are found in SCAP 1.2 Validated Products and Modules section listed on the NIST website. In addition, it is possible that other vendors, who are not found in this list, might incorporate a validated SCAP module from this list into their products.

When selecting a module from a vendor, verify that the application or product that is being offered is either a validated SCAP module itself or product uses an embedded validated SCAP module. Ask the vendor to supply a signed letter stating their product or module is a validated module or incorporates a validated module, the module provides the following SCAP Capabilities (ACS, CVE, and/or OCIL), and reference the modules validation certificate number. The certificate number will provide reference to the above SCAP Validation Program lists of validated modules. Each entry will state what version/part number/release is validated, and the supported platforms the module has been validated. The information on the SCAP Validation Program validation entry can be checked against the information provided by the vendor and verified that they agree. If they do not agree, the vendor is not offering a validated solution.

The phrase "SCAP 1.2 Validated" is a certified mark of NIST and is intended for use in association with SCAP products/modules validated by the National Institute of Standards and Technology (NIST) as complying with Security Content Automation Protocol (SCAP) Version 1.2 Requirements for Products and Modules. Vendors of validated SCAP products and modules may use the phrase and logo provided if they sign the "SCAP 1.2 Logo Use Agreement".

The phrase "SCAP 1.2 Inside" is a certified mark of NIST and is intended for use in association with products that have not been awarded a validation by NIST, but incorporates an SCAP validated module. The phrase "SCAP 1.2 Inside" can be used only if the vendors sign the "SCAP 1.2 Logo Use Agreement".

The SCAP 1.2 Validation Program validates products against NIST SP800-126 rev.2 and its component specifications. The SCAP 1.2 Validation Program supersedes the SCAP 1.0 Validation Program. The SCAP capabilities offered in the SCAP 1.2 program are authenticated configuration scanner (ACS) with optional CVE and OCIL validation. Vendors may choose one or both of the CVE and OCIL validation options in conjunction with the ACS capability. The optional validations may not be awarded without ACS. Products with an SCAP 1.2 validation are intended to be backward compatible in that they should correctly process well formatted SCAP 1.0, SCAP 1.1, and SCAP 1.2 data streams. Refer to NISTIR 7511 Rev. 4 for the SCAP 1.2 Derived Test Requirements.

NISTIR 7511 Revision 4 adds validation for SCAP-enabled software components, or modules. The update adds a labeling program that uses the following phases and logos: "SCAP 1.2 Validated" and "SCAP 1.2 Inside". The "SCAP 1.2 Inside" is used for non-validated products that incorporate an SCAP validated module. The new "SCAP 1.2 Inside" label and associated logo may be used by products that incorporate SCAP-validated modules after permission to use the logo has been granted by NIST. These marks do not imply product endorsement by NIST or the U.S. federal government, but they do help consumers to recognize non-validated products that incorporate an SCAP-validated module. "SCAP 1.2 Inside" products have not gone through the validation program testing process and will not be listed on the SCAP Validated Products web page. Only products that have successfully been awarded validation in accordance with the SCAP 1.2 specification and the NISTIR 7511 Revision 4 derived test requirements are listed on the SCAP Validated Products website. NISTIR 7511 Revision 4 also updated some test requirements and added several new ones, to further improve the overall reliability and consistency of SCAP results by validated products, and to assure that validated products comply with the SCAP specification. A Summary of Changes table in NISTIR 7511 Revision 4 provides a complete list of changes from Revision 3. Please refer to the Security Content Automation Protocol Validation Program for the latest documents and updates. The SCAP validation record includes information about the product, SCAP capabilities, and platforms tested. The vendor is responsible for choosing the SCAP capabilities and platforms required by their target market. Consumers are responsible for referencing the SCAP 1.2 validation record that details the level of SCAP testing performed on a product.

Yes, the SCAP 1.2 validated products are backward compatible with earlier versions of SCAP and related components and according to NIST Interagency Report 7511 Revision 4, the NVLAP accredited labs are required to use the SCAP 1.0 and 1.1 compatible content for validating SCAP 1.2 products.

GSA created a SmartBuy program to streamline the purchase of SCAP validated products in support of the OMB FDCC mandate. The GSA SmartBuy program is independent of NIST but relies on NIST, as an independent reference, to ensure vendors' products meet or exceed the NIST SCAP Validation Program requirements and can be validated by independent accredited NVLAP laboratories. As of 12/31/13, only SCAP 1.2 validated products meet or exceed the NIST SCAP Validation Program requirements. NIST suggests contacting GSA and/or OMB regarding contract requirements and compliance respectively.

Consumers of SCAP validated products should contact the product vendor with questions about performance and operational issues such as installation, configuration, erroneous results, or reporting. The vendor may be identified on the SCAP Validated Products list at Validated Products and Modules. The SCAP Validation Program team can answer questions about the information and resources available at Security Content Automation Protocol Validation Program.

All SCAP 1.0 product validations expired December 31st, 2013. Only SCAP 1.2 validated product will be considered valid SCAP products beginning January 1st, 2014.

The SCAP 1.2 Validation Program uses two broad categories of SCAP content for testing products. The broad categories of content include:

• Validation Test Content
  • OVAL Test Data (SCAP 1.2)
  • Content for specific DTR requirements (SCAP 1.2, SCAP 1.1, and SCAP 1.0)
• USGCB Content (SCAP 1.2, SCAP 1.1, and SCAP 1.0)

The Validation Test Content contains the OVAL Test Data that exercises the OVAL constructs used in Tier III and Tier IV content, and the data streams needed for testing specific DTR requirements. The content is organized as following:

• validationTestSuites - contains the discrete data streams organized by OVAL tests
• combinedDataStreams - the combined data streams are comprised of the discrete data streams organized by platforms
• requirementsTest - contains the data streams for testing specific DTR requirements such as SCAP.R.500, SCAP.R.600, SCAP.R.700, SCAP.R.800, SCAP.R.1100, SCAP.R.1200, SCAP.R.1800 (OCIL), SCAP.R.1900, SCAP.R.2100, SCAP.R.2200, SCAP.R.2910, SCAP.R.2920, SCAP.R.2930, SCAP.R.2940, SCAP.R.3005, SCAP.R.3010, and SCAP.R.3300.

NIST is responsible for providing validation test criteria and materials to NVLAP accredited labs so each independent lab can perform product validation testing.

NIST is making the SCAP 1.2 validation test content public so vendors may better prepare products for validation, and so end users can perform their own conformance testing when selecting products. The content is available for download from the Security Content Automation Protocol Validation Program webpage.

The test suite is a collection of SCAP 1.2 data streams, many based on OVAL test types. Each directory in the test suite contains several items:

1. SCAP 1.2 data streams
2. validation Content spreadsheet - describes the expected results and the required configurations to fully exercise the data stream.
3. Configuration files - scripts that aid in the configuration of test systems.
4. Catalog file - xml file with expected results. These files are used by the tester to automate scan results comparison using the compare.py script provided in the tools folder.
5. Readme file - provides information about the test.
6. Validation Files directory - output from SCAPVal tool indicating if the source content is valid according to SCAP specifications.

No. the validation test suite is not an SCAP security checklist. The validation test suite is similar to unit testing. The goal is exercising all possible operators of selected OVAL test types.

The users should follow the "Quick guide for using the consolidated data streams" included in the "combinedDataStreams" folder of the content bundle (see the SCAP Validation Test Content table available on Security Content Automation Protocol Validation Program). This content should be run on a non-production system. While attempts are made to clean up the changes made as part of testing, there is no guarantee that the system will be in a secure or usable state afterward. After running this content the system should be wiped and the operating system reinstalled and configured appropriately before the system is used.

The SCAP 1.2 Validation Program currently includes the Red Hat family of platforms and the Windows family of platforms.

Please provide comments to the National Institute of Standards and Technology (NIST) at scap@nist.gov.

The validation test suite uses a version scheme comprised of major version, minor version, patch release, and out of cycle release. For example, ValidationTestSuite-w-w.x.y.z where

• w-w = major release - A major release would be an update of the validation program for a new version of SCAP. Because the version method is being introduced with the SCAP 1.2 Validation Program, the initial release of the test suite in December 2012 is 1-2.0.0.0
• x = minor release - A minor release covers the expansion of the validation test suite such as the addition of new OVAL test types, expansion to the existing OVAL check types, or the addition of an SCAP capability or platform to the existing Validation Program.
• y = patch release - A patch release includes only corrections in the data streams. No new functionality or expansion is included in patch releases.
• z = out of cycle release - An out of cycle release includes only corrections in the data streams for errors that are deemed particularly serious or errors that could result in validated products that return erroneous output.

The development and maintenance cycle for the validation test suite is six months. The first four months are designated for planning, development, and quality assurance testing. These activities result in a beta release that is published for community feedback. After the feedback is reviewed and addressed, the final release is published to Security Content Automation Protocol Validation Program. Patch updates may be published every six months as needed. Minor releases may be published once a year as needed.

The validation test suite version that will be accepted for SCAP 1.2 validation consideration is determined by the date when NIST receives the test report from the accredited laboratory. In order for the product to be considered for an SCAP 1.2 validation, the validation test suite version used for testing should be either the currently posted version or the prior version released. Note that, test reports reflecting the use of validation test suite versions older than 12 months will not be accepted unless it is the last version released. The validation test suite version that was used for testing will be part of the product validation record and will be available on the SCAP 1.2 Validated Products web page.