Using eMASS and OSCAL Artifacts to Operationalize Continuous Monitoring Evidence Collection

Continuous monitoring evidence collection is one of those requirements that looks straightforward on paper and becomes genuinely difficult once a system is in production. The Authorization to Operate is granted, the security plan is approved, and then the real work begins: generating, capturing, organizing, and retaining evidence that proves controls are operating as documented, month after month, across a system boundary that rarely stays static.

Most federal security teams understand the obligation. NIST SP 800-137 defines the monitoring strategy. eMASS holds the system record. But the gap between those two realities is where programs struggle. Evidence arrives from scanners, configuration tools, ticketing systems, and manual attestations. It lands in shared drives, email threads, and spreadsheets. When an assessor or auditor asks for a control’s current implementation status, the answer often requires pulling artifacts from four different places and hoping the timestamps align.

OSCAL changes the structural foundation of that problem. The Open Security Controls Assessment Language provides machine-readable models for system security plans, component definitions, assessment plans, assessment results, and plan of action and milestones data. When those models are populated correctly and synchronized with eMASS records, the result is a traceable, repeatable evidence chain that does not depend on institutional memory or manual assembly before every assessment.

This article focuses on the operational mechanics of making that work. Not the theory of continuous monitoring, and not a general overview of OSCAL models, but the specific decisions security teams need to make about which artifacts to generate, how to version and retain them, how they map to eMASS fields, where the mapping breaks down, and how to build a recurring evidence pipeline that holds up under assessor scrutiny.

OSCAL Models That Matter for Continuous Monitoring

OSCAL defines several models, and not all of them are equally relevant to ongoing evidence collection. The System Security Plan model captures implementation intent. It describes how controls are satisfied, which components implement which requirements, and what the system boundary includes. That information is foundational, but it does not constitute monitoring evidence. It is the baseline against which evidence is measured.

The models that carry the operational weight of continuous monitoring are the Assessment Results model and the component definition model, with the Plan of Action and Milestones model handling the remediation side of the loop.

Assessment Results and Observations

The OSCAL Assessment Results model contains observations, findings, and risks. Observations are the atomic unit of evidence. Each observation records what was examined, the method used, the subject of the examination, the result, and the timestamp. For continuous monitoring purposes, observations generated by automated tools, such as vulnerability scanners, configuration compliance engines, or log aggregators, should be structured as machine-generated observations with explicit references to the tool, version, scan configuration, and asset scope.

Findings aggregate observations into a determination about a specific control or control objective. A finding of “satisfied” or “not satisfied” against a control requires at least one supporting observation. When assessors review monitoring evidence, they are typically reviewing findings and then drilling into observations to verify the supporting data. If observations are missing, incomplete, or lack asset-level specificity, the finding cannot be validated independently.

Risks in the Assessment Results model link to findings that represent deficiencies. Those risks then connect to the POA&M model, which tracks remediation actions, milestones, and closure status. This linkage is what makes OSCAL useful for demonstrating not just that a deficiency exists, but that it is being managed within an accepted risk posture.

Component Definitions and Asset Scope

The Component Definition model describes the technology components within the system boundary and their control implementations. For continuous monitoring, component definitions serve as the reference for asset scope. When a scanner generates a finding, the affected asset must be traceable to a component in the system boundary. If the asset is not in the component definition, the finding either represents an undocumented asset, a scope boundary issue, or a false positive. None of those outcomes should be silently discarded.

Keeping component definitions current is a sustainment problem that most programs underestimate. Assets are added, decommissioned, reconfigured, and migrated. Each change affects which controls apply, which evidence is required, and which inherited controls remain valid. A stale component definition produces evidence gaps that are difficult to explain during an assessment.

How OSCAL Artifacts Map to eMASS Records

eMASS is the system of record for RMF documentation in most DoD environments and many civilian agency programs. It stores the security plan, control implementation statements, test results, POA&M entries, and authorization documentation. OSCAL artifacts are not natively ingested by eMASS in a fully automated way across all implementations, which means teams must define explicit synchronization rules rather than assuming the two systems will stay aligned on their own.

The mapping between OSCAL and eMASS is conceptually straightforward but operationally messy. The table below outlines the primary correspondences and where alignment breaks down.

OSCAL Artifact or Field	eMASS Equivalent	Alignment Notes
SSP control implementation statements	eMASS control implementation fields	Generally mappable; formatting differences require normalization
Assessment Results observations	Test results and artifact attachments	eMASS does not natively store structured observation data; artifacts are typically uploaded as files
Assessment Results findings	Control test result status (Compliant, Non-Compliant, Not Applicable)	Status values align conceptually; OSCAL findings carry richer supporting data than eMASS status fields
POA&M entries with risks and milestones	eMASS POA&M module	Field-level mapping exists but milestone granularity and risk scoring differ between models
Component definitions with inherited controls	eMASS inheritance relationships and overlay packages	Inheritance in eMASS is managed through package relationships; OSCAL component definitions require manual cross-referencing
Assessment plan subject scope	System boundary and asset inventory	eMASS hardware and software inventory does not directly import OSCAL component definitions

The practical implication of this mapping is that OSCAL artifacts should be treated as the authoritative source for structured evidence, while eMASS serves as the authoritative source for authorization status, control determinations, and POA&M tracking. When the two diverge, the eMASS record governs for RMF purposes, but the OSCAL artifact provides the supporting detail that eMASS fields cannot capture at the same fidelity.

Teams that treat eMASS as the only record and upload evidence as unstructured file attachments lose the traceability that OSCAL provides. Assessors receive a PDF scan report attached to a control record with no machine-readable linkage between the finding, the asset, the control objective, and the remediation status. That approach works until the volume of evidence grows beyond what a human reviewer can manually correlate.

Building a Recurring Evidence Collection Pipeline

A recurring evidence pipeline is not a single tool or a single workflow. It is a set of defined processes, scheduled tasks, and handoff points that together produce a consistent, timestamped, traceable record of control status across the monitoring cycle. The monitoring strategy defines the frequency. The pipeline executes against that frequency and deposits evidence into a location where it can be retrieved, reviewed, and linked to eMASS records.

The pipeline has four functional stages: generation, validation, versioning, and retention. Each stage has distinct ownership and failure modes.

Generation covers the automated and manual processes that produce raw evidence. Automated sources include vulnerability scanners, configuration compliance tools, endpoint detection platforms, log management systems, and identity governance tools. Each source should produce output in a format that can be normalized into OSCAL observations or that can be referenced by an OSCAL observation with a pointer to the raw artifact. Manual sources include human attestations for controls that cannot be assessed programmatically, such as physical access reviews, personnel security verifications, and configuration change approvals. These require a different handling path because they carry human accountability that machine-generated artifacts do not.

Validation is the step most pipelines skip. Before evidence enters the authoritative record, it should be checked for completeness, asset coverage, timestamp integrity, and schema conformance if OSCAL is the target format. A scan that ran against 60 percent of the asset inventory because a network segment was unreachable is not valid evidence for the full system boundary. A configuration check that ran with an outdated baseline does not demonstrate compliance with the current control requirement. Validation catches these issues before they become audit findings.

Versioning ensures that evidence can be compared across monitoring cycles. Each OSCAL Assessment Results document should carry a version identifier, a creation timestamp, and a reference to the assessment plan or monitoring strategy that governed the collection. When a control status changes from one cycle to the next, the version history shows when the change occurred, what evidence supported the previous determination, and what evidence supports the current one. Without versioning, the evidence record is a snapshot with no context.

Retention is a policy question with operational consequences. Evidence must be retained long enough to support continuous monitoring reviews, annual assessments, and any audit or investigation that references the monitoring period. Retention schedules should account for the system’s authorization boundary, the data classification of the evidence itself, and any agency-specific records management requirements. Evidence stored only in a tool’s internal database without export and archival is at risk of loss during tool migrations, license changes, or contract transitions.

For programs that have adopted or are moving toward Continuous Monitoring (CONMON) as a structured practice, the pipeline described above is the operational layer that makes the monitoring strategy executable rather than aspirational.

The following sequence outlines how these stages connect in practice.

Control Inheritance, Provenance, and Exception Handling

Control inheritance is where continuous monitoring evidence collection most commonly breaks down in multi-tier environments. When a system inherits controls from a cloud service provider, a shared services platform, or a common control provider, the inheriting system’s ISSO often treats those controls as covered by the provider’s ATO package and stops there. That is not sufficient for continuous monitoring purposes.

Inherited controls still require evidence that the inheritance relationship is current and that the provider’s implementation remains valid for the inheriting system’s use case. If the provider changes their implementation, reduces their scope, or receives a finding against an inherited control, the inheriting system’s monitoring record must reflect that change. Without a defined process for tracking provider-side changes, the inheriting system’s evidence record becomes stale without anyone noticing.

In OSCAL, inherited controls are documented in the SSP through leveraged authorization references and component definitions that point to the provider’s package. For monitoring purposes, the Assessment Results document for the inheriting system should include observations that reference the provider’s current assessment results, with explicit timestamps showing when the provider’s status was last verified. This is not the same as copying the provider’s evidence into the inheriting system’s record. It is a traceable reference that can be validated independently.

Provenance metadata is the mechanism that makes evidence traceable. Every observation in an OSCAL Assessment Results document should carry enough metadata to answer four questions: who or what generated this evidence, when was it generated, what was the scope of the assessment, and what method was used. For machine-generated artifacts, provenance includes the tool name, version, configuration profile, and the identity of the system account or service that executed the scan. For human attestations, provenance includes the name and role of the attesting individual, the date of attestation, and a reference to the procedure or checklist used.

Exception handling requires explicit documentation rather than silent omission. When a control cannot be assessed during a monitoring cycle because an asset is offline, a tool is unavailable, or a process is suspended, that exception must appear in the evidence record with an explanation and a compensating measure or risk acceptance. An evidence record that simply lacks an observation for a required control is indistinguishable from a record where the collection was never attempted. Assessors will treat both the same way: as a gap.

Remediation linkage closes the loop between findings and POA&M entries. In OSCAL, risks identified in the Assessment Results model link to POA&M entries that track remediation milestones. When a finding is closed, the POA&M entry should reference the observation that confirmed closure, including the timestamp and the method used to verify remediation. This linkage is what allows an authorizing official to see not just that a deficiency was identified, but that it was resolved and the resolution was verified.

Common Mistakes

The most persistent mistake is treating the initial ATO package as the evidence baseline for continuous monitoring. The SSP and assessment results produced for the initial authorization reflect the system at a specific point in time. Once the system is in production, configurations change, assets are added, software is updated, and the threat environment shifts. Using the initial assessment artifacts as evidence for ongoing monitoring produces a record that describes a system that no longer exists.

A related mistake is generating OSCAL artifacts once and not updating them. Some programs produce an OSCAL SSP for the initial authorization because the process requires it, then continue managing the system through eMASS manual updates and file attachments. The OSCAL artifact becomes a historical document rather than a living record. When an assessor asks for the current system state in machine-readable form, the team has nothing to provide.

Conflating compliance status with operational effectiveness is another common failure. A control marked as compliant in eMASS based on a quarterly scan result may have been non-compliant for eleven weeks of that quarter. If the monitoring frequency requires monthly evidence and the team is only collecting quarterly, the eMASS record overstates the control’s operational status. The monitoring strategy defines the required frequency, and evidence collection must match it.

Ownership gaps between the ISSO, the system administrator, and the tool operator create evidence collection failures that no one notices until an assessment. The ISSO assumes the administrator is running scans. The administrator assumes the tool operator is exporting results. The tool operator assumes someone else is uploading them to eMASS. Defining explicit ownership for each step in the evidence pipeline, and verifying that ownership through periodic checks, prevents these gaps from accumulating into a monitoring failure.

Finally, many programs do not distinguish between machine-generated artifacts and human attestations in their evidence records. Mixing the two without clear provenance metadata makes it impossible for an assessor to determine which evidence was independently verified and which was self-reported. Machine-generated artifacts carry implicit credibility from the tool and configuration used. Human attestations carry credibility from the qualifications and accountability of the attesting individual. Both are valid, but they are not interchangeable, and the evidence record should reflect the difference.

Recommended Federal Approach

Start with the monitoring strategy, not the tooling. Before configuring any collection pipeline, confirm that the approved monitoring strategy specifies control frequencies, defines the scope of automated versus manual assessment, and identifies the authoritative sources for each control family. If the monitoring strategy is outdated or does not exist at the required level of detail, update it before building the pipeline around assumptions that may not match the authorizing official’s expectations.

Conduct a current-state inventory of evidence sources. For each control in the system’s baseline, identify what evidence currently exists, where it is stored, who owns the collection process, and whether the evidence frequency matches the monitoring strategy. This inventory will surface gaps, orphaned artifacts, and controls that have never had a defined collection process. It will also identify controls where automated collection is feasible but has not been implemented, which represents the highest-value automation opportunity.

Establish OSCAL as the target format for structured evidence, but phase the adoption. Not every control will have tooling that produces OSCAL-native output. Start with the controls where automated collection already exists and where the tool can export structured data. Build the normalization layer to convert that output to OSCAL observations. Expand to additional controls as tooling and capacity allow. Do not attempt a full OSCAL implementation in a single effort; the operational disruption will exceed the benefit.

Define the eMASS synchronization rules explicitly. Decide which fields in eMASS are updated automatically from the pipeline, which require human review before update, and which are managed manually. Document these rules in the system’s operating procedures. Assign ownership for each synchronization point. Review the rules whenever the system boundary changes or a new tool is added to the collection pipeline.

Implement a recurring review cycle that is separate from the formal assessment cycle. The formal assessment happens annually or at defined intervals. The monitoring review should happen at the frequency defined in the monitoring strategy, which for many controls is monthly or more frequent. The recurring review should include a check of evidence completeness, a review of any new findings, an update to POA&M entries, and a confirmation that the component definition still reflects the current system boundary.

Build the workforce capacity to sustain the pipeline. Automated collection reduces manual effort but does not eliminate it. Someone must validate coverage, review exceptions, manage tool configurations, update component definitions, and conduct human attestations for controls that cannot be automated. If the ISSO is the only person with the knowledge and access to perform these tasks, the program has a single point of failure. Cross-training and documented procedures are not optional for a sustainable monitoring program.

Validate the evidence pipeline before the next formal assessment. Run a tabletop exercise or a structured review where the team attempts to answer an assessor’s likely questions using only the evidence in the current record. If the team cannot produce a complete, traceable, timestamped response for each control without resorting to manual reconstruction, the pipeline has gaps that need to be addressed before the assessment, not during it.