Day-60

The elapsed time from the first item affected by a drift to the supplier telling the partner about it in writing. Two clocks are reported, because they answer different questions.

Lead time = the notice timestamp minus the onset timestamp, reported in hours and in business hours, with the detection clock reported beside it. Numerator and denominator: a single lead time is an elapsed time and has neither. Where a report gives a mean over several drift observations, the numerator is the sum of the lead times and the denominator is the count of drift observations in the window, and both are printed, together with the maximum.

The timestamp of the first input item belonging to the shifted population, established from the record after the fact. In a Day-60 exercise the injection timestamp is known in advance and held by the person running the exercise, not by the team being measured.

Drift the partner announced in advance, such as a scheduled format change, which is a planned change and not a detection. Injections the supplier was told about. Signals raised and withdrawn as false, which are counted separately as a false-alert count, because a monitoring system that alerts constantly also has a lead time near zero and is worth nothing.

If the exercise window closes with no notice, the figure is written "not detected within the window", with the window length stated. It is never extrapolated to a longer window.

"Drift detection lead time means the elapsed time from the first Item belonging to a shifted input population to the Provider's written notice to the Partner naming the affected class and the supporting evidence. The Provider shall give that notice within placeholder business hours of onset, and shall report both the detection and the notice timestamps for every drift observation in the Monthly Report. A drift not notified within placeholder business hours of onset is a Severity 2 Incident under the Retainer Schedule clause 4."

The mean elapsed time to restore service, computed separately for each severity in the Retainer Schedule clause 4. Reported with the count, the median and the maximum, because a mean over three incidents conceals the worst of them.

Mean time to restore for a severity = the sum of restoration durations for incidents of that severity closed in the window ÷ the count of those incidents. The numerator is that sum of durations, measured as 3.19.3 and 3.19.4 define. The denominator is the count of incidents of that severity closed in the window, and it is printed with the figure, because a mean over two incidents is not a rate.

The earlier of the supplier's own detection timestamp and the partner's notice timestamp. Not the time the ticket was opened, and not the time an engineer picked it up.

The time the covered system returned to the state where work submitted at intake reaches an output or the review queue within the service level, confirmed by a monitoring check. Not the time the ticket was closed, and not the time the root cause was found.

Incidents excluded under the Retainer Schedule clause 10. Incidents raised and withdrawn as not reproducible, counted separately. Time inside an agreed change window. Time waiting on a partner dependency, which stops the clock and is reported as a separate "waiting on partner" total, so that neither party can hide inside the other's delay.

"Mean time to restore means the sum of restoration durations for Incidents of a given Severity closed in the measurement window, divided by the number of such Incidents, where the clock starts at the earlier of the Provider's detection and the Partner's notice and stops when a monitoring check confirms that work submitted at intake reaches an output or the human review queue within the Service Level. The Provider shall meet the resolution targets in the Retainer Schedule clause 3 and shall report, for each Severity, the count, the mean, the median and the maximum, together with any time excluded as waiting on a Partner dependency."

The elapsed time from the decision to roll back to the previous release serving production traffic and passing the smoke set with its model pin, prompt version and threshold set restored. Two further times are reported beside it: decision time, and re-queue completion.

Rollback time = the smoke-set pass timestamp minus the decision timestamp, in minutes. Numerator and denominator: a single rollback is an elapsed time and has neither. Where a report covers more than one rollback, the numerator is the sum of the rollback times and the denominator is the count of rollbacks in the period, reported separately for rehearsals and for rollbacks performed on production, with the maximum of each.

A rollback that restores the service quickly but leaves the outputs of a bad release sitting in the partner's downstream systems is not finished. A rollback that takes four minutes after a two-hour argument is not fast. Publishing one number and not the other three is the usual way this measure is made to look good.

A rollback that also requires a data restore reports the restore time separately under 02-security/security-policy-set/07-backup-and-recovery.md. Rehearsals on a non-production copy are labelled as rehearsals and are never mixed in a table with rollbacks performed on production.

"Rollback time means the elapsed time from the recording of a rollback decision to the previous signed release serving production traffic and passing the smoke set with its model pin, prompt version and threshold set restored. The Provider shall complete a rollback within placeholder minutes, shall rehearse a rollback at least once per quarter and record the rehearsal result in the Monthly Report, and shall report decision time and re-queue completion time alongside rollback time for every rollback performed."

<!-- Benchmark charter, part 4. Indexed in ../methodology.md. Sections 3.21 to 7. -->

The share of required elements of the monthly report that are present, cover the whole period, and carry the basis or the sample size where the element is a figure.

Report completeness = elements present and passing ÷ elements required.

Elements that are present, cover the whole reporting period, and are supported. An element that is present but carries a figure without its sample size or its basis fails, because an unsupported figure in a partner report is the problem this whole charter exists to address.

The ten elements listed in 01-legal/ops-retainer-schedule.md clause 5.2, plus any element the statement of work adds. The denominator is printed with the figure, since it differs per statement of work.

Elements that do not apply in the period, such as incidents in a month with none. These pass if the report states "none" explicitly, and fail if they are simply absent. The count of not-applicable elements is reported.

Timeliness, being delivery by the fifth business day of the following month. A complete report delivered late and an incomplete report delivered on time are different failures and are not averaged together.

"Report completeness means the number of required Monthly Report elements that are present, cover the whole reporting period and carry the sample size or the basis for each figure, divided by the number of elements required by the Retainer Schedule clause 5.2 and this SOW. The Provider shall achieve report completeness of 100 percent, an element that does not apply in the period being satisfied by an explicit statement to that effect, and shall deliver the Monthly Report by the fifth Business Day of the following month. A figure reported without its sample size or its basis does not satisfy the element it belongs to."