OMB M‑24‑10 Sets a New Accountability Regime for Law‑Enforcement Analytics
Applying inventories, impact assessments, TEV/M, bias metrics, and transparency to Palantir‑enabled targeting across investigative missions
The federal government has crossed a governance Rubicon: under the Office of Management and Budget’s M‑24‑10 memorandum, AI oversight moves from guidance to binding requirements. That shift lands squarely on investigative analytics used by law‑enforcement agencies, where configurable tools like link analysis, entity resolution, and prioritization now face a formal regime of inventories, impact assessments, testing, and transparency. The stakes are high: these analytics synthesize sensitive datasets to generate leads, shape investigative hypotheses, and guide resource allocation across missions ranging from transnational crime to child exploitation.
This piece maps how OMB M‑24‑10, grounded in the trajectory set by Executive Order 14110, will reshape testing, fairness, and transparency for Palantir‑enabled analytics used in investigative functions. It shows where current governance already provides scaffolding—through DHS privacy artifacts, system provenance and auditing—and where new obligations will demand measurable performance evidence, bias assessment, and independent validation. Readers will learn how to categorize investigative functions under the AI definition, what to include in use‑case inventories and impact assessments, how to build TEV/M programs fit for targeting, and how to reconcile transparency and contestability with statutory constraints. A practical roadmap closes with near‑term steps agencies can implement to meet the moment.
The Regulatory Shift: From EO 14110 to OMB M‑24‑10—and Defining AI in Investigations
Executive Order 14110 directs agencies to ensure safe, secure, and trustworthy AI through concrete governance mechanisms: leadership roles, use‑case inventories, and risk‑tailored safeguards. OMB M‑24‑10 makes those mechanisms mandatory, requiring agencies to identify and assess AI uses—particularly those that can affect rights and safety—and to implement testing, evaluation, and monitoring (TEV/M) alongside transparency measures suitable to the risk profile.
Deciding what counts as AI: prioritization, link analysis, and entity resolution
The threshold question for investigative analytics is categorization. Palantir‑enabled environments used in federal investigations—such as consolidated search and analytical platforms built on Gotham—provide entity resolution, deconfliction across datasets, graph‑based link analysis, and geospatial analytics. They also support configurable dashboards and workflows that surface leads and prioritize attention using investigator‑defined criteria and business rules. Agencies must determine whether these configurations meet the policy definition of AI in practice. When such analytics materially influence investigative targeting or resource allocation, and carry plausible civil rights implications, the prudent approach is to treat them as AI subject to M‑24‑10’s requirements.
This categorization does not convert these systems into automated adjudicators. DHS artifacts and system designs emphasize human‑in‑the‑loop decision‑making; analysts and supervisors remain responsible for verifying evidentiary sufficiency before action. But “human in the loop” does not obviate governance: if analytic outputs shape who is visible to the system and which hypotheses rise to the top, M‑24‑10’s safety‑impacting AI framework is the appropriate lens.
Use‑case inventories and impact assessments tailored to investigative functions
Inventory entries should describe functions, not just systems. For Palantir‑enabled investigations, distinct entries are warranted for:
- Entity resolution and deconfliction across DHS and external datasets
- Graph/link analysis that surfaces associations among persons, entities, events, communications, and financial flows
- Geospatial mapping and proximity analytics
- Lead generation and prioritization workflows used to triage investigative focus
Each entry should capture data sources (including DHS systems, interagency feeds, and approved commercial/open‑source datasets), governance constraints (role‑based access, provenance tagging, audit logging), and risk characteristics (potential for disparate impact, feedback effects, and privacy sensitivities). Impact assessments must go further, describing foreseeable harms, mitigations, and oversight pathways, and clarifying the human decision points where verification and supervisory review occur.
Building TEV/M for Palantir‑Enabled Investigations
OMB M‑24‑10 requires testing, evaluation, and monitoring commensurate with risk. For investigative analytics, TEV/M must move beyond platform security and process checks to quantify how well lead‑generation and prioritization actually perform—and for whom.
Establishing precision/recall, robustness, and drift monitoring
Despite robust controls around provenance, auditing, and human review, publicly available quantitative evidence on investigative analytics’ accuracy remains scarce. Agencies should institute TEV/M programs that:
- Define operationally meaningful outcome labels for retrospective evaluation (for example, whether a prioritized lead was corroborated through independent evidence prior to operational action). Where specific metrics are unavailable publicly, agencies should report to oversight bodies and publish high‑level summaries that protect sensitive methods.
- Measure precision and recall for lead‑generation and prioritization configurations across representative investigative domains. Because the same platform can host multiple analytic workflows, evaluations should be use‑case specific.
- Test robustness to noisy, stale, or conflicting records that are inherent to multi‑source aggregation; document known failure modes tied to data quality and identity matching.
- Monitor distribution shift and drift in data sources—particularly commercial datasets and open‑source feeds—flagging when coverage or quality changes could skew outputs.
A change‑management process should tie these metrics to deployment controls: significant configuration changes or dataset additions should trigger re‑verification and change‑log updates, with summaries available to oversight teams.
Bias and disparate‑impact metrics: sampling frames, protected proxies, and mitigation plans
Fairness risks accumulate where coverage is uneven (e.g., geospatial datasets like license plate reader records), where commercial data reflects structural biases, and where analytic features act as proxies for protected characteristics. Agencies should:
- Establish sampling frames that reflect the populations actually affected by investigative prioritization, not just active cases or historical hotspots.
- Identify and document potential proxies (nationality, language, address history, network structure) and test for outcome disparities across relevant groups. Where statutory constraints limit public disclosure, report detailed results to internal oversight (privacy offices, civil rights units, inspectors general) and release aggregated summaries.
- Pair disparity findings with documented mitigation plans: adjust feature weightings or thresholds; require human verification steps for high‑risk feature combinations; constrain use of particular datasets to narrowed, necessity‑justified purposes; and expand analyst training on bias risks.
Experience from other law‑enforcement risk tools shows how opaque logic and inadequate oversight can degrade fairness, even when tools are framed as neutral. Investigative analytics that influence who is scrutinized must meet a higher bar: disparities should be measured, explained, and mitigated, not presumed away by human review alone.
Feedback loop risks and longitudinal evaluation of resource allocation effects
When prioritization and allocation decisions are informed by past enforcement outcomes, analytics can reinforce historical patterns regardless of underlying risk. Agencies should treat this as a testable hypothesis. Longitudinal evaluations can compare trajectories in similarly situated regions or case categories to detect whether resource shifts driven by analytics amplify disparities in who becomes visible to investigators. Mitigations may include periodic re‑balancing, caps on certain query types, or targeted data‑quality investments in under‑represented areas. Oversight bodies should receive recurring analyses of feedback effects alongside standard performance dashboards.
Transparency, Documentation, and Contestability—Within Statutory Constraints
Law‑enforcement analytics operate within a privacy and disclosure framework that both enables oversight and limits public visibility. Navigating this tension is central to M‑24‑10’s transparency expectations.
Model documentation: public summaries versus sensitive internal detail
Today’s public transparency rests largely on privacy impact assessments and system of records notices, which describe purposes, data categories, sharing, risks, and mitigations. Internally, platforms provide lineage and audit capabilities that let analysts inspect sources, queries, and linkages behind an analytic output. What is missing publicly is system‑ and use‑case–specific documentation—akin to model cards—that articulates datasets used, known limitations, evaluation procedures, and change histories for targeting and prioritization configurations.
Agencies should produce two tiers of documentation:
- Sensitive internal documentation with detailed TEV/M findings, disparity metrics, and configuration specifics for use by privacy offices, civil rights units, inspectors general, and legal counsel.
- Public summaries that explain functions, known limitations, and governance mechanisms in plain language, omitting operationally sensitive thresholds or weights but conveying accountability posture and results at a high level.
Transparency and contestability under statutory constraints and exemptions
Contestability is complicated by law‑enforcement exemptions, classification, and sensitive source protections. Individuals can seek records through established processes, but access is often limited when disclosure would reveal techniques or impede investigations. In practice, challenges to analytic outputs surface in criminal or immigration proceedings, where discovery and protective orders govern method access. Agencies can still improve contestability by providing notice in proceedings when analytics materially inform actions and by facilitating controlled defense access to underlying records and methodologies under court supervision.
Independent validation: roles for privacy offices, civil rights units, inspectors general, and external reviewers
Multi‑layered oversight already exists: departmental privacy offices review PIAs and compliance; civil rights units assess rights impacts; inspectors general investigate misconduct and systemic failures; and Congress can demand information. Under M‑24‑10, these actors should receive formal validation dossiers for safety‑impacting AI uses, including investigative targeting. Periodic independent audits—conducted internally or by qualified external reviewers—should evaluate accuracy, robustness, disparate impact, and feedback effects, with findings shared with oversight bodies and summarized publicly.
Data, Privacy, and Procurement: Sensitive Datasets Under Evolving Legal Doctrine
Analytic ambition is bounded by the legal and policy envelope around sensitive data. That envelope has tightened—and shifted—in ways that directly affect investigative analytics.
Evolving doctrine on digital privacy and location‑linked data
Constitutional doctrine has underscored the heightened privacy interests in certain categories of digital information, notably sensitive location records. Agencies accessing geospatial datasets—including license plate reader records or telephony‑derived location information—must align their practices with evolving expectations for legal process, minimization, and necessity. Documentation should reflect the legal basis, the safeguards applied, and the contexts in which such data is used to inform analytics.
State data broker laws: procurement implications and compliance choreography
The commercial data ecosystem has become a major input to investigative analytics—public records, utilities data, telephony metadata, vehicle and LPR records, and social media–derived signals among them. Emerging state data broker laws are reshaping what is available, on what terms, and with what notice and consent expectations. Agencies should:
- Map each brokered dataset to a specific, necessity‑justified use, retention period, and legal basis in updated privacy artifacts.
- Vet vendors for provenance, compliance with applicable laws, and data‑quality practices; require provenance tagging on ingest.
- Ensure procurement transparency ties contract line items to corresponding governance and risk‑mitigation deliverables.
Security posture and public accountability for data handling
Platform‑level security authorizations and object‑level controls provide a strong technical baseline. Yet data handling failures elsewhere in the ecosystem demonstrate how sensitive environments remain vulnerable to inadvertent disclosure. Agencies should harden release controls, expand red‑team exercises on data handling workflows, and publish lessons learned through existing privacy reporting channels.
A Near‑Term Roadmap: Pilot, Phase, Standardize
Meeting M‑24‑10’s bar for investigative analytics is achievable with disciplined sequencing and clear ownership.
- Launch pilots: Select 2–3 high‑impact investigative use cases (e.g., lead prioritization in complex cases) for end‑to‑end TEV/M build‑out—defining outcome labels, measuring precision/recall, testing robustness, and monitoring drift. Capture disparity metrics and feedback‑loop analyses with mitigation plans.
- Build inventories and impact assessments by function: Document entity resolution, link analysis, geospatial analytics, and prioritization as discrete entries with data source mappings, constraints, and risk characteristics.
- Produce tiered documentation: Create internal validation dossiers and public summaries; maintain change logs and update cadence tied to configuration changes and new data sources.
- Institutionalize oversight hand‑offs: Establish recurring briefings and secure data rooms for privacy offices, civil rights units, and inspectors general; define triggers for external review.
- Align procurement with accountability: Bake data‑quality audits, provenance tagging, and public‑summary deliverables into vendor contracts; map each brokered dataset to necessity‑justified purposes and retention.
- Standardize across components: Share templates for inventories, impact assessments, TEV/M metrics, and disparity analyses to reduce duplication and accelerate compliance. 🚦
Conclusion
OMB M‑24‑10 sets a new baseline of accountability for law‑enforcement analytics: if investigative configurations influence who is seen, flagged, or prioritized, they belong in agency AI inventories and must be tested, monitored, and explained. Existing DHS governance—privacy impact assessments, system of records notices, role‑based access, provenance, and auditing—provides a solid starting point. But the missing pieces are measurable evidence of performance, disparity‑aware evaluation, and independent validation—paired with public‑facing summaries that communicate limits and guardrails without compromising operations.
Key takeaways:
- Treat targeting and prioritization analytics as AI when they materially influence investigative actions and carry rights risks.
- Build TEV/M that measures precision/recall, robustness, drift, and feedback effects at the use‑case level, not just the system level.
- Evaluate and mitigate disparate impacts with clear sampling frames, proxy detection, and documented changes to features and thresholds.
- Produce tiered documentation and strengthen contestability through notices in proceedings and controlled method access.
- Tie procurement and data broker use to necessity, provenance, and transparent governance deliverables.
Next steps for agencies: stand up pilot TEV/M programs for a few high‑impact use cases, publish function‑level inventory entries and impact assessments, deliver internal validation dossiers to oversight bodies, and release public summaries that set expectations and enable measured accountability. The result will be stronger analytics and stronger legitimacy—a dual imperative for investigative missions operating under growing public and policy scrutiny.