Dutch government inspectorates are responsible for safeguarding public interests, from food safety and fair working conditions to educational quality and environmental protection. To ensure fairness in the execution of regulations and policies, it is important that inspectors are aligned on the interpretation. In practice, this can be challenging due to the ambiguity in those regulations and policies. Our research sits within the AI4Oversight lab, where we work hand in hand with multiple inspectorates to co-develop AI tools that can help with tackling this ambiguity in practice.

Key challenges in this domain

Regulations and policies can often be interpreted differently by different experts. Unfortunately, most conventional methods assume a singular ground truth, meaning that novel methods and datasets may be required to effectively capture regulatory ambiguity and detect gray areas. It is also important that users can judge output quality to accurately detect poor assessments, which creates a need for interpretable models and explainable outputs. Additionally, inspectorates regularly work with sensitive documents, meaning that using externally hosted LLMs such as ChatGPT may not be feasible, which creates a requirement for models that can be run locally.