AI Safety Institute Germany

The two ministries name as the central aim (in German) to assess the opportunities and risks of advanced AI models for Germany and to strengthen resilience in artificial intelligence over the long term. In the current first phase, the institute mainly analyzes leading AI models and their capabilities and risks for cybersecurity. After recent incidents, the ministries also see increasingly concrete risks from a loss of control over very capable models.

The federal government lists further tasks: the institute is to develop solutions for the safe use of AI systems (in German), exchange information with comparable institutes abroad and work toward uniform international standards. In its entry with the OECD, Germany names as core tasks the technical evaluation of advanced models for cyber capabilities, autonomy, resistance to manipulation and misuse potential, the analysis of their security effects on government, business and society, and the monitoring of international technology trends.

The BMDS under Federal Digital Minister Karsten Wildberger and the BMI under Federal Interior Minister Alexander Dobrindt build and lead the institute together. For the first phase, both ministries assigned the tasks to the BSI and the Federal Network Agency (Bundesnetzagentur): the BSI covers (cyber) security, the Federal Network Agency covers safety, and further actors from the security field are closely involved. Security means protecting AI systems and their data against attacks and misuse, while safety means unintended errors, such as an autonomous vehicle that detects an obstacle too late, as the Börsen-Zeitung (in German) explains.

Karsten Wildberger said that Germany needs a capable AI safety institute that keeps pace with the enormous speed of technological development. Alexander Dobrindt said that artificial intelligence offers great opportunities for citizens, business and the state, but that AI must remain controllable and Germany must take part in its rapid development.

The institute is based in Berlin, which according to Forschung & Lehre (in German) prevailed over bids from Munich and Saarbrücken. In a second phase, the institute grows in scope and staff and is then to ensure a structured transfer of security findings on AI to the federal government, public administration, business and civil society. The federal government also plans to use it for advice on technical evaluation, development scenarios and strategic needs for action.

The ministries published no figures on staff or budget when they founded the institute. The digital industry association Bitkom had called for adequate funding after the decision of the security council, and Susanne Dehmel of the Bitkom executive board asked for (in German) funding at least at the level of the British AISI.

Since 2019, the BSI has worked in its Centre of Excellence for Artificial Intelligence on test criteria, methods and tools for secure AI systems, and this center is the central AI contact point of the federal administration. An early result is the AIC4 criteria catalog for AI cloud services, an extension of BSI C5, whose criteria cover security, data quality, explainability and bias across the full lifecycle of a service.

For banks and insurers, the BSI published a test criteria catalog for AI systems in finance from its AICRIV project, with about 100 practical test criteria. The catalog covers IT security, data quality, model resilience, governance, human oversight and performance, addresses developers, providers, operators and auditors of AI systems in the financial industry and supports the implementation of the EU AI Act.

The institute already exchanges expertise with partner institutions in France and the United Kingdom and with the EU AI Office, and the two ministries (in German) give cooperation with European and transatlantic partners particular weight. The institute is also to take an active part in the international network of AI safety institutes, which serves knowledge exchange and common positions among the participating states, and to work with its partners toward uniform standards for the safe use of the most advanced AI.

The AISI Deutschland is not built as a regulator, and the federal government (in German) describes it as an analytical and operational competence center. Supervision under the EU AI Act lies with other bodies: under the German implementing act, BaFin supervises AI systems that banks and insurers use directly in regulated financial business. Other AI in the same firms, for example in HR, falls to the Federal Network Agency, which is therefore both a market surveillance authority under the AI Act and the safety pillar of the institute.

At EU level, the European AI Office enforces the rules for general-purpose AI models and may evaluate those models. The German institute exchanges expertise with this office, and banks meet these rules through the models they buy from large providers and bring into their AI governance.

The BMDS and BMI give the institute no mandate of its own toward banks or insurers, but the financial industry is part of the business sector that is to receive security findings from the institute in the second phase. Several risks the institute assesses in leading models already show up in finance as fraud: ENISA reports that by early 2025, AI-supported phishing campaigns reportedly made up more than 80 percent of observed social engineering activity worldwide.

The BSI warns that an attacker could call a person with the deepfaked voice of their manager to trigger a money transfer, the so-called CEO fraud. Credentials stolen through phishing lead to account takeover, which banks also counter with AI fraud detection. For risk managers, the evaluation of the cyber capabilities of leading models can show which attacks an available model makes easier, and the BSI test catalog for AI in finance gives them criteria for their own systems.

According to the BSI, deepfakes are a high threat to biometric systems because some methods already run in real time, and remote identification such as video identification is especially exposed. The checking party only receives the output signal and has no control over the camera or over changes to the recorded material. This is where injection attacks come in, feeding a prepared video stream past the camera directly into the app of a bank.

Since 17 January 2025, the Digital Operational Resilience Act governs ICT security in finance: Article 6 requires a documented ICT risk management framework, and Article 5 makes the management body responsible for it. Banks in Germany report major ICT incidents to BaFin, and NIS2 gives way to DORA for financial entities, because section 28(6) of the German BSI Act exempts them from its risk management, reporting and management duties.

Regulation (EU) 2024/1183 requires in Article 5a that each member state provide at least one EUDI Wallet within 24 months of the entry into force of the implementing acts. Under Article 5f(2), private relying parties that must use strong user authentication by law or contract have to accept the wallet no later than 36 months from that date, on the voluntary request of the user, and banking and financial services are named in the list of sectors. Micro and small enterprises are exempt, and because the first implementing regulations entered into force on 24 December 2024, the deadlines fall at the end of 2026 for the wallets and the end of 2027 for acceptance.

On the technical side, the EU Architecture and Reference Framework sets how data moves between wallet and bank: identity data and attestations are issued over OpenID4VCI and presented remotely over OpenID4VP, each in the HAIP interoperability profile. The German state wallet is developed by the Federal Agency for Breakthrough Innovations SPRIND for the BMDS, with partners including the BSI, Bundesdruckerei and Fraunhofer AISEC. A wallet presentation carries the signature of its issuer, which the bank verifies, while the other KYC and anti-money laundering duties remain.