AI Chatbots in Customer Service: GDPR and AI Act Duties

A customer-service chatbot looks like a small feature: a bubble in the corner of the page that answers questions about delivery, opening hours or invoices. Legally it is two things at once. It is an AI system that interacts directly with people, which the EU AI Act regulates through transparency rules. And it is a way of processing personal data, because customers type names, order numbers, email addresses and sometimes far more, which the GDPR regulates whether or not an AI is involved.

The timing is current. The European Commission states that the AI Act's transparency rules in Article 50 apply from 2 August 2026, and its implementation timeline lists the same date for the start of enforcement of the rules that apply. A Digital Omnibus amendment has since changed some dates, but not the chatbot disclosure duty. This guide is therefore written for businesses that are deploying or already running a bot in Germany and want to check it against both regimes.

It is a practical guide, not legal advice. It draws on the Commission's own explanation of Article 50, the German data protection authorities' guidance on AI applications and the European Data Protection Board's opinion on AI models. Ask your data protection officer or a lawyer to confirm the final set-up, particularly if the bot handles health, financial or employment matters.

The AI Act separates roles, and Article 50 gives different duties to each. The Commission's FAQ explains that a provider is a person or body that develops an AI system, or has one developed, and places it on the market or puts it into service under its own name or trademark. A deployer is anyone using an AI system under their authority in a professional context.

A typical German shop that licenses a chatbot service from a vendor and embeds it on its site is a deployer. The vendor is the provider. But two situations move you towards the provider role: building your own bot on top of a general model and releasing it under your brand, and presenting a vendor's bot under your own name as if it were your product. The Commission's definition turns on putting the system into service under your own name or trademark, so a branded white-label bot deserves a careful reading of the contract. If in doubt, assume you must meet the provider's design duty and ask the vendor to confirm in writing how its product satisfies it.

The practical consequence is simple. Whoever the law labels, the customer should see a clear disclosure that they are talking to an AI, and your contract should say which party ensures that the product supports it.

According to the Commission, Article 50(1) requires providers of AI systems that interact directly with natural persons to design and develop them so that the people concerned are informed that they are interacting with an AI system, unless this is obvious. The Commission's guidelines set four cumulative criteria for the duty to apply: the system must qualify as an AI system; it must be designed for a genuine two-way exchange with people, not just collecting data or giving automated responses; the interaction must be direct, meaning the AI itself communicates and not through a human intermediary; and the interaction must be with natural persons, whether consumers, professionals or other users.

A customer-service chatbot normally ticks all four boxes. A form that merely returns fixed answers, or a background system that never talks to a person, may fall outside the scope, but a free-text assistant that responds in natural language does not.

When and how to inform: the Commission says people must be notified from the start of the first interaction in a clear and distinguishable manner and in accordance with accessibility requirements. The exception for cases where it is obvious that one is dealing with an AI must be interpreted restrictively, because it deprives people of transparency. The Commission describes an average, reasonably well-informed, circumspect and observant person as the yardstick. A bot called "Assistant" with a human photo and a human first name will not pass that test.

In practice, put the disclosure in the first message and keep it visible: "You are chatting with an AI assistant. A colleague can take over at any time." Do not hide it in the footer or the privacy notice. Check that screen readers read it, and translate it into every language the bot supports.

Article 50 contains four further elements, and customer-service teams often confuse them. Providers of generative AI systems must mark synthetic audio, image, video or text output in a machine-readable format so that it can be detected as AI-generated. That duty sits with the provider, such as the vendor, though a transition until December 2026 is available for systems placed on the market before 2 August 2026, as the Commission's timeline records.

Deployers have separate duties: to inform people exposed to emotion recognition or biometric categorisation systems, and to label deepfakes and AI-generated text published to inform the public on matters of public interest without human review or editorial control. A chatbot answering a delivery question does none of these, but a bot that analyses callers' emotions or that publishes AI-written public notices does.

The Commission's quick-facts page adds that the national market surveillance authorities enforce the transparency rules, with fines of up to EUR 15 million or up to 3 % of total worldwide annual turnover for companies, with proportionality for small and medium-sized enterprises. These figures describe the upper limit of the range, not a typical outcome. The page also notes that the transition for older systems applies only to marking, not to the chatbot disclosure.

The German data protection authorities (Datenschutzkonferenz, DSK) published guidance on AI and data protection in May 2024. Its first message is the oldest rule in data protection: for every processing step in which an AI application processes personal data, a legal basis is required. For a customer-service bot that is often the contract with the customer (answering an order query) or the company's legitimate interest in efficient service, but each purpose needs its own assessment, and consent is needed if you want to use conversations for something unrelated, such as training or marketing.

The EDPB's Opinion 28/2024 on AI models addresses legitimate interest directly. It describes a three-step test and gives the example of a conversational agent to assist users as a service that can rely on legitimate interest, but only if the processing is shown to be strictly necessary and the balancing of rights is respected. Document that assessment before launch.

The other points from the DSK guidance that matter for a bot are these:

• Processor contract: where the bot is a cloud service, the provider is usually a processor under Article 28 GDPR and you must conclude an agreement under Article 28(3). The processor must also help you meet data subject rights.
• Open or closed system: in an open system, inputs leave your protected area and may be used for other users' answers or for further training, and may reach third countries, so Chapter V on transfers applies. The DSK prefers technically closed systems from a data protection viewpoint.
• Training use: check whether inputs and outputs are used for training, whether people are told, and whether use can be switched off. If it cannot be excluded and personal data are affected, a legal basis for that purpose is required.
• Data protection by design: Article 25 requires technical and organisational measures. The guidance gives the example of accounts configured so that no inputs are used for training and no history is stored beyond the session.
• Impact assessment: where processing is likely to result in a high risk, a data protection impact assessment under Article 35 is required, which the DSK says will often be the case with AI applications.
• Special categories: health, religion and similar data are in principle prohibited under Article 9(1) unless an exception in Article 9(2) to (4) applies, which must be checked for inputs, processing and outputs.

The AI Act disclosure is separate from GDPR information duties, and you need both. The DSK points out that deployers who do not develop the system must make sure that the vendor provides enough documentation for them to meet Articles 12 and following, and that information about automated decision-making, including the logic involved and its significance and consequences, can be required. Treat the vendor's documentation as a deliverable in the contract.

Update your privacy notice with the bot as a new processing activity: purpose, legal basis, categories of data, recipients (the vendor and any sub-processors), transfers outside the EU and the retention period. Add a short notice in the chat window itself, linking to the full text, for example "Do not enter health or payment details. Messages are processed to answer your request." This is where the two regimes meet: the first message says it is an AI, and the link says what happens to the data.

Also give customers a way around the bot. A visible "talk to a person" option, and a real handover to a human colleague who can see the conversation if the customer agrees, makes the service more useful and reduces the risk of unfair automated outcomes. Our AI chatbot integration includes this handover design.

Chat logs are useful for quality control and dangerous as archives. Decide in advance what you store and for how long, and write it down. A workable pattern is to keep conversation text for a short, defined period for quality review and complaint handling, then delete or anonymise it; keep a minimal technical log (time, session ID, error codes) for longer for security. Separate the logs from the customer database unless a ticket needs the transcript.

Control what the bot can see. Do not connect it to the whole customer database; give it access to the specific order data needed for the question, after the customer has authenticated. A bot that can reveal an address to anyone who guesses an order number is a data breach waiting to happen.

Warn users not to enter sensitive data, and filter obvious patterns such as card numbers and national insurance-type identifiers before they reach the model. The DSK stresses that outputs can contain personal data even when inputs do not, so test the bot with prompts designed to extract other customers' data. Finally, include the bot in the incident response plan: who is called if it gives out wrong or personal information?

A support bot should not make decisions with legal effect on customers. The DSK notes that Article 22(1) GDPR generally allows decisions with legal effect only by humans, with limited exceptions such as consent, and that where an AI proposes a result, the human decision-maker must have genuine discretion and not simply follow the suggestion. Refund approvals, contract cancellations, account closures and credit decisions are examples where a bot should prepare the case and a person should decide.

Write boundaries into the bot's instructions and into your process: what it may answer, what it must escalate, and which topics (legal claims, complaints about discrimination, safety issues) go straight to a human. Check transcripts regularly and keep a record of the checks.

The bot also needs content governance. Give it an approved knowledge base, version it and date it, and make sure that prices, delivery times and terms are drawn from the same source as the website. Where the bot's wrong answer could be treated as a binding promise, have a lawyer review the terms of use and the disclaimer text.

Review the table whenever the vendor changes the model or the terms. For budgeting, our pricing page lists simple websites from EUR 300 excl. VAT; chatbot projects are scoped separately, so contact us with a short description of your use case and the data involved. You may also want to read about GDPR-compliant websites.

• Transparency obligations under Article 50 of the AI Act - European Commission
• Quick Facts: Transparency rules for AI systems - European Commission
• Timeline for the implementation of the EU AI Act - AI Act Service Desk
• Orientierungshilfe KI und Datenschutz, May 2024 - Datenschutzkonferenz
• EDPB opinion on AI models: GDPR principles support responsible AI - EDPB