Abstract

We propose a federated framework for training diffusion models on decentralized and private datasets. The method learns a shared generative model alongside personalized client models, allowing clients to benefit from cross-client structure while ensuring that the shared model cannot reproduce any client’s data on its own. We provide formal differential privacy guarantees for each client and establish utility bounds for conditional generation under a Gaussian mixture model, showing that collaboration improves sample quality relative to private non-collaborative training. Experiments on CIFAR-10, Colorized MNIST, and CelebA support these results: the method generates high-fidelity samples, improves performance on minority and underrepresented classes, and maintains strong protection against membership inference, memorization, and reconstruction attacks.

Abstract

We study the problem of federated training of diffusion models (DMs) with privacy guarantees. For high-dimensional data, existing private federated DMs often exhibit a poor privacy-utility tradeoff, since the privacy noise introduced to protect client data becomes increasingly damaging as the dimension of the image representation grows. To address this challenge, we propose Personalized Federated Training of Latent Diffusion Models (PF-LDM), a framework that performs private federated diffusion training in latent space. The key idea is to identify latent representations whose distributional structure is more favorable for private learning. In particular, we show that latent spaces with more discriminative feature representations can better preserve utility under privacy constraints. PF-LDM further combines a shared server-side diffusion model with personalized client-side refinement models: the server captures cross-client generative structure from privatized latent data, while client-specific models refine samples to recover fine-grained local details. Experiments on the CelebAHQ dataset demonstrate that our method enables high-dimensional image generation, improves performance on underrepresented classes across clients, and maintains strong privacy protection.

Abstract

A task-oriented virtual assistant (VA) refers to an artificial intelligence-driven system that can assist users to perform daily activities. The utilisation of deep learning algorithms has enabled VAs to attain noteworthy advancements in high-resource languages, such as English. However, languages with limited resources, such as Bengali, have not encountered significant advancements in this domain. In this paper, we propose a multilingual task oriented voice-to-voice conversational agent, which is capable of proficiently managing diverse tasks such as weather forecast, date & time query, hospital and blood bank search etc in both Bengali and English language. Our developed system can understand voice command using Automatic Speech Recognition (ASR) and Natural Language Understanding Unit (NLU). Then the system generates an appropriate reply by gathering information from the internet via APIs, data retrieval techniques and by employing dialogue management. Finally, Natural Language Generator (NLG) and Text to Speech (TTS) techniques are used to construct and deliver proper response. We integrated all the units using RASA framework and python script. Our developed ASR system has an average word error rate of 13% and NLU system has an intent and entity extraction accuracy of 93% and 96.2% respectively. The overall action prediction accuracy of our developed system is 99.4%