Data PrivacyUse Of AI & Data Privacy Implications in Healthcare Sector

INTRODUCTION

The premier authority for biomedical research in India, namely, the Indian Council of Medical Research (hereinafter referred to as “ICMR”), released a comprehensive set of guidelines titled “Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare” (hereinafter referred to as the “Guidelines”) that provide detailed procedures for the use of Artificial Intelligence (hereinafter referred to as “AI”) for research and healthcare purposes. The guidelines are specifically designed to govern the use of AI and related technologies in connection with data related to health and ensure that their deployment in biomedical research, as well as healthcare delivery, adheres to high ethical standards.

The guidelines released by the ICMR apply broadly to all AI tools developed for  biomedical and health research involving human participants or their biological data. These guidelines not only emphasize  need to conduct a risk-based assessment in order to identify potential harms but also to create and establish a transparent framework with the objective of communicating AI processes and limitations, and obtaining explicit and informed consent from individuals regarding the use of algorithms.

Moreover, continuous and ongoing impact assessments are mandated in order to address unanticipated consequences of the use of AI. This comprehensive approach by the ICMR aims to safeguard patient rights, promote accountability and data privacy, and foster trust in AI-driven healthcare innovations.

AI IN HEALTHCARE SECTOR

Rapid growth and advancements are taking place in both AI as well as the healthcare sector. Recently, in the year 2023, the ICMR acknowledged this continuously evolving landscape and, therefore, released a set of Guidelines for the use of technologies and AI in research as well as healthcare services. These Guidelines are intended as a living document subject to periodic refinement to keep pace with technological progress and emerging ethical challenges. Moreover, the Guidelines issued by the ICMR are relevant to various stakeholders like healthcare professionals, technology developers, researchers, research institutions and other persons who use healthcare data for research and delivery purposes.

The integration of AI in healthcare is rapidly transforming the way we approach the field of healthcare. It is already making a momentous and substantial impact in various areas such as diagnostics and screening, where it helps to reduce the burden on clinicians by improving accuracy and minimizing human error.

Furthermore, the promise of precision medicine is also coming to life, as AI analyzes vast and enormous databases in a proper and specific manner to tailor treatments to each individual’s unique profile. Beyond this, AI is revolutionizing the operational side of the healthcare system as well. AI can also ensure the accuracy of medical data, assist in finding the right specialists, and verify insurance claims, thus making the entire healthcare experience more efficient and patient-centred.

ETHICAL PRINCIPLES AND RESPONSIBLE AI IN HEALTHCARE RESEARCH

In the present scenario, every biomedical research project, whether it uses traditional or advanced digital AI methods, should be built on and follow fundamental principles ethical values. The core ethical principles include respecting individual autonomy, acting for the betterment of everyone and ensuring fair as well as equal treatment for all individuals. In the year 2017, ICMR released the National Ethical Guidelines, which outlined principles to protect the dignity as well as the rights of the person.

But today, as AI becomes more common in health research, new ethical questions arise. AI systems often depend on large amounts of human data, which brings up concerns about privacy, bias, data interpretation, and whether people truly understand and provide consent to how their information is used. While responsible AI frameworks outline values like fairness, transparency, security, and inclusiveness, putting these into practice can look very different depending on the field.

The Guidelines provided by the ICMR in the year 2023 provide for ten (10) ethical principles for AI technology in healthcare. The ten different ethical principles are as follows:

Autonomy, Collaboration, Trustworthiness (Data Security), Risk Minimization and Safety, Accountability and Liability, Data Privacy, Validity, Non-Discrimination and Fairness, Optimization and Data Quality, Accessibility and Equity.

These ethical principles contribute to the effective collaboration and functioning of AI in healthcare research and other biomedical research activities.

Moving forward, ideologies like inclusiveness, fairness, security, and transparency are central to most responsible AI frameworks. But, along with this, it is also vital note that the application of  these values can vary across practices.

Today, there is an ongoing debate and discussion about whether the present frameworks truly address both overt and implicit biases in AI systems. Forming a secure and ethical AI requires a collaborative and informed approach among different disciplines. However, even a year after the release of the Guidelines, there is a lack of consensus on ways to employ AI in medical research, and this has left many medical professionals still struggling to find clear guidance on how to use AI responsibly.

DATA PRIVACY IMPLICATIONS IN THE GUIDELINES BY THE ICMR

Today, in the technologically progressive era, AI in healthcare must prioritize privacy as well as the protection of personal data at every stage of development and deployment. Earning and maintaining the trust of all stakeholders, especially patients, is necessary for the successful adoption of AI solutions in the healthcare industry. The medical data and records are of a sensitive nature. Therefore, the misuse of such records can lead to discrimination or harm in various manners.

Data privacy safeguards should involve mechanisms to prevent unauthorized access, data tampering, and breaches. AI systems should not infringe on individual liberty and must apply the principle of data minimization as well as anonymization unless identifiable data is absolutely required for research or any other purposes.

The Salient points on data privacy in the Guidelines are:

• Individuals must have the right to access, modify, or delete their personal data. Therefore, it ensures individuals have control over their data.
• There must be transparency and informed consent. It requires individuals to be informed about how the data is being collected, where it will be used and what measures are taken for protecting the data.
• Predictive algorithms could produce unintended or harmful results. Explicit, informed consent must be obtained before deploying such algorithms on participants or patients.
• When the AI systems use biometric data of any participants or patients, then advanced security measures must be incorporated. Apart from this, prior approval must be obtained from Ethics Committees (hereinafter referred to as the “EC”) and regulatory bodies.
• The developers as well as manufacturers of AI must ensure that the personal data of all the stakeholders are de-identified swiftly, and that personal data cannot be re-identified or leaked.
• Additional informed consent be obtained before sharing data beyond the original use for which it is recorded.

In addition to these, it is also vital to mention that currently, in India, the Digital Personal Data Protection Act of 2023 (hereinafter referred to as “DPDPA”), as well as the Draft Digital Personal Data Protection Rules of 2025 (hereinafter referred to as the “Rules”), provide for the protection of digital personal data of patients. The legislation provides for various new standards for healthcare organizations to manage, store, as well as transfer personal data, emphasizing explicit consent, data localization, and strict penalties for non-compliance.

Now, in order to meet these standards, healthcare providers are required to appoint Data Protection Officers (hereinafter referred to as the “DPOs”), conduct impact assessments for high-risk activities, along with maintaining transparent data practices. All these steps must be taken to safeguard the trust of the patients and avoid significant financial and reputational risks.

The effect of DPDPA and the Rules is felt among various stakeholders. The hospitals must protect the sensitive personal records of the patient and comply with retention and transfer rules. Additionally, diagnostic labs and other healthcare research centres also have the duty and responsibility to secure both test results as well as personal data from breaches and leaks. Last but not least, academic institutions in the healthcare industry are also under obligation to strictly follow consent and security guidelines when handling patient data.

AMLEGALS REMARKS
The release of Ethical Guidelines for the Application of Artificial Intelligence in Biomedical Research and Healthcare is of utmost importance in the contemporary world because it is developing and becoming technologically advanced with each passing day. ICMR released the Guidelines with the aim and objective of maintaining ethics while conducting research, handling data of participants or transferring data of patients under treatment. The Guidelines stressed data privacy implications and the importance of consent , as it is of a vital nature in the present era.

The introduction of DPDPA and the Rules strengthen  data protection measures and mandates on hospitals as well as other healthcare research centres to protect data and prevent it from breach and unwarranted use.

– Team AMLEGALS assisted by Mr. Aditya Raj Pandey (Intern)

For any further queries or feedback, feel free to reach out to rohit.lalwani@amlegals.com or mridusha.guha@amlegals.com