CJI BR Gavai flags deepfake, AI misuse as SC hears plea for regulating GenAI in judiciary

The Supreme Court on Monday acknowledged the growing misuse of Artificial Intelligence (AI) and deepfake technology, with Chief Justice of India (CJI) BR Gavai remarking that even judges have become targets of AI-generated fake content, Bar and Bench reported.

Hearing a public interest litigation (PIL) seeking regulation of Generative AI (GenAI) in India’s judicial ecosystem, the CJI commented, “Yes, yes, we have seen our morphed pictures too,” underlining the increasing digital threats faced by the judiciary.

The bench, which was hearing the plea filed by advocate Kartikeya Rawal, adjourned the matter for two weeks after asking the petitioner whether they wished to withdraw or continue the case.

As per the report, the PIL urges the Union government to establish a policy or legal framework governing the use of GenAI in judicial and quasi-judicial institutions. It highlights the potential risks of unregulated AI systems, particularly those capable of generating fabricated data or “hallucinated” case laws that could distort legal proceedings.

Rawal’s petition draws a distinction between traditional AI and GenAI, noting that while the former operates on existing data, the latter has the ability to create new content, a trait that can introduce ambiguity and false precedents in judicial decisions.

The plea warns that GenAI’s opaque “black box” nature makes it vulnerable to producing misleading or entirely fake case laws, a phenomenon referred to as AI hallucinatio. Such inaccuracies, the petitioner argues, could undermine the rule of law and violate Article 14 of the Constitution, which guarantees equality before the law.

“The ability of GenAI to generate new data, uncover hidden patterns, and automate complex processes can result in hallucinations, bias, and even fabricated legal interpretations,” the plea states.

It further points out that GenAI systems trained on biased or discriminatory data could perpetuate prejudices against marginalised communities, aggravating ethical and legal challenges within the judicial system.

The plea highlights the rising integration of AI tools across sectors, including law, noting that while automation may improve efficiency, unregulated use in judicial decision-making could compromise fairness and transparency. It calls for ensuring data quality, accountability, and ownership clarity in all AI-driven judicial functions.

The Supreme Court will take up the matter again in two weeks, as India grapples with the global debate around regulating AI to prevent misuse while fostering innovation.