Gen AI and its Contributions
In July 2026 EJIL published its ecosystem-wide generative artificial intelligence (Gen AI) policy. The policy is carefully worded and wisely narrow. While it also addresses a series of concerns which would merit discussion individually, such as those concerning plagiarism and accuracy, in this contribution I will focus on what the policy identifies as “the key question” in a passage that is worth quoting in full (with emphasis added).
The key question for deciding whether Gen AI use is permissible or not is whether a submission retains a significant intellectual contribution from the named author(s), with the author’s contribution clearly outweighing that of Gen AI. The policy prohibits the use of Gen AI that would render the author’s contribution negligible or that would, if made by a human, require co-authorship. Conversely, in case the contribution, if made by a human, would require acknowledgement, the use of Gen AI is permitted – provided the author properly discloses and explains such use.
The policy, in short, accepts a “Gen AI research assistant”, but not one that would require naming a Gen AI model as a coauthor – something that some Gen AI models insist on and which is not a trivial conceptual problem. This is a good rule in the abstract – by way of illustrative contrast, Bloomsbury is seemingly more concerned with intellectual property implications, will not accept AI-created content even if it undergoes substantial editing and prohibits submitting work to publicly accessible systems – a training-data problem I have addressed elsewhere.
This is not to say that there are no difficulties. The first is conceptual: as a general matter, it cannot be easy to police a disruptive technology through one of the most unstable cultural concepts in human history. Thus, Foucault reminds us that the author is a function that discourse acquires under particular institutional conditions; it is undisputed that the modern notion of authorship has been construed on the basis of some foundational myths – with significant consequences for the legal regulation of the phenomenon; and of course the boundary between a contribution that merits a co-authorship credit as opposed to a simple acknowledgment is sometimes difficult to identify, as one may gather from this illuminating Google Scholar query.
The second, more significant problem is evidentiary: EJIL’s policy says that “[i]f the team discovers evidence of non-permitted Gen AI use, or of use that was improperly disclosed, it may exercise its discretion not to send a manuscript to peer review”. But how can the share of intellectual contribution be assessed on the basis of the available evidence? This is a difficult and problematic exercise which can result in sanctioning the wrong thing.
As Krzysztof Pelc has recently observed, writing is incapable of revealing its provenance, or at least of revealing it conclusively. Worse, as he puts it, once the suspicion is introduced that a text may be LLM-generated, it is difficult for a human writer to displace it – more so than, for example, for a musician, as this recent story concerning a string manufacturer demonstrates. As a result, signs of LLM may cast doubt on the credibility of the authorial, intellectual contribution. With the improved accuracy of AI detection tools, acceptable and disclosed uses of LLMs are also likely to be penalised this way. It does not appear, therefore, that the incentives for an AI disclosure (He and Bu found a rate of only about one out of one thousand among 5.2 million papers published since 2023) will improve. Compliance may be rewarded by the editor, less so by the readers: Schilke and Reimann’s experiments found that readers trusted writers who disclosed AI use far less than writers who did not.
This is a problem of credibility that goes beyond the question of accuracy – still a serious one in law, and one for which I refer the reader to Damien Charlotin’s excellent database of legal “hallucinations”. It is in many cases related to stylistic features capable of undermining, by engendering the suspicions discussed above as to the provenance of the text concerned. Of course, in our field of international law, questions have arisen about the dynamics and implication of practices relating to “manufacture” and drafting of judgments, panel reports, and even arbitral awards, sometimes with momentous consequences. And, in our field, we might have a better normative claim than most as to why this matters, since scholarly writing may qualify under Article 38(1)(d) of the ICJ Statute as the “teachings of the most highly qualified publicists”, notwithstanding the difficulty of identifying that group, as discussed by the International Law Commission’s 2026 report at p. 173 and in this EJIL article.
Is There Any AI Writing in the PIL Blogosphere?
That EJIL had to come up with a policy might be a clue, but is there any evidence of AI writing in PIL scholarship?
To answer this question, I examined two widely read international law blogs, EJIL:Talk! and Opinio Juris. I first used stylometric techniques adopted since the birth of humanities computing, ranging from Father Roberto Busa’s Index Thomisticus, to Mosteller and Wallace’s work on the disputed Federalist Papers. I applied them to the full corpora of both blogs, totalling – after removing announcements and miscellanea – 2,705 and 3,249 substantive posts respectively. Rather than hunting for any single article showing symptoms of AI use, I initially sought to establish whether after ChatGPT became available in 2022, words that had been elsewhere found to be associated with LLM-assisted writing had become more common than we should have otherwise expected.
To do so, I used a list of words from a study of Kobak and colleagues, who studied 15.1 million biomedical abstracts hunting for evidence of Gen AI use. Their main list contains 291 rare words that became unexpectedly (and otherwise unexplainably) common in 2024. I then counted the words in 250-word chunks of texts so as to control for post length. The results were striking: in 2025, an average 250-word block of EJIL:Talk! text contained about 0.41 more words in Kobak’s list than during the 2015 to 2022. For Opinio Juris, the value was 0.74. These are not small numbers – they are, after all, rare words – and the consistency of the movement is striking.
The finding was also confirmed by searching, for any comparably large jumps before the advent of ChatGPT, and by removing extremely prolific authors (such as Marko Milanovic). This changed nothing. Finally, I looked at whether I could reproduce the movement by using LLM models to rewrite posts of either blog, which succeeded.
Figure 1. Left: Kobak’s published 291-word list, shown as the number of additional listed words in each 250-word block compared with 2015 to 2022. The dotted line marks ChatGPT’s public release. Right: an illustration that converts the 2025 legal-word shift into complete model-draft equivalents. It is not a count of AI-assisted posts.
To give a sense of the scale of the phenomenon, I also compared the 2025 shift with the largest word shift produced when a model drafted or substantially rewrote a test post. The movement is quantifiable as the equivalent of 16.7 complete drafts per hundred EJIL:Talk! posts and 13.1 per hundred Opinio Juris posts, or a far greater number of lightly-LLM-edited drafts.
We Have AI Detectors at Home
Having proved a shift in the aggregate dimension, is it possible to pinpoint (with an adequate level of confidence) that an individual document resembles machine-generated text? This is important, because claims made through detectors tend to be reputationally more consequential. I therefore decided to do some testing on my own to see how well they performed in the specific domain of international law writing.
First, I chose 120 (60 per blog) articles written before the release of ChatGPT and I had various language models either rewrite them entirely to generate a complete draft or edit them heavily or lightly. For a complete draft, the model was given the title, tags, and first 650 words of the blog post and instructed to rewrite those and generate the continuation. For a heavy edit, I gave the model the whole article and prompted it to restructure it preserving the argument, facts, and authorities cited. Light editing only allowed small cosmetic or flow changes. I also experimented with a combination of human and LLM-generated sentences until I obtained 4,464 texts covering as full a gamut of model involvement I could reasonably examine.
On these texts, I tested several detectors. Four of these, DeskLib DeBERTa, TMR RoBERTa, MPU v3, and SCRN, had been elsewhere trained on examples of human and LLM writing. The other two, Falcon Binoculars and Fast-DetectGPT work differently, essentially asking a language model whether a passage feels unusually predictable. A seventh method was also pursued by fine-tuning a language encoder, ModernBERT on the corpus and generations.
Detecting LLM-generated text requires some compromises. Specifically, it requires one to accept a proportion of false positives. I set the threshold somewhat conservatively, so that about one genuine 2015 to 2020 post in a hundred would be falsely flagged, and then adjusted until the detectors could reliably identify over half of the LLM-generated texts for both corpora. The mission, as it is sometimes said, was failed successfully: used with a strict threshold, Falcon Binoculars and Fast-DetectGPT were able to detect about 98 per cent of the Gemini drafts, but would completely disregard Claude Sonnet 5 or GPT-5.4 Nano mock posts.
Would it be possible to improve the result by simply accepting a slightly greater number of false positives? To an extent, yes, for some of the detectors. For DeskLib, accepting a threshold where three or four certifiably human (that is to say, pre-2022) control posts would be flagged in the EJIL:Talk! and Opinio Juris corpora respectively, the number of detected AI mock posts increased significantly. Conversely, in the case of Falcon Binoculars and Fast-DetectGPT, lowering the threshold enough to reliably detect a majority of AI-generated mock posts in for models from every vendor would have flagged the majority of certifiably human. That is no reliable tool.
Figure 2. How many genuine pre-ChatGPT posts had to be wrongly flagged before each detector could reliably detect a majority of every current-model drafting and rewriting group. DeskLib required false-positive rates of 2.8 and 4.4 per cent. Fast-DetectGPT and Falcon required rates above 60 per cent. Lower is better.
The fine-tuned ModernBERT classifier performed a bit better, but it also invariably flagged as AI generated the vast majority of lightly polished posts. The classifier could, in other words, recognise prose that a model had touched, but not how important the model’s contribution had been. RAID and DetectRL report the same general difficulty: results change with generator, subject, passage length and human revision. Above all, my ModernBERT fine-tune was trained within this project, and it might have learned something peculiar to our prompts or generated examples. Accordingly, it was not reliable enough for our purpose.
Big Guns, Big Bucks: Better, More Expensive, Less Transparent Detectors
I decided to also test Pangram, a commercial (and painfully expensive) detector with a particularly good reputation for spotting model writing, even in difficult tests where a model is asked to generate a random (the crux of the matter) list of words. I tested a sample of known-origin posts as well the entire 2025 corpus with Pangram 3, the slightly worse (but cheaper) model they offer. While this is a commercial vendor and I cannot examine the code, it has been used somewhat successfully in similar studies: Josh Morrow ran it on over 2,250 US appellate opinions, and, as Eugene Volokh reported, while a January 2022 baseline of roughly 300 opinions was not flagged at all, more than fifty opinions from 2026 promptly did.
In my testing, Pangram performed admirably. It correctly identified all 250 certifiably human (again, meaning pre-ChatGPT) articles and all twenty light copyedits “Human”. The sixty complete drafts generated with current models were found to be either “AI” or “Mixed”. Of these, seventeen out of twenty Claude drafts were strictly “AI”, as were all twenty Gemini drafts and nineteen of twenty GPT-5.4 Nano drafts. Interestingly, but perhaps not surprisingly (there might be limited value in training on obsolete models), the older GPT-3.5 was much harder to catch.
Table 1. Pangram 3.3.2 on texts of known origin
Note: “AI or Mixed” combines Pangram’s two non-human document labels. The twenty light copyedits were drawn across six current model families.
Figure 3. Left: Legal ModernBERT and Pangram 3.3.2 on the same 350 known-origin texts.
Pangram was also much better at classifying uses of AI to polish the text. On a sample of 350 known-origin texts, ModernBERT and Pangram were both very accurate on the 250 verifiably human posts the entirely AI drafts from current models. However, ModernBERT flagged sixteen out of twenty light-polish copyedits, where Pangram, correctly, flagged none.
With this confidence, I applied Pangram to all 677 substantive posts published in 2025, removing metadata and tags. The averages of Pangram’s estimate of how much machine text appears inside every article produced are displayed in Table 2.
Table 2. Pangram v3.3.2 classifications of substantive posts published in 2025
Note: Author names and biographies were removed before scoring. The final column is the average of Pangram’s estimated machine-text fraction across all posts. It is not the percentage of posts classified as AI.
It is therefore implausible that the 2025 corpus contains no model-written text. But of course, a “Mixed” label may reflect different degrees of editing, and I have not tested, for example, to what extent a translation might get flagged as a wholly AI contribution.
Is Regulating Provenance a Disguised Regulation of Resemblance?
Texts have always been made with tools of some sort. Dictation allowed Milton to compose Paradise Lost aloud, and Henry James’s late style is commonly attributed to dictation. Kittler and Kirschenbaum, in very different ways, show that writing technologies participate in shaping thought. Gen AI, however, is unique in its capacity to promptly provide polished prose, as well as arguments, leaving no trace except a stylistic one.
My results support the conclusion that both blogs changed, stylistically, after 2022. Pangram found the difference meaningful in 2025 posts against a control. But even an eerily accurate AI detector cannot answer the question that EJIL’s policy asks: whether the human contribution clearly outweighs the model’s. A detector cannot enforce this distinction.
Regulating provenance by resemblance – or by the required inclusion of watermarking, as per Article 50 of the EU AI Act – does nothing to show who supplied the ideas or why the model was used. Even more problematic, two can play that game: a public watermarks-remover toolkit was published almost immediately after Anthropic announced its watermark policy.
These questions are not inconsequential – in the NeurIPS 2026 Position Paper Track the organizers partnered with Pangram, resulting in 178 rejections and another 123 requests to authors to prove or clarify the extent of their human involvement (the opposite of what the Agentic Law Journal provocatively asks). In other words, a rule intended to ensure provenance is enforced by leveraging evidence of resemblance.
Any tenable Gen AI policy should resist that slide. Detectors may, of course, justify further investigations, but they cannot by themselves establish or rule out authorship or misconduct.
Declaration regarding generative AI: OpenAI Codex, DeepSeek through OpenCode, Claude through Claude Code, and Gemini 2.5 Flash-Lite through the Google Gen AI API assisted with the software implementation, classification of posts, checking, and the like; six model families supplied experimental outputs. The author designed the study, made the methodological decisions, checked sources and calculations, wrote and revised the text, and accepts responsibility.