ChatGPT Year in Review + GenAI Look Ahead to 2024
ChatGPT Year in Review + GenAI Look Ahead to 2024
Today is the one year anniversary of ChatGPT
One year ago today, ChatGPT was released to the world, and what a year it has been!
I’d been digging deep into GPT-3 throughout the previous year, working with Megan Ma and others to clarify how well or poorly that earlier version of the technology could understand and correctly apply the fiduciary duty of loyalty and other related legal frameworks and rules. I was impressed by what GPT-3 could do, and well aware of its limitations as well. But when I got my hands on GPT-3.5, which is the model that powered ChatGPT, I was - and continue to be - astonished at its human-like natural language capabilities. Within a couple of weeks, I determined the ChatGPT model was performing well on the same fiduciary duties tests and evaluations that had stymied GPT-3 and was able to go well beyond anything I had previously devised to test it. I had to invent new tests just to begin to map the contours of the new capabilities. Here is a snapshot of some of those early tests, from Dec 17th of 2023: https://www.civics.com/pub/chatgpt-session-2022-12-17/ Later this year, when GPT-4 was finally released to the public, it blew away even those boundaries.
I’d first seen GPT-4 a few months before ChatGPT was released, when Pablo Arredondo showed me a private demo of his pre-release version of CoCounsel. So I was aware there were even more powerful models in the pipeline, but being aware and being able to have open, easy, web-accessible access are two very different things. When OpenAI, to their credit, made GPT-4 available to everyone through their premium access and the API, it was a revelation for those of us who apply this technology to legal use cases and more broadly to solve business challenges or realize creative new ideas in industry and the professions.
Meanwhile, I hasten to add there are still serious limits and flaws with this technology. It is prone to so-called hallucination and providing factually incorrect information, for example, and there are myriad conundrums about the role of intellectual property and personal or confidential data, to name a few issues. I have found that anchoring the technology to authoritative data, such as by adding that information to the prompt (e.g., via the context window) or through processes like Retrieval Augmented Generation, the hallucinations go down and the factual grounding goes way up, as well as the reasoning-like process.
In light of these shortcomings, and to begin providing guidance on the responsible use of GenAI for law, I started a Task Force at MIT on the Responsible Use of Generative AI for Law and Legal Processes, which convened a group of super-stars who collaborated to develop some solid, if preliminary, principles and guidelines (published here, toward the bottom of the page: law.MIT.edu/ai). I’m delighted to say that I also served as an advisory member of the California State bar association working group that started with the MIT draft and formulated it into a very solid (not perfect, but very good) set of more formal guidance for attorneys to observe when using this technology as part of their practice (available here: https://board.calbar.ca.gov/docs/agendaItem/Public/agendaitem1000031702.pdf).
The primary need among attorneys at this point in time is to learn about this technology and to acquire skills in knowing how, when, and for what to use the technology. Much of this can happen by simply using the technology with an eye toward experimentation and exploration. To that end, over the past year I’ve personally shared quite a few resources for lawyers, as well as other professionals (e.g., in tax, audit, consulting, etc.) on the emerging skill known as prompt engineering, both through open and free resources at law.MIT.edu/ai and more so through private consulting and workshops. Next year, I’m on track to release several more resources and to provide a few new services to make these skills ever more accessible. More on that in the weeks and months to come.
The past year has seen many people and groups raise a lot of fears and objections and outright resistance to this technology, for a range or reasons and from a range of perspectives and priorities. I think this is natural and I know it is to be expected, and yet, in my view, the temptation - especially among lawyers and institutions of the law - to react with prohibitions and overly restrictive regulations is a mistake. Appropriate regulation and policy should balance the enormous utility of this technology against the largely speculative risks, and in general, the main thrust of investment and policy at this point should be toward the beneficial adoption, adaptation, and leveraging of this very useful new technology and the capabilities it affords. Nonetheless, there are demonstrable limits and flaws with the technology and there remains a distance to go before these and other issues are fully addressed.
In the arena of law, in 2024 we’ll see more helpful guidance along the lines of the pioneering guidelines by the California Bar late this year to help lawyers use and integrate GenAI into their practices in a responsible and ethical manner.
I expect we will also see more focus on the second-order implications of this technology in broader institutional contexts, such as updated training of law students and new hire lawyers, better judicial processes (and perhaps more importantly, non-judicial alternative dispute resolution systems) to make this technology available in effective ways for unrepresented and under-represented litigants and criminal defendants, and some potential reforms in the rules of evidence to better deal with the coming wave of deep fakes, among many other ripples in law and legal processes.
More broadly, I foresee a new area of off-line, on-premises, and even on-device LLMs taking hold this year. For example, this week many tech enthusiasts are talking about llamafile (https://hacks.mozilla.org/2023/11/introducing-llamafile/), a groundbreaking multi-gigabyte file that revolutionizes personal computing by bundling both the model weights and the necessary code to run Large Language Models like ChatGPT on your own device. That’s right - you can now easily run a functional LLM on your desktop and even on your laptop! This innovation marks a significant leap in making advanced AI more accessible, bridging the gap between professional AI applications and everyday tech enthusiasts.
A key implication of this on-device approach to running GenAI is that users can now integrate their own confidential, proprietary, and otherwise sensitive data in a completely air-gapped system. I’ve already been exploring the usefulness and security benefits of this approach with enterprise clients of my consulting company, CIVICS.com, but for this blog post, I want to connect this capability to something even more important, namely, YOU! What I mean is that individuals will soon have the tools needed to easily run our own powerful GenAI systems and we’ll be able to connect the rich treasure trove of our personal data to anchor the technology to our contexts, our knowledge, our relationships, and our unique goals and priorities. My esteemed friend Doc Searls has recently begun speculating about the advent of so-called “Personal AI” (e.g., https://projectvrm.org/2023/11/11/individual-empowerment-and-agency-on-a-scale-weve-never-seen-before/) and I predict this will be among the true killer apps (or more accurately, sets of connected apps) to drive adoption and beneficial use of GenAI in the coming year and beyond.
I also see the emergence of automated or quasi-autonomous personal agents and the increasing integration of Generative AI with a wide set of existing widely used apps and platforms not only as major trends in their own right, but also as capabilities that will be super-charged by on-device models with private access to personal and sensitive data.
In my law.MIT.edu capacity, I’ll be working with our team to kick-start 2024 with some major GenAI initiatives, including the annual MIT IAP Computational Law Workshop happening this January with a remarkable set of speakers, topics, and learning activities, and an associated GenAI Online Legal Hackathon. We’ll be announcing those shortly, and to get on the list you can use our pre-registration form here: https://forms.gle/92WwhEWwpGdLyfE5A The MIT Computational Law Report is also about to announce a special collection on GenAI for Law, which will be featuring written works as well as open source applications and Jupyter or Colab Notebooks representing a range of legal use cases that can be achieved with GenAI. And that’s just January!
In my private capacity, through the CIVICS.com consultancy, I’ll be leaning into projects with companies, law firms, and some open source initiatives, who are making innovative use of GenAI both to make their current work faster, less expensive, and better as well as for creating totally new types of products, services, and even novel lines of business. I have also totally revised my standard lunch-talk and private workshop offerings to provide more accessible and flexible opportunities for companies and legal teams to bring me in for learning sessions or to focus on emerging projects. If you’d like to set up a consultation, a talk, or workshop in 2024, reach out through CIVICS.com here: https://www.civics.com/contact
In my public capacity, I plan to keep contributing to standards efforts and professional association efforts, such as through my membership on the ABA Task Force on AI and contributing to open and free community building and skill sharing efforts, such as through meetups, hackathons, and my favorite group Legal Hackers. In a couple of weeks, for example, anybody who wants to meet up with a group of like-minded creative types to share generative AI prompts, ideas and solutions is invited to join the Bay Area Legal Hackers Happy Hour in Oakland. You can learn more and register for this event here.
Looking ahead, I predict two overarching GenAI trends in 2024, first a lot of catch-up by companies, teams, and individuals who are currently aware of this new technology but only have a superficial understanding of it and few skills in using it. We are still at the early part of the adoption curve, and that is normal. We will climb that curve in the coming year. Second, I foresee a number of major changes and breakthroughs in the technology itself, both in the form of integrations of the technology in current common products as well as totally new capabilities and deployment models. I mentioned on-device and secure personal data design patterns as one example of a new deployment model, and there are many others in the pipeline and some that have not even been conceived of yet.
In the face of all this emerging change, the task today is to get educated. Now.