We've devoted much of this year and this particular space to confronting capital-B Big Issues -- everything from access to justice, to the gap in technology competence, to the corrosive effects of discovery-driven gridlock, to the disincentives stemming tides of change in legal, to the data saturation that is threatening to bring the US court system to its knees.
For our final posts of the year, we decided to cram all those topics into one wide-ranging conversation with one of the relative few people in the country who can speak articulately, candidly, and from experience, about all of them: US Magistrate Judge Kristen L. Mix.
In a 2005 "Meet the Rainmakers" feature for Law Practice Today, Mix, then a distinguished trial lawyer in Denver, described herself as "highly organized," "thorough," "efficient," "suffering no fools lightly" and "actively interested in making a difference." In Logikcull, she has a kindred spirit.
In part one, we focus on blind spots and black boxes -- unknowns of sorts. Judge Mix addresses:
- E-discovery competence: "Occasionally I see folks who don't know anywhere near what they should know about electronic discovery and how it works."
- E-discovery costs: "They are largely opaque to me.... There isn't any effort to train judges about what the marketplace looks like."
- Proportionality determinations in the absence of cost visibility: "There are many tools with which I am utterly unfamiliar. But I'm in the position, like my colleagues, of making decisions as to the appropriate use of these tools and the costs of these tools and who bears the cost."
- How much discovery is too much: "If it's going to cost in excess of 50 percent of the highest value of the case that anybody's come up with, that is clearly disproportional in my view... But there are no bright lines."
- Predictive coding use: "A lot of what lawyers do with respect to e-discovery is opaque to me unless there's a dispute."
- Technology-based disputes: "The extent of those challenges has all related to predictive coding."
Part two, which focuses on structural challenges to the legal system, will be posted later this week.
The grade she'd give the US judicial system's ability to deliver just and inexpensive outcomes amid the incursion of technology and data
It's improved in the time I've been on the bench, which is more than eight years... I think on the whole, we're at a solid B. The learning curb was sharp at first for electronic discovery in particular, but I think through the efforts of the Federal Judicial Center and private entities like Sedona Conference, there have been a lot of attempts to bring judges up to speed.
On the general competence level of practitioners facing e-discovery
The ones that appear before me have a better grasp of it than they used to... It helps that there's a certain level of expectation set by the court, and by the magistrate judges in particular, with respect to these issues. Some courts, like ours, have protocols for e-discovery, and they recommend in the protocols that attorneys familiarize themselves with certain aspects of e-discovery, and reach out to opposing counsel to make agreements about how it will be handled. If that kind of expectation is posted on the court's website and made clear to attorneys that practice in that jurisdiction, I think attorneys take the obligation fairly seriously to either inform themselves or find someone who has worked with it before.
There's an exception to every rule. Occasionally I see folks who don't know anywhere near what they should know about electronic discovery and how it works. But by and large, lawyers tend to be pretty well prepared and understand what they need to understand to address these issues at the beginning of the case -- which is really when they need to be addressed.
On how economic resources correlate to e-discovery competence
Lawyers from bigger firms generally tend to have more familiarity with technology tools, which can be expensive. And lawyers at bigger firms tend to have bigger budgets with which to work -- and therefore they tend to have used or have a colleague that has used e-discovery tools. This is speaking in the most general terms.
The other differentiator is practice area. Lawyers who practice in intellectual property, for example, tend to be much more familiar with the e-discovery lingo and software. Aside from that, it's hard to generalize.
On the visibility judges have into e-discovery costs for the purposes of making proportionality determinations
Those costs are largely opaque to me. Let's be real: I don't use e-discovery tools in my daily job. I came on the bench in 2007 when there were some of those tools available, but not anywhere near the number of tools available today. There are many with which I am utterly unfamiliar. But I'm in the position, like my colleagues, of making [decisions as to] the appropriate use of these tools and the costs of these tools and who bears the cost. So what I need from the parties is to be informed about the how the tool works, why it's the best choice for what the parties are attempting to obtain, and how much it costs. And I have had circumstances where the parties have advocated different tools and they have been very clear and precise in setting forth the benefits of different tools, and the cost of the different tools. And they've even offered expert testimony relating to the cost of the utilization of the different tools. That's very cumbersome, but when the stakes are high enough, it can save a lot of time and money to better inform the judge at the beginning about what's really involved here.
"E-DISCOVERY COSTS ARE LARGELY OPAQUE TO ME. LET'S BE REAL: I DON'T USE E-DISCOVERY TOOLS IN MY DAILY JOB."
On what can be done to increase visibility into e-discovery costs
I really wish that there was a program for judges that addressed this issue. I wish that judges, when they first came on the bench, could receive some baseline level training about electronic discovery tools. We learn a lot when we first come on the bench and of course we learn a lot over the course of our jobs as judges. But at least when I came on the bench, and even now, there isn't any effort to train judges -- even magistrate judges, who tend to deal with these issues relatively frequently -- about what the marketplace looks like and what tools are available to parties and litigants to use, how those tools differ, what the cost associated with those tools generally are, and what the weaknesses and strengths of those tools are. I would find that to be very educational.
It's a difficult topic of course because these tools are created, manufactured and promoted by private entities. And judges have to make decisions about use of tools in a way that's neutral to the profit-making factor. Nevertheless, you can't make those decisions unless you know how the tools work and how much they cost. And you have to rely on the lawyers to do that unless you have some independent education or independent sources of education on that subject. There are some judges, like Judge Richard Posner, who would say, "Get on the internet, and find out what you can find." I'm not sure that approach is universally endorsed.
"JUDGES HAVE TO MAKE DECISIONS ABOUT USE OF TOOLS IN A WAY THAT'S NEUTRAL TO THE PROFIT-MAKING [VENDOR] FACTOR. NEVERTHELESS, YOU CAN'T MAKE THOSE DECISIONS UNLESS YOU KNOW HOW THE TOOLS WORK AND HOW MUCH THEY COST."
On recommending guidelines or estimates for proportionality, such as what percentage of the estimated value of the case discovery should cost
I don't have any rules of thumb. I'd like to be comfortable developing a set of rules about proportionality but proportionality is such a nebulous concept and, of course, there are always two versions of what discovery is proportional in the case because there are at least two versions of what the case is worth. The plaintiff thinks the case is worth a million dollars. The defendant thinks the case is worth $50 dollars. Their views of proportionality are informed by their judgments about the value of the case. So, judges are in a position where we have to use our best efforts to make a reasonable decision about what's proportional in light of the case. And factors other than the value of the case, of course, can enter into it.
"PROPORTIONALITY IS SUCH A NEBULOUS CONCEPT. THERE ARE ALWAYS TWO VERSIONS OF (HOW MUCH) DISCOVERY IS PROPORTIONAL IN THE CASE BECAUSE THERE ARE AT LEAST TWO VERSIONS OF WHAT THE CASE IS WORTH."
If you're talking about discovery that's going to cost in excess of 50 percent of the highest value of the case that anybody's come up with, that is clearly disproportional in my view. But aside from that kind of an egregious circumstance, which I think probably would never come up, it's just hard to draw a bright line. But different factors end up influencing the court in its determination of proportionality and it has to be done on a case by case basis. And cost is one of the factors that obviously has to be considered.
On how often predictive coding or other analytics-based discovery methods are used in her cases
It's a great question, and I always start off answering it by saying that a lot of what lawyers do with respect to e-discovery is opaque to me unless there's a dispute, or unless they have put together a proposed e-discovery protocol which they have asked me to approve as an order of the court. In those rare circumstances where there's an actual e-discovery protocol ordered by the court, often those protocols include predictive coding. But I have e-discovery protocols in less than 10 percent of my civil cases. So the lawyers may be using predictive coding a whole lot, but I may just not be aware of it because unless they have a dispute about it, it isn't brought to my attention. And if they don't have any discovery protocol, I'm not aware of it.
"LAWYERS MAY BE USING PREDICTIVE CODING A WHOLE LOT, BUT I MAY JUST NOT BE AWARE OF IT BECAUSE UNLESS THEY HAVE A DISPUTE ABOUT IT, IT ISN'T BROUGHT TO MY ATTENTION."
Occasionally predictive coding will be mentioned in the scheduling order because the parties will address what they attend to do with respect to e-discovery in their scheduling order. Or occasionally I'll get a dispute which makes it clear that predictive coding has been agreed upon to a certain extent, but there's a dispute about search terms for example. But overall, it's a pretty small number of cases that involve that (predictive coding) on an annual basis -- surely, less than 10 percent of my civil docket.
On whether the use of a specific technology or technological process has ever been challenged in her court
Yes, and the extent of those challenges have all related to predictive coding -- in other words, whether the parties can, or the court can, come up with a search protocol that's going to effectively find documents from a vast number of potential documents that are relevant to the litigation. Often the way that dispute is brought to my attention is that there's a test run of a certain body of documents using a search protocol, and the lawyers control this so that, [hypothetically], out of this [control set of] 1000 documents, a hand search has produced 550 that have relevant terms based on what the subject matter of the litigation is. When they do it by predictive coding, they come up with 200. That kind of argument is made in attempt to convince the court that predictive coding is not an efficient method for searching, and that something else has to be done.
The problem I have with a judge is that if the parties don't have a suggestion about how to handle these types of disputes aside from predictive coding, you know, I certainly don't have a suggestion. I'm not going to out of whole cloth construct some type of e-discovery protocol for the parties, especially one that doesn't use predictive coding. If the parties are dissatisfied with that, or if one party is dissatisfied with predictive coding, they have to propose alternatives, and they have to make an argument about the efficiency of the alternatives compared to predictive coding -- and the costs of the alternatives as well. So the disputes I have about use of e-discovery methods have all related to predictive coding and whether the party seeking the information is going to actually get the information using the technology as it's been presented by the opposing party.
As told to Robert Hilson, a director at Logikcull. He can be reached at robert.hilson@logikcull.com.
