While I didn't have a chance to catch all the speakers at the RELEVENT (called the un-conference) in Nashville earlier this week, the ones I did see were great! The topics for the conference varied from legal marketing to eDiscovery to cost control. many of the speakers were able to offer real-world experience and scenarios, which added value to the talks.
Some of the highlights:
I have to say that Casey Flaharty from KIA motors did not disappoint. After finding a little bit about his family, he moved on to how he holds his outside counsel's feet to the fire, using a test or "audit", mainly for the associates. His perspective is that they are doing the majority of the often mundane work, so he wants them to have a minimum level of technology skill. His test, which he says takes him less than an hour to complete, takes a shocking average of 5 hours for his outside counsel's associates to complete! If they do poorly, he asks for a discounted rate. Seem fair.
Babs Deacon gave an entertaining, yet thought-provoking talk about the state and future of eDiscovery. After a few comments about the many problems relating to eDiscovery, she got to the nugget of her talk: Her philosophy that the future of review will take place in situ, or in place. On other words, in the future, instead of doing discovery "productions", organizations will simply allow access to their data. She admitted that there are several hurdles, but this was her vision.
Kelly Tigger, from ESI Attorneys, Inc. gave the audience an update on the latest intersections between the law and social media, mainly as related to eDiscovery. Among other issues, she cautioned attorneys about automatic cross-posting (LinkedIN > Twitter > Pinterest > Facebook). She prompted an interesting discussion about appropriate use of social media by attorneys as well as the pitfalls of social medai in eDiscovery.
There several other informative panels that discussed topics from legal marketing, to law school curricula to entrepreneurship. This is definitely a conference to keep an eye on as I believe many more of the industry movers and shakers, as well as us regular folks, will be looking to share their ideas and experiences. Check cicayda.com for the next RELEVENT.
Examining the Costs and Risks Related to Electronic Discovery
Thursday, October 10, 2013
Friday, March 15, 2013
5 Questions General Counsel Should Ask Themselves Before The Next Big eDiscovery Matter
1. How Can I Leverage My Records Management Program To Support My Organization's eDiscovery Efforts?
Gone are the days when Information Technology (IT) was the only part of the organization to have and need knowledge about where electronically stored information (ESI) resides. While the updated federal rules don't require that counsel know exactly where their organization's ESI is at all times, the rules make counsel accountable for that data's preservation, review and production.
A good records management program tracks information about ESI like custodian or owner, type, dates and location, as well as policies related to the retention or destruction of the ESI. Techniques such as data-mapping can provide this information in a condensed, easily retrievable format. This information can in turn be used for quick access to ESI for litigation. Knowing the custodian and where their related ESI resides can save time, money and resources when it's time to collect the ESI for search and review.
One must be careful, however, that the records management policies respect the litigation hold process. So another aspect of a good records management program is that it can be flexibile and allow for exceptions.
2. How Do I Balance Defensibility and Cost When Preserving and/or Collecting ESI?
"Dragging and dropping" or simply copying ESI to a hard drive or CD is becoming a less acceptable option for the preservation or collection of ESI. As more attorneys become familiar with metadata, like file creation or modification times, and realize the value of metadata, more will demand that ESI metadata be preserved, and that copies of ESI match the original ESI.
Does that mean you have to hire a forensic technician for all your collections? No. But if you are going to do the preservation and collections in-house, your staff must be trained on the importance of preserving the integrity of the ESI. Several tools are starting to emerge that automate the process so that someone with low technical skills - or even the custodian - be able to defensibly copy ESI. (See RemLox or BlackBox)
Because IT operations are not consistent, and sometimes even contrary to eDiscovery processes, IT must be trained on handling ESI for litigation. One way is to have a consultant help draft procedures that are geared towards your organizations infrastructure. Then, when litigation hits, have the consultant provide guidance during the process, looking over the shoulders of the IT personnel, helping them with the current matter and with refining the procedures for the next matter. After a couple of matters, your procedures will be refined and your IT people will be ready to take on the tasks themselves.
One caveat to this is the danger of having IT too involved in controversial ESI or a highly contentious matter, and providing opinions related to the ESI at issue. This, in my opinion for a variety of reasons, should be handled by a consultant accustomed to testifying in court and explaining technical concepts to attorneys, judges and juries.
Just to drive home the point, in a recent matter (Brown v. FPI Mgmt, No. 4:11-cv-5414 YGR (KAW), 2013 U.S. Dist. LEXIS 1040 (N.D. Cal. Jan. 3, 2013)), the court held that the Defendant did not make a good faith effort to search their documents. This could have been avoided if the Defendant had hired a consultant to find a way to manage the document search, or had at least hired a consultant to support their argument that searching the documents was overly costly and burdensome. It appears that the "our system just isn't meant to be searched" argument will no longer fly.
3. I Keep Hearing About Predictive Coding. Should I Be Using It?
I've not seen anything in eDiscovery like the buzz about predictive coding. While I don't think it's just a buzz, but rather a legitimate new and useful technology, I do think that it's misunderstood. The idea behind predictive coding is to have the computer use an algorithm to do the work of categorizing documents using textual patterns in the documents. An attorney goes through a small portion of representative documents during the "training" process, in which the algorithm "learns" the different categories of documents. Eventually, in theory, the algorithm will then be able to look at a document and categorize it without human help. Some claim that review costs can be reduced by up to 98% using predictive coding.1
While there is some data to support claims of it's effectiveness2, there exist, however, certain problems inherent in the technology. First, representative documents must be properly identified during the training process so the computer can "learn" which documents go in which category. If one category is under-represented, it could throw off how the algorithm assigns future documents. So training is important and must be performed until some type of stability is reach. In other words, training must continue until the algorithm predicts a document's category at an acceptable rate.
Second, predictive coding is not free. Industry standards are somewhere around $0.05 to $0.09 per document. Industry standard for document review is around $1 per document. So two factors come into play here: (1) Are there enough documents so that the training process pays off, and (2) Will you really save any money by using predictive coding rather than traditional technology-assisted-review techniques, such as Boolean or concept searching. You will need to find that break-even point. I've heard the break-even to address the training is at least 50,000 to 100,000 document. Which means that predictive coding shouldn't even be considered for smaller document populations.
Third, not all document productions are conducive to predictive coding because of their nature. For example, if you have a wide variety of document types you are searching, patterns may not exist to allow predictive coding algorithms to be of much help. Predictive coding works best when patterns within documents can be identified. I can see how employment matters would not be good candidates for predictive coding, whereas intellectual property cases, like patent or other disputes where technical terms and concepts are at issue might be good candidates.
Finally, the issue of court acceptance must be considered. While there have only been a handful of cases that I know of in which predictive coding has been addressed, if you are practicing in an area where technology is welcomed by the courts, court acceptance may not be an issue. However, if you practice in an area where courts are still struggling with accepting some technologies - either because of lack of exposure or lack of opposing counsel's knowledge or acceptance - you may have to educate the parties.3
4. My Organization Has Only Been Involved A Few Small eDiscovery Matters. Should We Hire A Consultant To Help If We Get A Big eDiscovery Case?
While handling low-value, repetitive eDiscovery matters in-house can make sense, there will most likely come a time when the complexity or size of an eDiscovery matter requires outside help. This expertise comes at a price, but there are ways to use a consultant intelligently so that their value is optimized. For example, use the consultant wisely for high-level tasks with your team watching and learning. Encourage your team to ask questions to understand why the consultant is doing what they are doing and why they are doing it. Likewise, make sure the consultant understands that part of the purpose of their engagement is to train your team.
A good, battle-scarred consultant possesses valuable lessons learned that were financed by someone else along the way. It's advantageous for you, as general counsel, to leverage this knowledge and put it to use in your organization. A good consultant can help identify pitfalls during the process that can save valuable resources. As your in-house team works with the consultant, they are able to pick up on best pratices so that next time they are more prepared for the response to complex litigation. Over time, the consultant can be used less and less as your team improves their own expertise, thus saving the organization money in the long run.
Finally, a seasoned consultant can help with your offense. Since they have most likely been on the receiving end of several types of attacks, they bring a valuable arsenal for going after the opposing side and their experts. Aiding with legal offense is one of the most effective uses of a consultant.
5. What Other Actions Can I Take That Would Help Me Prepare For Large eDiscovery Matters?
As the Boy Scouts say, "Be Prepared". eDiscovery truly is a "pay me now or pay me more later" proposition. A small investment in readiness can save a lot of time and resources later. Whether, by training your personnel or hiring a consultant to help your team put together a litigation readiness plan, doing some preparation outside of the potentially chaotic environment of discovery for litigation makes a lot of sense. In addition, you might consider:
In summary, general counsel should do their best to apply technologies and other resources like consultants intelligently. One technology or software tool may work for one organization, but not fit with another. Counsel should consider developing a tracking mechanism to identify the most common types of cases for the organization and act appropriately based on those data. After all, every organization is unique and every legal matter is unique.
References
1. Symantec Website: http://www.symantec.com/predictive-coding
2. Maura R. Grossman & Gordon V. Cormack, "Technology-Assisted Review In E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review", Richmond Journal of Law and Technology, Vol. XVII, Issue 3: http://jolt.richmond.edu/v17i3/article11.pdf
3.Jonathan, Berman. Jones Day Website, "Predictive Coding—A Dispatch From the Front Lines of E-Discovery", http://www.jonesday.com/predictive_coding/
Gone are the days when Information Technology (IT) was the only part of the organization to have and need knowledge about where electronically stored information (ESI) resides. While the updated federal rules don't require that counsel know exactly where their organization's ESI is at all times, the rules make counsel accountable for that data's preservation, review and production.
A good records management program tracks information about ESI like custodian or owner, type, dates and location, as well as policies related to the retention or destruction of the ESI. Techniques such as data-mapping can provide this information in a condensed, easily retrievable format. This information can in turn be used for quick access to ESI for litigation. Knowing the custodian and where their related ESI resides can save time, money and resources when it's time to collect the ESI for search and review.
One must be careful, however, that the records management policies respect the litigation hold process. So another aspect of a good records management program is that it can be flexibile and allow for exceptions.
2. How Do I Balance Defensibility and Cost When Preserving and/or Collecting ESI?
"Dragging and dropping" or simply copying ESI to a hard drive or CD is becoming a less acceptable option for the preservation or collection of ESI. As more attorneys become familiar with metadata, like file creation or modification times, and realize the value of metadata, more will demand that ESI metadata be preserved, and that copies of ESI match the original ESI.
Does that mean you have to hire a forensic technician for all your collections? No. But if you are going to do the preservation and collections in-house, your staff must be trained on the importance of preserving the integrity of the ESI. Several tools are starting to emerge that automate the process so that someone with low technical skills - or even the custodian - be able to defensibly copy ESI. (See RemLox or BlackBox)
Because IT operations are not consistent, and sometimes even contrary to eDiscovery processes, IT must be trained on handling ESI for litigation. One way is to have a consultant help draft procedures that are geared towards your organizations infrastructure. Then, when litigation hits, have the consultant provide guidance during the process, looking over the shoulders of the IT personnel, helping them with the current matter and with refining the procedures for the next matter. After a couple of matters, your procedures will be refined and your IT people will be ready to take on the tasks themselves.
One caveat to this is the danger of having IT too involved in controversial ESI or a highly contentious matter, and providing opinions related to the ESI at issue. This, in my opinion for a variety of reasons, should be handled by a consultant accustomed to testifying in court and explaining technical concepts to attorneys, judges and juries.
Just to drive home the point, in a recent matter (Brown v. FPI Mgmt, No. 4:11-cv-5414 YGR (KAW), 2013 U.S. Dist. LEXIS 1040 (N.D. Cal. Jan. 3, 2013)), the court held that the Defendant did not make a good faith effort to search their documents. This could have been avoided if the Defendant had hired a consultant to find a way to manage the document search, or had at least hired a consultant to support their argument that searching the documents was overly costly and burdensome. It appears that the "our system just isn't meant to be searched" argument will no longer fly.
3. I Keep Hearing About Predictive Coding. Should I Be Using It?
I've not seen anything in eDiscovery like the buzz about predictive coding. While I don't think it's just a buzz, but rather a legitimate new and useful technology, I do think that it's misunderstood. The idea behind predictive coding is to have the computer use an algorithm to do the work of categorizing documents using textual patterns in the documents. An attorney goes through a small portion of representative documents during the "training" process, in which the algorithm "learns" the different categories of documents. Eventually, in theory, the algorithm will then be able to look at a document and categorize it without human help. Some claim that review costs can be reduced by up to 98% using predictive coding.1
While there is some data to support claims of it's effectiveness2, there exist, however, certain problems inherent in the technology. First, representative documents must be properly identified during the training process so the computer can "learn" which documents go in which category. If one category is under-represented, it could throw off how the algorithm assigns future documents. So training is important and must be performed until some type of stability is reach. In other words, training must continue until the algorithm predicts a document's category at an acceptable rate.
Second, predictive coding is not free. Industry standards are somewhere around $0.05 to $0.09 per document. Industry standard for document review is around $1 per document. So two factors come into play here: (1) Are there enough documents so that the training process pays off, and (2) Will you really save any money by using predictive coding rather than traditional technology-assisted-review techniques, such as Boolean or concept searching. You will need to find that break-even point. I've heard the break-even to address the training is at least 50,000 to 100,000 document. Which means that predictive coding shouldn't even be considered for smaller document populations.
Third, not all document productions are conducive to predictive coding because of their nature. For example, if you have a wide variety of document types you are searching, patterns may not exist to allow predictive coding algorithms to be of much help. Predictive coding works best when patterns within documents can be identified. I can see how employment matters would not be good candidates for predictive coding, whereas intellectual property cases, like patent or other disputes where technical terms and concepts are at issue might be good candidates.
Finally, the issue of court acceptance must be considered. While there have only been a handful of cases that I know of in which predictive coding has been addressed, if you are practicing in an area where technology is welcomed by the courts, court acceptance may not be an issue. However, if you practice in an area where courts are still struggling with accepting some technologies - either because of lack of exposure or lack of opposing counsel's knowledge or acceptance - you may have to educate the parties.3
4. My Organization Has Only Been Involved A Few Small eDiscovery Matters. Should We Hire A Consultant To Help If We Get A Big eDiscovery Case?
While handling low-value, repetitive eDiscovery matters in-house can make sense, there will most likely come a time when the complexity or size of an eDiscovery matter requires outside help. This expertise comes at a price, but there are ways to use a consultant intelligently so that their value is optimized. For example, use the consultant wisely for high-level tasks with your team watching and learning. Encourage your team to ask questions to understand why the consultant is doing what they are doing and why they are doing it. Likewise, make sure the consultant understands that part of the purpose of their engagement is to train your team.
A good, battle-scarred consultant possesses valuable lessons learned that were financed by someone else along the way. It's advantageous for you, as general counsel, to leverage this knowledge and put it to use in your organization. A good consultant can help identify pitfalls during the process that can save valuable resources. As your in-house team works with the consultant, they are able to pick up on best pratices so that next time they are more prepared for the response to complex litigation. Over time, the consultant can be used less and less as your team improves their own expertise, thus saving the organization money in the long run.
Finally, a seasoned consultant can help with your offense. Since they have most likely been on the receiving end of several types of attacks, they bring a valuable arsenal for going after the opposing side and their experts. Aiding with legal offense is one of the most effective uses of a consultant.
5. What Other Actions Can I Take That Would Help Me Prepare For Large eDiscovery Matters?
As the Boy Scouts say, "Be Prepared". eDiscovery truly is a "pay me now or pay me more later" proposition. A small investment in readiness can save a lot of time and resources later. Whether, by training your personnel or hiring a consultant to help your team put together a litigation readiness plan, doing some preparation outside of the potentially chaotic environment of discovery for litigation makes a lot of sense. In addition, you might consider:
- Developing templates and plans for litigation holds and preservation letters for the types of litigation your organizations faces, and look into tools that may help smooth the processes around those templates.
- Thinking ahead about checklists for different phases such as meet and confer and discovery so that you have a list of everything to pick from or whittle down instead of making it up as you go.
- Consider having a standard ESI protocol that addresses custodians, data types, data locations and data sources, so that you can be more prepared than opposing counsel to address the relevant ESI.
In summary, general counsel should do their best to apply technologies and other resources like consultants intelligently. One technology or software tool may work for one organization, but not fit with another. Counsel should consider developing a tracking mechanism to identify the most common types of cases for the organization and act appropriately based on those data. After all, every organization is unique and every legal matter is unique.
References
1. Symantec Website: http://www.symantec.com/predictive-coding
2. Maura R. Grossman & Gordon V. Cormack, "Technology-Assisted Review In E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review", Richmond Journal of Law and Technology, Vol. XVII, Issue 3: http://jolt.richmond.edu/v17i3/article11.pdf
3.Jonathan, Berman. Jones Day Website, "Predictive Coding—A Dispatch From the Front Lines of E-Discovery", http://www.jonesday.com/predictive_coding/
Sunday, February 17, 2013
eDiscovery Economics: Part 2
In Part 1 we looked at the economic theory in play during litigation and applied some sample data to analyze the dynamics. Now that we have some of the basics of the economic models for litigation, let's look at the dynamics during the eDiscovery process, as specified by the Electronic Discovery Reference Model (EDRM). The EDRM is shown graphically below (courtesy of EDRM.net).
The phases (or stages) of this model are:
Information management refers less to a distinct eDiscovery phase and more to the ongoing record-keeping rules and procedures. As such, it so does not have a large role in eDiscovery dynamics, except perhaps that having good records management may help to reduce the overall cost of the identification, preservation and collection phases.
The identification phase is where legal counsel and their agents identify custodians and relevant electronically stored information (ESI). This phase is important, since incorrectly executing it, by missing custodians or relevant topics, may result in having to re-do a lot of the work in subsequent phases. I think of this phase as the design phase. If executing the EDRM is like building a house, this phase is where the blueprint and specifications for the house are created.
Preservation and collection are sometime combined, however, I see a distinct difference between these phases in some situations. Preservation can simply mean implementing rules that cause relevant ESI to not be deleted. Preservation can also mean making a weekly, bi-weekly or monthly copy of relevant ESI. Preservation is for securing the relevant ESI. Collection is the processing of gathering up the relevant ESI specifically for moving through the rest of the EDRM. Sometimes these are two distinct phases, sometimes they are not.
Processing is where de-NIST (removing system and other know files), de-duplication, filtering (removing files by date range, custodian, file type, etc.) and searching (removing files by keyword or keyword combinations) happen. This is also where technology-aided review (TAR), like predictive coding begins and continues into the review phase. The purpose of this phase is to reduce the amount of ESI that goes into the analysis and review phases.
Analysis and review involve looking at the ESI to see what is there. During analysis, counsel may realize that some important known documents got left behind, so the keywords need to be revisited. Perhaps one custodian worded a relevant issue or item one way, while another worded it another way. Both need to be captured.
Review is normally the most costly part of the EDRM. Estimates put average review costs at around $1 per document1. This is the phase where ESI that made it to this point is evaluated and classified as relevant, privileged, etc.
Production involves taking the relevant ESI and producing it in the agreed-upon format (native, image files, load files, etc.) to opposing counsel or for your own legal counsel team.
Presentation if the ESI takes place in a deposition, hearing or trial. It is simply the ESI presented and explained before an audience.
It's important to note that the EDRM is iterative, meaning that all throughout most of the EDRM, legal counsel may go back to revisit a previous phase, then move forward, then go back again. This is actually one of the ways that I recommend handling ESI, since the eDiscovery team (counsel, client, consultants, etc.) often learn more information as the EDRM progresses and must take this new information and apply it to the EDRM. This can often mean adding new custodians, changing search keywords or adjusting review criterion.
eDiscovery Dynamics
Now let's look at some of the dynamics during the eDiscovery process, in particular, the costs and the probabilities. Recall equation (5) from Part 1,
which demonstrates the S that will result in a settlement.
Keeping this in mind, and focusing on the costs for a minute, most estimates place the highest costs in the processing and review phases of the EDRM. Processing can make up anywhere from 15% to 40% of the total cost, and review can take 50% to 80% of the total. A recent RAND study2 asserts the following normalized percentages per phase cost based on empirical data:
The RAND cost curve shown below is similar, except missing the production piece.
These curves look like they could be approximated with the sigmoid or "S-curve" function.
The curve starts with a low cost and builds over time, with an acceleration during review, then decelerates as it approaches the total cost.
So let's apply this to the eDiscovery process and revisit our settlement functions. Going back to Part 1, and the example we used in which two parties were involved in a lawsuit where the settlement was $50,000, each of their costs were $10,000 and we let the probabilities vary.
Applying the s-curve eDiscovery cost function above and assuming both plaintiff and defendant total costs (they both approach $10,000), the probabilities are both 50% and cost functions are the same. The settlement curves, plotted by day as the eDiscovery process progresses and the costs go up, should look something like
Recall from Part 1 that a settlement is possible in the area below the defendant's curve (Sd) and above the plaintiff's curve(Sp).
But if the defendant's costs are more expensive than the plaintiff's (growing towards $20,000 instead of $10,000), then we see a wider gap between the curves, a steeper, faster climb of the defendant's settlement curve and a much better chance for a settlement, as shown below.
Making the total final costs even again at $10,000, if we look at the two extremes where the probabilities are both 10%, we get the graph below, where a settlement is always possible at under $15,000.
When both parties believe there is a 90% chance that the plaintiff will win, we get the curves below, which show the settlement curves shifted up and a settlement always possible under $55,000.
Finally, the case where there is vast disagreement about the probabilities is shown below. Here, a settlement cannot be made at any point, since at no point is the plaintiff's curve below the defendant's curve.
But it's probably not realistic to assume the probabilities stay the same throughout discovery. Disclosure of information should bring the probabilities together. Below shows the settlement curves as well as the probability curves, where the probabilities start apart and converge at 50% halfway through discovery. A logarithmic scale is used so both sets of curves can be shown on one graph.
We can see that in the beginning a settlement is not possible. But as the probabilities cross and reach the 50% mark, a settlement becomes possible.
Conclusion
Hopefully the dynamics shown here have shed some light on how parties are expected to behave during discovery, especially considering the costs and dynamics of eDiscovery. If one wants a settlement, discovery is potentially the best time to press for it, since that's when a lot of the costs and probabilities shift. The conclusion based on what we've seen here is that the best way to a settlement is:
(1) Make sure your opponent is spending a lot (at least more than you), and
(2) Use discovery to change the opposing side's probability by sharing information as soon as you get it to keep your own costs down.
Another dynamic which a lawyer friend of mine suggested was the theory that once a party is already in for a certain amount, they might as well stay in and see the matter through. This is similar to a game theory premise that poker players face: If you're already playing at a loss, what's a little more money to play out the game and see where it goes? I have not represented that dynamic here, but it may be an interesting one to consider.
Next time we'll look at some empirical studies to try to tackle eDiscovery cost estimation.
References
1. Palazzolo, J., "Why Hire a Lawyer? Computers Are Cheaper", http://online.wsj.com/article/SB10001424052702303379204577472633591769336.html
2. Pace, Nicholas M. , Laura Zakaras, "Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery", http://www.rand.org/content/dam/rand/pubs/monographs/2012/RAND_MG1208.pdf
3. Degnan, D., "Accounting for the Costs of Electronic Discovery", Minnesota Journal of Law, Science & Technology. 2011;12(1):151-190. http://mjlst.umn.edu/prod/groups/ahc/@pub/@ahc/@mjlst/documents/asset/ahc_asset_366139.pdf
The phases (or stages) of this model are:
- Information Management
- Identification
- Preservation
- Collection
- Processing
- Analysis
- Review
- Production
- Presentation
Information management refers less to a distinct eDiscovery phase and more to the ongoing record-keeping rules and procedures. As such, it so does not have a large role in eDiscovery dynamics, except perhaps that having good records management may help to reduce the overall cost of the identification, preservation and collection phases.
The identification phase is where legal counsel and their agents identify custodians and relevant electronically stored information (ESI). This phase is important, since incorrectly executing it, by missing custodians or relevant topics, may result in having to re-do a lot of the work in subsequent phases. I think of this phase as the design phase. If executing the EDRM is like building a house, this phase is where the blueprint and specifications for the house are created.
Preservation and collection are sometime combined, however, I see a distinct difference between these phases in some situations. Preservation can simply mean implementing rules that cause relevant ESI to not be deleted. Preservation can also mean making a weekly, bi-weekly or monthly copy of relevant ESI. Preservation is for securing the relevant ESI. Collection is the processing of gathering up the relevant ESI specifically for moving through the rest of the EDRM. Sometimes these are two distinct phases, sometimes they are not.
Processing is where de-NIST (removing system and other know files), de-duplication, filtering (removing files by date range, custodian, file type, etc.) and searching (removing files by keyword or keyword combinations) happen. This is also where technology-aided review (TAR), like predictive coding begins and continues into the review phase. The purpose of this phase is to reduce the amount of ESI that goes into the analysis and review phases.
Analysis and review involve looking at the ESI to see what is there. During analysis, counsel may realize that some important known documents got left behind, so the keywords need to be revisited. Perhaps one custodian worded a relevant issue or item one way, while another worded it another way. Both need to be captured.
Review is normally the most costly part of the EDRM. Estimates put average review costs at around $1 per document1. This is the phase where ESI that made it to this point is evaluated and classified as relevant, privileged, etc.
Production involves taking the relevant ESI and producing it in the agreed-upon format (native, image files, load files, etc.) to opposing counsel or for your own legal counsel team.
Presentation if the ESI takes place in a deposition, hearing or trial. It is simply the ESI presented and explained before an audience.
It's important to note that the EDRM is iterative, meaning that all throughout most of the EDRM, legal counsel may go back to revisit a previous phase, then move forward, then go back again. This is actually one of the ways that I recommend handling ESI, since the eDiscovery team (counsel, client, consultants, etc.) often learn more information as the EDRM progresses and must take this new information and apply it to the EDRM. This can often mean adding new custodians, changing search keywords or adjusting review criterion.
eDiscovery Dynamics
Now let's look at some of the dynamics during the eDiscovery process, in particular, the costs and the probabilities. Recall equation (5) from Part 1,
| PpJ - Cp ≤ S ≤ PdJ + Cd | (5) |
which demonstrates the S that will result in a settlement.
Keeping this in mind, and focusing on the costs for a minute, most estimates place the highest costs in the processing and review phases of the EDRM. Processing can make up anywhere from 15% to 40% of the total cost, and review can take 50% to 80% of the total. A recent RAND study2 asserts the following normalized percentages per phase cost based on empirical data:
- Collection: 8%
- Processing: 19%
- Review: 73%
- Collection: 4%
- Processing: 36%
- Review: 58%
- Production: 2%
The RAND cost curve shown below is similar, except missing the production piece.
These curves look like they could be approximated with the sigmoid or "S-curve" function.
The curve starts with a low cost and builds over time, with an acceleration during review, then decelerates as it approaches the total cost.
So let's apply this to the eDiscovery process and revisit our settlement functions. Going back to Part 1, and the example we used in which two parties were involved in a lawsuit where the settlement was $50,000, each of their costs were $10,000 and we let the probabilities vary.
Applying the s-curve eDiscovery cost function above and assuming both plaintiff and defendant total costs (they both approach $10,000), the probabilities are both 50% and cost functions are the same. The settlement curves, plotted by day as the eDiscovery process progresses and the costs go up, should look something like
Recall from Part 1 that a settlement is possible in the area below the defendant's curve (Sd) and above the plaintiff's curve(Sp).
But if the defendant's costs are more expensive than the plaintiff's (growing towards $20,000 instead of $10,000), then we see a wider gap between the curves, a steeper, faster climb of the defendant's settlement curve and a much better chance for a settlement, as shown below.
Making the total final costs even again at $10,000, if we look at the two extremes where the probabilities are both 10%, we get the graph below, where a settlement is always possible at under $15,000.
When both parties believe there is a 90% chance that the plaintiff will win, we get the curves below, which show the settlement curves shifted up and a settlement always possible under $55,000.
Finally, the case where there is vast disagreement about the probabilities is shown below. Here, a settlement cannot be made at any point, since at no point is the plaintiff's curve below the defendant's curve.
But it's probably not realistic to assume the probabilities stay the same throughout discovery. Disclosure of information should bring the probabilities together. Below shows the settlement curves as well as the probability curves, where the probabilities start apart and converge at 50% halfway through discovery. A logarithmic scale is used so both sets of curves can be shown on one graph.
We can see that in the beginning a settlement is not possible. But as the probabilities cross and reach the 50% mark, a settlement becomes possible.
Conclusion
Hopefully the dynamics shown here have shed some light on how parties are expected to behave during discovery, especially considering the costs and dynamics of eDiscovery. If one wants a settlement, discovery is potentially the best time to press for it, since that's when a lot of the costs and probabilities shift. The conclusion based on what we've seen here is that the best way to a settlement is:
(1) Make sure your opponent is spending a lot (at least more than you), and
(2) Use discovery to change the opposing side's probability by sharing information as soon as you get it to keep your own costs down.
Another dynamic which a lawyer friend of mine suggested was the theory that once a party is already in for a certain amount, they might as well stay in and see the matter through. This is similar to a game theory premise that poker players face: If you're already playing at a loss, what's a little more money to play out the game and see where it goes? I have not represented that dynamic here, but it may be an interesting one to consider.
Next time we'll look at some empirical studies to try to tackle eDiscovery cost estimation.
References
1. Palazzolo, J., "Why Hire a Lawyer? Computers Are Cheaper", http://online.wsj.com/article/SB10001424052702303379204577472633591769336.html
2. Pace, Nicholas M. , Laura Zakaras, "Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery", http://www.rand.org/content/dam/rand/pubs/monographs/2012/RAND_MG1208.pdf
3. Degnan, D., "Accounting for the Costs of Electronic Discovery", Minnesota Journal of Law, Science & Technology. 2011;12(1):151-190. http://mjlst.umn.edu/prod/groups/ahc/@pub/@ahc/@mjlst/documents/asset/ahc_asset_366139.pdf
Subscribe to:
Posts (Atom)