file types, as contentious as custodians
and queries and as complex as sampling
methodologies and search algorithms.
Coincident with the growth of agreed
search is the fall off in attorney eyes-on
linear review of collections. Brute force
review is deemed too costly by those
who believe human review works and
too wasteful by those who know how
poorly human reviewers perform. Both
contingents expect technology to solve
the problems technology wrought.
Technology can almost certainly do
so, but not without re-examining cherished notions of legal counsel’s role.
American discovery law is founded
on the premise that lawyers supervise
and mediate efforts to identify information that will be produced. Implicit
in this are the elements of lawyer discretion and professional responsibility;
that is, courts accord substantial deference to the process and judgment (i.e.,
to attorney work product) presumed
to have been employed to identify relevant evidence and to produce or withhold same.
Objective culling, keyword search,
and emerging technologies such as predictive coding make clear that the idealized view of counsel as ultimate arbiter of relevance is mostly myth. Consequently, as more parties forge detailed
agreements establishing objective evidentiary identifiers such as dates,
sources, custodians, circulation, data
types, and lexical content, litigants and
courts grow impatient with the cost and
time required for attorney review and
reluctant to give it deference.
Broader use of mechanized search
and technology-assisted review brings
us closer to the day that counsel’s act of
countermanding a machine’s characterization of a document as “responsive”
will be viewed with suspicion. In the face
of multiple objective indicia of relevance,
producing counsel may be required to
defend subjective decisions not to produce on grounds of relevance — assessments once immune from scrutiny.
But ambiguity and unmet expectations aren’t just problems for requesting parties. Responding parties may
contend that their agreement to deploy
a requester’s keyword searches frees
them from a further obligation to review
and cull documents for responsiveness
(in much the same way as they currently
assume use of agreed-upon keyword
searches frees them from an obligation
to review documents not turned up by
keyword search). In turn, requesting
parties may object to productions rife
with noisy, irrelevant keyword hits as
obstructive “data dumps.”
So, is keyword search a filter prefa-
tory to review or a proxy for review?
Reasonable minds may differ, such that
the best course is to set expectations by
expressly setting out the respective roles
of search and review when agreements
to use key words are reached. Don’t sim-
ply recite, “The responding party will
search its collections for documents con-
taining the following keywords,” with-
out also stating whether the responding
party is permitted or obliged to review
the results of keyword search with an
eye toward their responsiveness to for-
mal requests for production.
Consider: “Responding Party shall
be obliged to interrogate reasonably
accessible, electronically searchable
sources using the agreed-upon search
terms and queries, review the search
results and produce only such non-privileged items as Responding Party deems
If you enter into an agreement with the
other side to use keywords and queries
for search, be clear about the disposition of items hit by queries. Assuming the
items aren’t privileged, are they deemed
responsive because they met the criteria used for search, or is the producing
party permitted or obliged to further cull
for responsiveness based on the operative Requests for Production? You may
think this is clear to your opponent, but
don’t count on it. Likewise, don’t assume
the court shares your interpretation of the
protocol. Just settling upon an agreed
list of queries may not assure a meeting
of the minds.
Practice Tip
responsive to outstanding Requests for
Production, subject to objections previously lodged to such Requests.”
Compare it to: “Responding Party
shall be obliged to produce all non-priv-
ileged items identified using the agreed-
upon search terms and queries and may
not withhold non-privileged items on
the grounds that they are not relevant
or not responsive to a request for pro-
duction. The use of agreed-upon search
terms and queries does not relieve the
Responding Party of its obligation to act
with reasonable diligence to identify
potentially responsive information by
other means.”
In the first formulation, keyword
search is simply a filter, limiting the
scope of review but not dispensing
with subjective assessment of respon-
siveness to requests for production.
In the second, keyword search serves
as an objective criterion for produc-
tion, uncoupled from any right or duty
to measure responsiveness against
requests for production. Neither formu-
lation is superior to the other, but each
implicates markedly different duties
and expectations.
Negotiated protocols are upending traditional notions respecting the
limits and duties of discovery. Producing counsel cannot expect opponents
to devote significant effort to negotiating objective search and culling criteria and then voice no objection when
documents with hits aren’t produced.
Likewise, producing counsel has little incentive to run an opponent’s proposed search unless doing so reins in
some significant element of cost, burden, or risk.
When parties negotiate agreements
largely without precedent in discovery jurisprudence, the old assumptions
no longer apply. Accordingly, counsel
negotiating objective culling and search
criteria must think through and spell
out the consequences of objective culling and search.
Austin-based Craig Ball (craig@ball.
net) serves as an ESI Special Master and
computer forensic examiner.
