Thursday, May 9, 2013

Technology-Assisted Essay Grading

The NY Times recently reported on the growing use of automated essay grading systems, what we in the legal & records space might call "Technology-Assisted Grading," or "TAG." This is yet another instance of the rest of the world utilizing predictive technologies in conjunction with statistical pattern-matching to create an ultimately subjective judgment of the content of a document. Even more interesting than the fact that MIT & Harvard are making this technology available for free via edX (my alumni donations at work??), is that the higher education community is having the same heated arguments that are occurring right now in the legal arena. Here is the best comment from the piece:

"Although automated grading systems for multiple-choice and true-false tests are now widespread, the use of artificial intelligence technology to grade essay answers has not yet received widespread endorsement by educators and has many critics.."

Sound familiar? It should. This is exactly the argument raging now by outside counsel (playing the part of professors in the article) attempting to hold onto their turf, once considered "untouchable" by technology. While it is true that computers can't "read" either student essays or litigation emails, they can be trained to recognize the salient elements that make the essay strong or the litigation email privileged. Those traits are easily describable as Rules. Either a document fits the Rules, or it doesn't. Nuances are accounted for with confidence scoring and sampling for accuracy (precision & recall). As long as there is sufficient auditing and exception handling, the work quality should be outstanding at a fraction of the time and expense of the purely manual method.

If recent advances have taught us anything, it is that nothing, and certainly no job function, is immutable. It doesn't matter whether the work task is rote (like tightening bolts), cerebral (like computation) or subjective (like analysis), it can all be done by the right algorithms, utilizing the proper training, feedback and statistical sampling.

Furthermore, when subjective work product is automated, society gains impartiality, consistency, speed and reduction of cost for the same services. That allows us to do more, work faster and create better results with fewer resources – the very definition of progress.

It is time to stop fighting the obvious, accept the reality, incorporate the efficiency gain and move on. I'm ready for my essay grade, please.