Prior even to [13] was a paper by Jack Hollingsworth,
published in 1960, describing activities dating back to September of
1959. This paper is noteworthy in a few ways:
Further, it is nice to see that the state-of-the-art has advanced some in the past 45 years: ``The grader cannot be completely automatic. Over-flow, invalid addresses, built-in stops and other effects can make the computer stop, and will usually cause it to stop again during tracing. Manual reentry is necessary.'' 3.2
We also find that the claim is made once again that security is a concern: ``Student programs can modify the grader itself.'' Also, it is worth noting that a strong set of constraints were imposed upon the programs in order to make automated grading feasible.
On the other hand, this paper as well as [13] demonstrate strikingly the elegance forced upon programmers during those earlier periods of computing: both papers provide full algorithmic or code depictions of the grading systems. [13] provides full code for their grader in ALGOL, and [17] provides a description of the 108 machine-language instruction framework that made up his grader. For comparison, Agar is composed of approximately nine thousand lines of Python code.
This grading system can be viewed as a very primitive version of the Unix shell snippet
if [ `./studentProg < test1.in | diff - test1.out` ] ; then echo Incorrect else echo Correct fi