First, some background.
December and January are high season for letters of recommendations — students are applying to summer programs, graduate programs, etc.
Actually writing the letters is usually a pleasure. It is great to promote my wonderful students, and to share their strengths with others.
Almost all letters these days are submitted online, most via websites constructed on behalf of colleges and universities by third-party vendors. The quality and ease-of-use of these sites is highly variable, to say the least.
In addition to uploading my letter, these sites typically ask me to rate a number of features of the student. For example, I might be asked to rate the student on “Diligence/Persistence” or “Intellectual Promise” or “Interpersonal Skills”. For each category there is a drop-down menu, allowing me to select “Top 1%” or “Top 5%” etc. It is a curious thing that the scales (1%, 5%, etc.) vary highly by institution, but tend overall to be logarithmic scales.
Now, the rant: At the end of all this, there is sometimes a question along the lines of “How likely would you be to accept the applicant to a similar program at your institution?”
This question is, in most cases, sheer nonsense. We do not have graduate programs in mathematics or medical schools or anything of that sort at our small, four-year college. Furthermore, even if we did have, say, a medical school, why would I (as part of the math faculty) have any way to answer that question? What’s worse, such questions are frequently required — it is impossible to proceed without responding.
It seems that I am not the only one who finds many aspects of these websites difficult to negotiate. In a recent opinion piece (pdf) in Notices of the American Mathematical Society, the authors suggest that the mathematics community, at least, come up with a uniform approach, similar to the MathJobs system. I, for one, would be in favor. (I note that some REU programs already use the related MathPrograms system, which is great.)