Saturday, October 20, 2007

If You Put Me to the Test, If You Let Me Try

Ask and you shall receive. Lots of useful responses to my post yesterday about why some departments don't allow some of their students to apply for the best jobs. There seem to be a couple of themes in what people are saying, so I want to try to address those.

First, I'd been assuming there'd be no downsides to sending applications to the best departments, but Robert's pushed back on that assumption. Maybe sending out less-than-outstanding applications can hurt candidates the way giving mediocre presentations or publishing mediocre work in mediocre journals can hurt people. Maybe, but how exactly? Especially if your department's not pushing your application very hard at that top school, it's going to get looked at for a few minutes by one, maybe two people. Maybe they'll read your writing sample, and maybe, to push this line a little harder, having the best people in the field see your underdeveloped work could hurt you. Again, maybe. But will it hurt so badly they wouldn't give your work another look after you'd got in good enough shape to appear in a decent journal? Which, remember, is what you'd have to do anyway to have a real shot at getting so much as a flyback from their department?

To take the opposite view, there are senior faculty in my department who say it can hurt to not apply to the best departments, even though we have no chance at getting those jobs. These profs' thinking is, it can hurt a candidate with middle-tier schools if it somehow gets around that the student's department didn't think they had a shot at the top-tier.

In any case, I need to think about the possible harms of applying a little more. As of yet, I'm not convinced.

Another idea people have is, somehow limiting the jobs students can apply for will help them get jobs. Anon. 7:43 is dead-on right about the importance of a very hands-on placement process in getting people jobs. But after people have been mentored throughout graduate school about producing good work and worked their way into the profession with presentations and maybe a publication or two, after they've had their CVs, writings samples, abstracts, and teaching portfolios vetted by the relevant senior and junior faculty, after even their letters have been vetted by trusted senior faculty, and after their profs have made calls to all the friends they honestly think should give their student an interview--if they've done all that hands-on stuff, what's the extra help of not letting students apply to Princeton?

John T. has a different take on how limiting the schools students can apply to can actually help them get jobs. One idea seems to be that departments can better influence another department's search process if they present a unified front with respect to a student's candidacy. But as Anon. 2:51 points out, besides offering honest assessments of different students' work, profs can tell their friends about the differences in students' interests and hope one or another candidate will appeal to the search committee as a possible fit for what they're looking for.

But John T. and others also have something else in mind, something about the psychology of search committees. John talks about a department wanting to be taken seriously when they tell a top-tier department, "This is the best student we've had in a decade." Fair enough, they should want to be taken seriously. But it seems like the best way to do that would be not to say a student's the best in a decade unless they're actually the best in a decade. Problem solved.

Along similar lines, Anon. 9:25 asks why anyone should expect their letter writers to say they're qualified for a job at a top-tier school. The answer is, of course, no one reasonable should expect that. But there's no problem here unless there are profs who tailor-up dishonestly inflated letters specifically for top-tier schools. If a prof writes a letter that just gives a useful description and honest assessment of a student's work, where's the problem?

I feel like I must be missing the thrust of what John and others were talking about with this business about being sensative to the psychology of the search committee. I have to admit, I'm somewhat suspicious it has something to do with a willingness dishonestly to inflate assessments of a student for some audience but not for others. If it's not that, I'm all ears as to what it really is.


Robert said...

I don't think I have made it clear enough that I don't endorse the practice in question. I just wanted to (1) caution students about the inference from (advisor thinks I shouldn't apply to best jobs) to (advisor thinks I'm garbage or too pathetic to introduce me to important folks), and (2) give a possible factor in why one might worry about applying to top jobs.

I think the practice itself would be prone to all sorts of serious worries about entrenched views of students and fairness, not to mention gender and race dynamics.

I am also skeptical about the justification I gave about publishing, presenting, and applying with mediocre work. Since I don't think I discount a philosopher's work because I have read some previous mediocre paper or seen some mediocre presentation. But, and this is what worries me, what if lots of other people DO? If they do, I would be happy to know this.

What if some conscientious young philosopher reads most of the writing samples for all the candidates from at least decent schools? How many times have I applied to open positions at Pitt, Princeton, or Cornell for several years in a row?

It could be that this philosopher has read more or less the same paper of mine a couple of times. I just don't know what would happen if she sees me in the elevator at the APA. She could cringe or she could think--"Oh, you wrote that paper, I thought it was sort of good!"

I'm not sure. As I noted in my original post, I tend to apply to all jobs, but I guess that might have been a mistake.

Jon Cogburn said...

Yeah, I'm on PGOAT's side of this one. I've been on the other side of some hires now (admittedly not at a top tier institution) and nobody's ever been irritated when more than one student from the same school applies.

Likewise, many of us (albeit nowhere near a majority) would rather hire somebody from a worse institutional affiliation but with better promise of publications, teaching, and taking part in the institutional life than somebody from a better institution wanting in any of these. I have to believe that some of the faculty at top schools feel the same way.

Likewise, there is so much uncertainty here that your own department really doesn't know if you are the best or worst in your class. This, plus well intentioned faculty member's attempt to avoid damnation by faint praise (and to place their students), reasonable people don't take letters of recommendation that seriously beyond checking them for red flags (unfortunately some letter writers put these in unintentionally and really hurt job candidates; a couple of times now I've contacted placement officers begging them to either yank somebody's letter or get the clueless prof to change it).

I think a lot of philosophy professors egotistically think too much about how they are viewed in the field. Preventing students from applying is I think almost totally a function of them being more concerned about their reputations than the good of the students. This is really goofy though. Even the most famous among us will almost certainly not be read twenty years from now (how many people are still writing dissertations about Nelson Goodman or Suzanne Langer, excellent and both world famous philosophers both in their days? Can Fodor really be that far behind?).

My point is, we should all be humbled by Lady Philosophy, so why be a tool about this stuff?

Anonymous said...

There are two practices that overlap. One is not letting some students apply for top-tier jobs. The other is not letting too many students from one's department apply for the same job. One can imagine these practices coming apart (say, if one had only one student on the market, one might nonetheless tell that student that they couldn't apply for top-tier jobs).

The justification at my institution for not letting students apply to top-tier jobs is that recommenders are free write "the strongest letters possible" -- and not have their colleagues at top-tier institutions think ill of them for writing inflated letters. The solution, as PGS pointed out, is for recommenders to stop writing inflated letters.

The justification at my institution for not letting too many students apply for the same position is that apparently some departments won't be able to decide between two or more candidates from the same department and will decide to interview none of them. It's an empirical question whether departments do this. I don't think it's happened on any search that I've been on.

So I'm inclined to be against both practices. We're revisiting those practices in my department this year. We'll see what happens.

-- A Guy on Hiring Committees

Anonymous said...

Let us know how that goes, Guy on Hiring Committees. Maybe this goes without saying, but I'd love to know how current hiring committees react to these applications (i.e., applicants from lower ranked schools applying to highly ranked ones -- or even just higher ranked ones, and multiple applicants from the same school). I take it most people reading this blog are currently on the market (or just enjoy reading about the pain of those of us who are), but if there are readers who are actually on the committees we're trying to impress, it'd be interesting to hear what they're thinking this time of year.

Anonymous said...

What Guy on Hiring Committee says about his department's rationale for discouraging individual students from applying to top-tier jobs is what my cohort was told when we went on the market: you want your recommender to go 'all out' for you, not building explicit qualifications into the generally positive spin. I didn't really get it at the time, but now that I'm on the other sides of the recommendation process and of the interviewing table, I do get it. If I know my recommendation is going to top-tier research departments, I know I'm going to have to work harder -- that is, be more explicit about possible reservations -- to gain credibility than if the I knew the letter wasn't going to those schools. And I can attest that when I sit on search committees those qualifications can be hard to ignore -- even when I know they weren't written for me (in my second-tier research department). So it really can be in an applicant's interest to reach an understanding with his or her recommenders that the letter won't be going to, say, the top twenty research departments.

It's too simple to say this is merely a question of honesty. I think it would be dishonest to pitch a student substantially differently for different-tier schools. There should only be one letter, recommending the student more or less univocally. (I say 'more or less' because I can imagine a student's teaching qualifications being presented diffferently as between, say, a research job and a liberal arts college job, though my own graduate department never did that.) So if the student is applying to NYU, among all the other schools, the recommender's deliberative question is 'How strongly can I recommend this student to NYU?' And in many cases, the answer is clear: not at all strongly. If the student isn't applying to NYU or any other top-tier department, the deliberative question is different.

Nor does going 'all out' for such a student entail inflating the recommendation. One simply wants to present a positive case, leaving reservations mostly implicit. When a reservation is made explicit then the search committee has to discuss it, and as an applicant you don't want your recommenders influencing committees in that way!

(I do concede that this sort of thing can be very tricky to negotiate with students. When I was on the market we all tremendously resented being 'ranked' in this way: even when one came out 'on top' it wrongly introduced the idea that we were competing with each other and that faculty were in the business of preventing some students from getting jobs. In fact, it can be such a morale-destroyer that I can sympathize with the thought that the practice should be discouraged on those grounds, despite the abovementioned benefits.)

One further thought, on the diffferent question of students from the same department competing with each other. The issue here, I take it, is that despite the placement director's best efforts you'll get something like this result: three equally good students apply to the same fifty jobs, and A gets twenty interviews, B gets five, and C gets none. (Why does this happen? Because it often happens that search committees negotiate an interview short list by negotiating which one of recommender R's or of department D's three students to interview. Yes, it's a stupid thought that we should interview at most one of R's or D's students, but we're exhausted with the hundreds of dossiers read on top of grading and recommending our own students, while perhaps also writing our upcoming apa talk or comments, and we're not getting along with each other, and this is an easy tactic for getting the short list whittled down at the end of a four-hour meeeting so we can go home. And twenty-five departments do this with R's or D's equally excellent students, and something about how R or D has phrased things, which the placement director didn't catch when comparing the letters, makes A 'win' twenty of these competitions, B five, and C none.) So A gets so many interviews that s/he blows some from sheer repetition, while C might as well not go to the convention. Alas, I've seen it happen. Better that these three students not apply to all the same jobs.

Jon Cogburn said...

I think anonymous gives the best reason. For example, you want to be able to say with conviction that the person will be tenured in the department in question. However, a few points are in order.

(1) Anonymous' point is predicated on the practice of only writing one letter for each candidate. If it's such a big issue why not just change the letters a little bit depending upon the kind of place in which the person is applying? Is that unreasonable? Am I wrong for thinking that a department shouldn't have a Ph.D. program unless they are willing to do extra stuff like this to place their graduates?

(2) Again, my perspective is from hiring into a non-first tier program. But a couple of good programs now schools allow simultaneous application, and it's never even remotely been an issue in our hires. One person we hired was from one of these schools and competing against some of his own people. Given the market, had his school had the stupid requirement, he might be unemployed now!

Sorry for my unsympathetic postulations above about the psychological reasons people defend this practice. In this case I do think my usual uncharitable truculence is correct- the restrictions are a result of a combination of laziness and the letter writers getting in a weird tizzy about faculty members at top tier schools actually reading something they wrote. But to focus on such genetic issues is certainly jejune in this context.

Anonymous said...

I'm the 'anonymous' from last night, checking back with a clarification and a brief elaboration, and now that I see Jon Cogburn's reply one reply to that.

The clarification is this. Am I saying that students from the same department compete specifically with each other for interviews? Well, not in a normative sense: no one would say they should. But a tired or lazy search committee may wind up ensuring that they do -- indefensibly, but understandably and foreseeably.

The elaboration is that it's perhaps more likely for C to 'lose' to A and B together, in my example. That is, the search committee reasons (unreasonably) 'We can interview at most two of R's or D's students, and A and B seem clearly better than C' -- where, again, this appearance is an unintended artifact of the letters that the placement director should have caught before the letters were sent.

Jon Cogburn assumes that not sending multiple versions of a recommendation is a product of departmental laziness, but I presented it as an issue of honesty, as I believe it is. How would you feel if as a member of a search committee you knew you were getting the version written for 'lower-tier' departments and that 'higher-tier' departments got a substantially different letter? I'd feel lied to. (Though for all I know some departments do this.)

(By the way, Jon, I apologize for my anonymity. I know it's not exactly a conversation when only one party is recognizable.)

Anonymous said...

May I say, first of all, that this is all incredibly fucked up? That off my chest, I do understand the arguments presented by folks on search committees: it sounds like you'd admit that the situation is to some extent irredeemably fucked up, so I don't mean to offend anyone by starting my post this way.

Now, first a question, then a comment. I've written letters for undergrads applying to grad school, and also institutional letters for students applying to med school. In the latter case, there is a standard format that all schools agree on: you start and finish with code that ranks the student against other students from that school, in current and previous years. So maybe "we recommend with strongest reservations" means top 5%, "we recommend with strong reservations" means top 20%, and "we recommend to you" means, well, he's our student and he's paid us a shitload of money, so how can we not recommend him. Something like that. So my question: is it true that that letters for job candidates work similarly (for all the detail and substance they might contain, ultimately there's got to be a sentence in code that's all many departments care about, unless the letter also contains a caveat that you buggered the bursar, in which case the candidate is just fucked)?

Second, a comment. I'd like to think that schools wouldn't take such rankings so seriously. Imagine a letter of the following form: "Candidate X works in epistemology, and has a highly original research program; so much so that his dissertation is very rough. I think he'll do fine research as (and if) he continues along these lines, and might be the next big thing. But he recognizes that he's got a long way to go, simply because the project he's decided to take on is so tough. I recommended as a matter of course that he pick an easier topic, but I've got to say he's done far better than I thought any of our graduate students could, at least thus far. Now, he knows his lack of publications and rough writing sample from his diss. may work against his candidacy at many schools, so he's spent a lot of time on his teaching, and he's as a result a really good teacher. Therefore, I think he'd be a great prof at a teaching school, and I heartily recommend him for such positions, without reservation. He'd also be competitive, in my opinion, with the best of the best at research universities, where he could spend very little time on his teaching and still be competent. But recognize with him that there's a 60% chance he'll work out the kinks in his research and fly through tenure, being one of the best tenure candidates even if he's at a top-10 school, and a 40% chance that he'll have a tough time at a top school come tenure time because he hasn't had the time to work through everything."

A single, detailed letter, in other words, addressed to multiple audiences, that would give search committees useful information but not a simplistic ranking. Would the candidate get fucked at your school because alternative candidate Y got a letter that effectively said "hire Y, unequivocably" (and had sufficiently traditional form)?

(By the way, I'm not describing a letter I think I could ever get, but one that addresses the discussion as it's evolved thus far.)

Jon Cogburn said...

Last Night Anonymous- I don't take any humbrage at the anonymity! And my penchant for interpreting everything in the most Hobbesian possible manner is in constant need of correction. So I don't take humbrage at that either (this being said: (1) Hobbesian interpretative bias clearly does have some purchase in understanding the philosophy job market, and (2) given said market, and the way it is set up by the departments and the APA, those of us with jobs have a moral obligation to feel like schmucks.)

I hate to complicate things, but I don't think other Anonymous' idea of qualifying the letters would work. The problem is that letters are so inflated that any qualification almost always gets read by someone as a red light or poison pill, which can get your file thrown out in two seconds.

I don't think that writing separate letters has to be dishonest. If the letter contained appropriate use of the indexical "your institution" nobody would get steamed.

Anonymous said...

As it turns out, the practices have come apart in my department this year: we have a small number of candidates on the market, and none of them is allowed to apply to top-tier departments. So clearly the "we don't want our students competing against each other" rationale is not operative.

I was thinking about the "the problem with the two-letter alternative is that it's dishonest" line of thought. I'm not at a top 20 department. I might not like it if I read an unqualified letter from someone who wrote a more qualified letter for better departments. But I'm not sure I'd like it any better if I read an unqualified letter from someone who could write such a letter only because they wrote NO letter for better departments. I'm not sure what the general principle is -- perhaps if it's dishonest to say P to A and Q to B, and it's not dishonest to say Q to B, then it's dishonest to say P to A and nothing to B -- but I think some criticisms of the "different letters for different schools" approach carry over to "no letters for some schools" approach.

-- That Guy on Hiring Committees Again