All this talk about Princeton's reminded me about something I was thinking about in the days after the APA, and it's come up over at Feminist Philosophers too. Princeton doesn't do APA interviews. I don't really know why not, but I assume it's at least partly because they're aware of the social psych literature showing that interviews don't really add much to an organization's ability to pick good hires. And Princeton's not the only school that's giving up on the APA. If I'm reading the wiki right, Washington didn't do APA interviews this year, either. No doubt there are others I don't know about. Also, if we pull the lens back a little, it's worth noticing that in a lot of academic disciplines, they've just done away with conference interviews entirely.
As far as I can tell, Princeton and Washington are making the smart decision. Going to the APA is fuck-off expensive, for both departments and candidates. So what does everybody get for the money? A fifty minute dog and pony show so contrived that in most cases it's not really going to say much that's reliable about a candidate? The chance to get punked by your own subconscious cognitive biases into thinking the tall, good-looking guy is really more promising than the short guy with the speech impediment?
Now, I know APA interviews can tell you some important stuff. Like, maybe a candidate who looks awesome on paper turns out to be really terrible or a complete asshole or just not the fit you thought they'd be. I get that. But how often does that happen? And wouldn't that sort of screening happen just as well--or, really, actually a lot better--on a campus visit? So why not take all the money a department spends on the APA and sink it into bringing a couple more people out to campus for job-talks?
This really does seem like no-brainer to me. But since I've never been on the searh committee's side of all this, I'm curious to see if anyone wants to make the case for APA interviews. In particular, does anyone want to make not just the case that they can be useful, but that they're more useful than skipping the APA in favor of a couple more job-talks? Why do we do the APA?
Subscribe to:
Post Comments (Atom)
69 comments:
A guy I know who conducted interviews at the APA indicated that his dept.'s main motivation for interviewing at the APA was to discover which of the candidates were "weird" in a way that would interfere with doing the job--we're talking about philosophers, after all--before they spent money to fly them out. But he admitted that the value of this was dubious.
There was a conversation about this topic over at PEA Soup about a year ago. Link
I take it that there are three possibilities:
(1) Status quo: committee reviews 100-300 files, 2-5 committee members go to APA and interview 10-20 candidates (or do phone interviews), 2-4 finalists come for campus visits, decisions get made.
(2) Less noise: committee reviews 100-300 files, 3-6 finalists come for campus visits, decisions get made.
(3) Least noise: committee reviews 100-300 files, decisions get made (no interviews of any form).
I take it that the majority of depts use model (1), a significant minority are moving toward model (2), and some even use model (3).
I've been on SCs at both a liberal arts college and a research university, and I can see why LACs might feel they need the dog and pony show. LACs need to hear the candidates talk, and not just over the phone -- since they're looking for someone 'articulate' and 'clean' (dressing reasonably well, e.g., is part of the job, right alongside a pleasant speaking voice and non-spastic gesticulatory tendencies). So LACs need the APA.
Research depts don't, however. The only good reason I can think of for research depts to do APA interviews is the observation that if they didn't they'd all tend to go after the same ten candidates. Those ten candidates botch some of their APA interviews, thereby giving other candidates a shot, and giving less elite hiring department a somewhat greater shot at actually making a hire.
Anon. 2;29 --
Thanks for the disambiguation. I atek it UWash. is a (2), whereas Princeton (as I understand it) is a (3).
In fact, I'm really more interested in (2), since lots of places are going to want some sort of gut-check on weirdness (and I can't blame them--this is philosophy, after all), fit, or whatever else.
Having said that, Mr. Zero and Anon. 2:58 --
Yes, it's clear that lots of departments have (possibly) good reasons for wanting to look candidates over before they make offers. But what I'm after is reasons to do that look-over at the APA, rather than on a campus visit.
So, one consideration here: is it regularly the case that there's more than two or so candidates eliminated from a dept's top 6 by considerations of weirdness, lack of cleanliness, etc? NB that if not, we still don't have a reason to do APA interviews. Smaller places could do just as well skipping the APA and bringing a couple more people to campus.
So why do APA interviews? Because often way more than a couple of a dept's initial top (say) 6 are weird and stinky? Sure, if that's true. But is it? Or some other reason I'm not thinking of?
There are a few possible reasons why the APA is still done:
1) Many SC's are not familiar with the relevant psychological literature concerning interview effects and the heuristics and biases tradition;
2) They don't think that the psychological literature has anything important to say about how people reason in these situations; or, a variant on two:
3) They take it seriously and think they can filter out the noise through some magical, philosophical pixie dust.
In any case, there are no good reasons to keep conducting APA interviews in favor of going straight to fly-outs (or not interviewing at all as Princeton does [because of Harman's influence, I presume]. Noisy information is not good information, as PGS has pointed out so eloquently many times.
And, I presume, if we're not skipping fly-outs altogether, the information is much less noisy when the candidate feels a bit more secure and is not fretting about the possibly better interview he is to have later that day inside a fucking hotel room.
Besides, being stinky, badly dressed, soft-spoken, loud, or rude, shouldn't disqualify people from jobs outright, but should be something that we can talk as grown-ups about to try to rectify any of these deficiencies in potential colleagues. We don't need the APA to discover this. And besides information pertaining to these things gleaned from the APA may be just terrible information given the circumstances from which they are gleaned.
So, as PGS points out above, we need some better reasons for conducting interviews at the APA given the huge disincentives (noise, basing decisions on silly biases about weirdness or cleanliness) it carries with it.
My gut reaction is that there are no good reasons.
Why do we do the APA? It's the only time of the year some of our powerful and *beloved* (note sarcasm) colleagues get anything resembling adequate socialization. Gotta keep the feral philosopher population in check.
Yeah, PGS, I don't know. I guess when I think about it, the idea of hiring somebody sight unseen is psychologically repellant. But it's not like you find anything out just from interviewing the people. For every guy I knew was an asshole right away, there's another it took me years to discover was an asshole. And that's not to mention the non-assholes I was wrong about.
But it's not as though you get good information from fly-outs, either. Somebody--I forget who, or where I heard this, might have been here--put it this way: suppose you're doing a senior hire. You've got 20 years worth of research to look at. But the guy eats it on the flyout. It's a total disaster. How willing to disregard it will you be? Because it was almost certainly a fluke, right?
And Gilbert Harman has this to say on that PEA Soup thread: suppose the candidate blows your campus visit. You'd pass him over. But suppose you just heard he blew some other visit. You wouldn't give a shit. If that doesn't highlight the arbitrariness of it all, I don't know what will.
Also, a question: where do you fall in the taxonomy if you do phone interviews? If you were to do that, you could probably interview more people, and you wouldn't be distracted by how devastatingly handsome I am. Where does that rate on the noisiness scale?
(3) would be ideal if SCs were generally in a position such that they didn't know _any_ of the candidates ahead of time. But our profession is very small. I suspect that under strategy (3), there would be noise arising from the fact that the philosophers that one has interacted with will stand out as better than ones one has never met. In other words, in our profession, I suspect that employing (3) more widely would lead to search committees hiring their friends.
Are a lot of schools doing more campus visits this year than normal? Seems like a lot of schools are flying out 5 candidates total.
i think princeton uses option (3). arizona and davis use option (2)
Anon. 5:00 pm:
Hm, do search committees really have a lot of friends who are graduate students in other departments? Or were you thinking of senior searches?
I can think of only two cases in which I was on a search committee and knew one of the candidates. Strangely enough, in each case we would have loved to have hired the person, but a much higher-ranked department grabbed them up before we could even make an offer.
A few comments:
First, based on my limited search committee experience (once in the past few years), the APA was helpful in learning about candidates. (For context, I'm at a department with a poorly ranked PhD program.) To give just one example, going into the interviews, there was a rough consensus among the SC that candidate Z was ranked last of the ten we were interviewing. But Z was really impressive in his interview. We moved him up to #4. It wasn't a high enough rank to bring him to campus, since the dean would allow us to bring out only three. But if we had skipped the APA and brought out the on-paper top 5, we would not have brought him, and what we learned at the APA was that that would have been a mistake. He got a good job elsewhere and seems to be flourishing, suggesting that whatever biases are operative in the APA interview setting, they did not cause us to overestimate his quality. In my view, underdogs, particularly those from lower-ranked institutions or with less-famous letter-writers, like Z, have the most to gain from APA interviews. If that's right, APA interviews might help counteract the effects of pedigree prejudice.
Second, it is not always up to the department how many candidates can visit, so it is not a matter of merely deciding to skip the APA and start bringing 5 or 6 candidates to campus. There are institutional hurdles, like deans, that interfere with these changes.
Third, departments form search committees in part to save everyone else in the department time. Everyone has their own work to do. Of course even those not on the search committee are very interested in who will end up being their colleague, but it is nice to be able to entrust the beginning stages of that task to a select few (we take turns based on the job's AOS). Having job candidates visit is very time consuming, with interviews, meals, talks, and so on. Again, I do not mean to suggest that meeting with candidates is a chore--sometimes it is, but usually it is a delight. But even delights have opportunity costs. Having the SC screen candidates at the APA so we can only bring back three has the advantage of saving most of us time.
Fourth, since we are not a Leiterrific department, we often face the problem of figuring out whether some stellar (on-paper) candidates would actually be interested in joining us. We want the candidates we think would be the best for us, given our needs, but we don't want to waste time and money flying out someone who at best would use our offer to get a higher salary elsewhere. APA interviews allow us to identify the most obvious of the disinterested candidates and scratch them from our list. (I suspect Princeton doesn't have this problem, at least not to the degree many other institutions do.)
And last, APA interviews might be helpful sources of information for candidates. Don't forget that the APA is your chance to feel out a department. For some candidates, this doesn't make a difference. But for some it might. Some folks I know had 15, 20, even close to 30 APA interviews. 15 APA interviews is difficult to manage. It would be extraordinarily difficult for a candidate to do 15 campus visits. Being interviewed at the APA allows a candidate to later make somewhat better-informed decisions about whether to turn down a campus visit because they already have too many scheduled. I know I'm talking about a small number of lucky candidates here, but even they are justifiably worried about getting through the job-hunting season successfully. After all, there is no number of campus flyouts the having of which guarantees you a job, let alone a job you like.
I do think that there is something to be gained from talking to people. Especially for the smaller departments where fit is so important.
Am I alone in having interviews where it was obvious where we didnt get along, and others where we 'clicked'? both when I was on the market and as an interviewer.
I understand the reasons for not having the interviews at the APA.
Cost, difficulty, etc. I am surprised that more places dont do phone interviews. You can weed out some of those that dont fit and get a better group to come on campus.
But I think the idea of hiring someone just based on what's on paper bizzare, so bizzare that I am skeptical that it happens very often at all. I cant imagine that it happens at slacs, like the one where i teach, and I have hard time imagining that a search committee could ever justify to a dean "yes this candidate was the best on paper so we hired her"
Prof J: I've read advice to the effect that you want to be well known from conferences and presentations while you're a grad student (save the case where you are well known in virtue of infamy). It's impossible to meet everyone out there, but perhaps the person you schmoozed with at the last conference will be on your hiring committee.
I suspect this is especially true of candidates who are aiming for prominent jobs at research universities.
Of course this is unfortunate for the hygenically challenged.
1) Cheaper to do ten APA interviews, which are usually paid for on the candidate's or their department's own dime, than it would be to do ten flyouts, and departments have found the initial screening (deciding that the guy that looked good on paper isn't worth a flyout) is worth the hassle of the APA.
That would be my guess about the reasoning. Plus, if your top three all turn you down, you have something more concrete to go on when you try to flyout the next group.
I had two friends contacted by Washington. Washington said that they weren't doing interviews at the APA but that they would have a table at the smoker to talk with candidates. From what my friends told me, this was the interview. Fwiw.
Caveman --
Okay. Your story about your #4 choice guy looks like exactly the sort of reason for APA interviews I was wondering about. I wonder how regular this sort of thing is, or whether it's more of a fluke. In any case, too bad you couldn't the fly the guy out.
I'll have to take your word that APA interviews are useful for sussing out how seriously some superstar candidate is taking your search. I just can't wrap my head around the idea that some people can't get interested in a job--any job--at a stage when they don't know their preferred prospects are going to work out. But there are enough stories out there to make me think that's just my lack of imagination.
As for your last reason, well, let's just say that's not a problem I've had. . . .
Why not use video conferencing instead of APA or phone interviews? It's a lot cheaper and you get some of the advantages of seeing the person "in flesh" respond to the questions.
A little awkward maybe, but really, given that the APA involves 1) sitting at round tables smooshed together in the same ballroom where debutantes danced two weeks before and where now one can literally smell the anxiety and desperation, or 2) someone perched on a hotel bedroom or ignoring that one's in a bedroom at all (except when the phone rings announcing another candidate), well, uh...awkward is de rigueur.
I've heard some departments are going this way and the wiki seems to bear that out a bit (the lit wikis, too).
Caveman's example is a nice one; we can perhaps extend it to cases in which someone's work seems interesting but perhaps somewhat marginal (esp relative to the advertised AOS), but upon talking to the person, it's discovered that the person's work is even more interesting than suspected, and the person would be a great complement to the dept - in terms of faculty interaction and course offerings, and beyond. Some of this might not come out in an application b/c the applicant (esp when applying to 50-100 jobs) couldn't possibly know the specific (unadvertised) desires and interests of the department. So initial interviews seem very important, in some fashion or other. FWIW, although I'm quite certain I'm not Caveman's example (as I did not interview with GEICO), I did climb from the bottom of a group to 4th in somewhat similar circumstances to what I describe above.
But I agree with the suggestion of phone and (esp.?) videoconferencing. (All new Macs have cameras, I think. Please buy one - my stock is suffering.) It seems that it would provide most of the useful information that the APA interviews are meant to provide, such as what I suggest above. And even if such options provide their own noise and artificiality, they can't be any worse than - as someone else already pointed out - sitting in a crowded, noisy ballroom trying to focus on your own shit instead of the loud jackass with the infantile project at the next table while wondering how the hell he got an interview with that school but you didn't.
If anyone has suggestions for what would be left out of phone/videoconferencing (preferably the latter) interviews that could be gained (plausibly) from APA interviews, it would be very helpful to get a handle on *that*. I can't really think of anything (other than smell).
caveman's story about z's meteoric promotion from #10 to #4 provides a partial vindication of apa interviewing...
only if we think that this interview-based reassessment was magically immune from the psych literature, i.e. was somehow uninfluenced by z's height, color, clothing, etc.
look, of course we feel *better* about the process when we focus on an underdog whose chances are improved by it, rather than looking at the people whose chances were impaired.
but that is an illusion, too. if there are insuperable noise-inducing and information-distorting problems built into the process, then the process is no more fair or reality-based when it promotes people than when it demotes people.
there's no reason to think that z's advance (along with the consequent demotion, by the way, of the earlier #4, earlier #5, and so on) was anything other than further evidence of why we shouldn't use the apa: because it produces changes in the sc's beliefs that are vivid, extreme, and probably ungrounded in the candidate's merits.
and the fact that z was later hired by a good school does not provide evidence that z's promotion reflected z's real merit, either; it just suggests that the good impression he made on caveman's sc was not entirely anomalous. but if it was based partly on his height or his full head of luxuriant hair, then it was just as irrelevant when it influenced other schools as when it influenced caveman's.
(i mean, using the guy's later success to vindicate your reaction is like saying "and he wasn't just the beneficiary of some sort of irrational crowd behavior on our part--heck, there was a whole crowd of people who agreed with us!")
It's not the case that interviews don't help: the evidence shows that they lead to worse decisions. And to all the people who have said, based on their experience, that this is wrong, the point is that this not the kind of thing that we can learn from experience. That's simply testing our heuristics and biases by how well they line up with our heuristics and biases. Even in fields in which the answers are much more objective, like medicine, experts swear that experienced judgment outperforms more mechanical procedures. And they are usually wrong (see Bishop and Trout's book for an example of resistance of physicians to replacing clinical judgment with a simple checklist to see whether someone is having a heart atttack. Shown the figures, physicians say - my experience is that I will outperform the checklist. And they are always wrong).
Socially Awkward,
Okay, good point. Probably if we ever did a search in my area I would know one or two of the candidates, that seems right. (I can't find the 'bitter' tag in my html index.)
Lacanista,
The APA has a rule against interviewing someone in a bedroom. Did that really happen to you?
Some people here seem shocked and awed by the research conclusively proving that interviews of academics in the humanities are noisy.
Could someone please post a reference?
Here's a proposal, based on a remark made by Kris McDaniel in the PEA Soup thread I linked to earlier. I call it the "Dissertation Method"
Step 1: cut a hole in the box. Wait, that's something else.
Step 1: create a "short list" of 15-20 people as you normally would. These are the people you will "interview."
Step 2: Do not interview them. Instead, read their dissertations from beginning to end. This will give you a much better impression of the candidate's philosophical ability than a 45-minute interview possibly could, and also tells you much more than a job talk would.
Step 3: talk for a long time to the candidate's references. Try to get a feel for what kind of person she is. Granted, no reference is going to come out and intentionally reveal that the candidate is an a-hole, but if she really is an a-hole, I bet it will be hard for the references not to let something slip.
Step 4: taking into account teaching credentials, letters of rec., writing sample, the diss, and your conversations with the references, make an offer.
I think this procedure has several virtues. One, it seems to be less infected with vivid noise. Two, you'd probably end up spending less net time per candidate, even with reading the diss, so you could consider more candidates. Third, this procedure forces you to pay attention to what's really important. "Looking good on paper," while not especially important in, say, baseball, is extremely important in philosophy--because philosophy is actually done on paper.
It has a couple of drawbacks. One, it doesn't work as well for candidates who graduated a while ago. Of course, you could read a bunch of the candidate's more recent work. Second, you have to hire the person without meeting her first. But who cares? You don't get any reliable information from meeting her, and any information you do get is going to be outweighed by the vivid noise.
Mr. a priori,
I posted stuff about the research on interviews in a previous thread. Not sure which one or I would cut and paste. My cursory survey made me think that the research does NOT conclusively show interviews are more a source of noise than pertinent information. But I am no expert.
Here is a helpful survey article to start from:
"Beyond employment interview validity: A comprehensive narrative review of research and trends over time." by Richard A Posthuma; Frederick P Morgeson; Michael A Campion -- Personnel Psychology; Spring 2002
So there's a lot of "objective evidence is better than intuition/testimony/heuristics" sentiment around here.
Understandable. But let's not get ahead of ourselves and pretend that there's a robust body of evidence that interviewing doesn't help or that it hurts. There simply isn't. There's one or two somewhat inconclusive studies done (by empirical standards)that people have rallied around in a fairly political way. I'm not saying they're wrong, but let's not pretend the findings are so robust as to throw away all intuitions. Also, on the other side, there's plenty of empirical studies that suggest that instinct is often a far _more_ reliable guide than explicit reasoning in circumstances where a) there is much uncertainty and b) the evaluator is very experienced.
Also, let's not pretend that there's some agreed upon standard of what makes a candidate best. Sociability and other such factors that don't come through on paper are very important in some departments.
Finally, regarding the "dissertation method" mr. zero suggests:
1) what's so special about the dissertation? why not just ask for x pages of whatever the candidate thinks best represents her?
2) mandatory interviews with letter writers seems horribly noisy! much more so than interviews with candidates! the personality of the interviewee will come through quite strongly in what he/she chooses to say and how he/she chooses to say it. At least with letter writing, there is a period of consideration to make sure the letter fits the "norm". I know for my letters, one was caught by my adviser before going out. It was a very positive letter but raised some yellow flags that just didn't belong in the letter. Im not saying what was said was untrue, but the fact that it was in the letter made the letter seem very negative (by comparison to the expected unequivocal puffery). When this was pointed out to the writer, he was aghast and apologetic. He had meant to write a very positive letter - one that would get me into a good department.
I know I know there are studies that show that there is too much noise in face to face interviews.
I have never been on the interviewer side of the table. But as an interviewee, I feel like I really got a sense of what the people in the interview would be like as colleagues. And, in my better interviews when my nerves didn't get in the way, I feel like they got a sense of what I was like.
As an interviewee I would definitely want to have this experience before being offered a job. The video conference would probably work just as well.
...philosophy is actually done on paper.
Some of us think that philosophy is done in discussion. You can see why we might want to interview people.
"You don't get any reliable information from meeting her, and any information you do get is going to be outweighed by the vivid noise."
Only in Philosophy - check that, the academic Humanities - could this statement be rendered as anything other than patently ridiculous and/or absurd. Departments are searching not only for the best philosopher, but for the best _colleague_ and _teacher_ (esp. SLACs). That one could possibly imagine that you learn "nothing" from a face-to-face (whether that be APA or on-campus) could only come from a pie-in-the-sky, ivory-tower secluded academic.
Personality, it seems to me, means a great deal. And one's personality does tend to come out while meeting face-to-face. And that, believe it or not, is an incredibly important factor. Not only "Will this person do a good job?" but also "Can I and other persons at the school reasonably get along with this person?" And yes, I do believe that such things can be determined by shaking a persons hand and looking them in the eye...Perhaps that's just an old-school folkism to those of you up on "current research", but I think I'll take my chances...
Anon. 8:40
Amazing.
On the one hand, we've got some studies that show that the information people get at interviews is very noisy and degrades and dilutes the information that's actually useful. And on the other hand, we have from you: "Personality, it seems to me, means a great deal."
And, here's the incredible part: you think that the other side is typical of the academic humanities?
No, the worst, the most embarrassing tendency of academic humanists is to rely on their 'intuition' instead of on science to answer plainly empirical questions. Scientists don't do that. Well, they do it less, anyway.
The fact that you are so smug and condescending about it would be insulting if it weren't so patently idiotic.
Anon 7:41,
1) For a newly-minted Ph.D, the dissertation is going to be the best, most complete philosophical statement the candidate has produced. It's likely to have been the only philosophical project she's thought about in any level of seriousness since finishing her coursework. But if the Ph.D. isn't newly-minted, then whatever substantial amount of research she wants to send would do the trick just as well.
2) You're right, it's noisy. But the noise is less vivid, and, I can only imagine, less prone to be misleading in the way that a fouled-up interview or job-talk would be. You also won't be mislead by how tall and handsome he is, or by any latent biases you may unconsciously have.
Anon 8:37,
True, discussion is a big part of it. But you don't get tenure for being a good discussant. Also, that's why I suggested an in-depth conversation with the letter-writers: you can find out more about what kind of person you're dealing with by talking to someone who has actually known her for years than you can in any interview situation--which is just a glorified first-impression.
Anon 8:40,
Sure, personality goes a long way. But how much do you imagine you could possibly learn about a person in a 45-minute APA interview? Even if the guy is a total fucking dick, he probably knows enough to tone it down for the hour he's at your table and the day or two he's on campus. Do you really think you're getting reliable information in that big, crowded, noisy banquet room? I don't.
Bottom line: if you want a good researcher, go by the research, not just the one talk. If you want a good teacher, go by the teaching credentials, not just the one mock class. If you want a good discussant, go by the people who have been discussing philosophy with the person for years, not your one 45-minute conversation.
A query (not that I think it decides the question either way): do the academic psychologists who perform this research avoid interviews in their own job searches?
Anon 7:41 a.m. said:
"So there's a lot of "objective evidence is better than intuition/testimony/heuristics" sentiment around here."
And then admonished us to not get ahead of ourselves and discount intuition altogether because of a few studies that are inconclusive.
Fair enough (or not), but before I start taking the intuition/testimony/heuristics seriously, I'm going to need some studies that tell me those intuitions, testimony, or heuristics are reliable determiners of future job performance as regards teaching, research and being a good colleague. Or that these intuitions are better than, say, a simple rule that has us look at these things on paper (student evaluations, publications, letters, et cetera).
And we haven't been provided that by anyone. We're just holding onto our intuitions in these cases because they're our intuitions, damn it, and our intuitions are good (despite the evidence otherwise).
Bishop and Trout's Epistemology and the Psychology of Human Judgment covers very similar ground here. Especially interesting is the stuff on the doctors who think that despite the fact statistical prediction rules routinely outperform them in diagnosing patients, they still think that there's something to be said for the intuitions that they get from meeting the patient face to face.
Say what you will about B & T's philosophical conclusions, but I think they're right to say that if we're looking for reliable reasoning strategies that help us accomplish a certain task, say hiring people, and the evidence shows that intuitions gleaned from interviews are often noisy or that a certain rule is more reliable at choosing the best candidate, then it seems the reasoning strategy to go with would be the latter.
Is video-conferencing a feasible option as a replacement for the APA interviews? I know that the video-interviews themselves can be very productive, so I'm thinking more about technological feasibility. Do enough schools have the technology so that all candidates would be able to make use of it, even if at a campus that is not their own?
We do APA simply because it gives us a way to meet the candidates face to face. Plain and simple.
Over the years I have seen some excellent CV’s, yet when you meet people and they have no social skills, why bother bringing them to campus.
We aren’t just hiring a scholar, we hiring a teacher, a member of the community, perhaps a future Dean – if that person doesn’t fit – we are all saving ourselves a lot of trouble.
Several years ago I chaired a university search committee for the Director of the University Press – a big job.
One candidate looked stellar on paper – yet, when we flew that person in they managed to insult every member of the search committee over dinner the first night – the 3 day interview on campus went down hill from there.
Yes, you can be a brilliant scholar, but if you don’t have the social skills to engage your students, your colleagues, and the community – what good are you?
I assume that active PhD candidates should have access to videoconferencing services, since they're at major research universities. But please note that many good candidates are on one or two year visiting appointments, or TT, at schools that lack that kind of technology. It can be a huge expense and trouble to arrange a videoconference in such a situation -- I speak from experience.
By the way, the videoconferencing itself was, I thought, worse than a phone interview. Very awkward and unusual.
I am a job candidate, and I think the APA interviews are a great idea.
For one thing, you get them all done in a short period of time, which lessens the total stress; if we instead did video conferencing or phone interveiws, they might stretch out over weeks, which sounds horrible to me. (Fly-outs also stretch out over a matter of weeks, so this is only a consideration in favor of APAs rather than some other method of determining who gets a fly-out).
For another thing, it is a good data point for the candidates: I got a much clearer ranking of departments after having met the professors. Candidates want to end up at jobs where they will be happy, and I imagine that factors like the personality of their future colleagues do make a big difference to their happiness in the long term, and are possible to evaluate in a brief meeting. (And for those of you who hate the Leiter report, giving candidates additional salient info besides the Leiter rankings will make more candidates less likely to choose simply based on these rankings).
Finally, interviewing at the APA really did make me feel more like a professional and less like a graduate student, and I think it might help along that transition. (This might be purely antecdotal, but I think it's worth saying anyway -- there is a reason for rites of passage, after all)
As for the question of expense, the costs seem totally manageable if your search does result in a job, but not manageable for many people whose searches don't. Thus, I propose that departments should grant students "conditional loans" for the APA expenses -- the department gets paid back when the candidate gets a job. ;)
i find myself somewhere between the views expressed above.
i certainly don't agree with the science-be-damned view that says "enough of this empirical stuff--my intuitions tell me that my intuitions are just fine!"
on the other hand, i also am reluctant to extrapolate directly from some studies of md's diagnosing heart attacks, to the question before us about interviewing philosophers.
sure, the empirical work is relevant, and should be taken seriously. but that's different from thinking you can just read off the answer in the interviewing case from the answer in the heart-attack case.
i mean--we are debating over the sensitivity and specificity of a particular diagnostic, sc. apa interviewing. but the condition we are trying to diagnose is a little more complex than the presence or absence of a recent myocardial infarction.
in fact, i don't even think we agree on what condition we're trying to diagnose. good writer? good talker? good colleague?
clearly an accurate diagnostic for some of these conditions (and others) will not be an accurate diagnostic for others.
there also may be a mistaken assumption underlying all this, to the effect that there even *is* a stable condition to be detected, i.e. "being a good philosopher".
if you think there is such a fundamental fact--that god could look at each grad student and assign them a phi-number that would represent their goodness as a philosopher--then you will think that interviews and other methods of selection will succeed or fail to the extent that they accurately measure that phi-number.
but if the debates over intelligence have left you disillusioned about the existence of a unitary 'g' factor, then you might also doubt the existence of a unitary phi-factor that interviews either do or don't measure accurately.
sorry to sound platitudinous, but i think the debate so far has been a bit too dominated by the expression of extremes.
Somebody told me a story once about the well-known statistic that 90% of everybody thinks they're above-average drivers. But of course, the way averages work, it's impossible for 90% of people to be above-average at anything. So a huge amount of people who think they're above average at driving are wrong. Nothing new there.
The funny thing is, when somebody who thinks he's an above-average driver is told that everybody thinks that and most of them have to be wrong (i.e., you're probably wrong), the typical reaction is to think about it for a few minutes, try to compensate for the phenomenon, and then for him to come to the conclusion that he has compensated for it. As a result, he becomes more confident in his judgment, not less.
But that's stupid. The lesson is that people are very bad at introspecting whether they are above-average drivers, since everyone but the worst 10% thinks they're better than average. So unless you've got some more objective evidence that you're an excellent driver, you should be cautious about judgments like that. It's not the sort of thing you can compensate for just by working it out in your head.
I'm not saying that's what people here are doing, but... Ok, I am saying that. When you learn that interviewing is unreliable in a variety of ways--it's too little time to learn anything useful; everybody is on their best behavior; you're likely to think tall guys are better than shorties; you're likely to prefer men to women; people sometimes blow it in an interview and it doesn't mean anything; etc.--the correct response is, oh, geeze. I guess I should think more carefully about this. It is not, Oh yeah, well one time I conducted an interview and I learned something really interesting and true. Know what I mean?
One reason that I'm in favor of the interviews for philosophy jobs is that there really isn't a lot of "objective" evidence for them to interfere with.
Without an interview, your letters and the school you came out of play an even greater weight, and since these letters are deliberately written to 'hype' candidates, they aren't always a good source of evidence.
The only 'objective' piece of evidence in your file is your writing sample, and at most schools you apply to, there won't be anyone qualified to evaluate it. If they don't have anyone in the department doing M&E, then they will be able to tell how good a writer you are, but won't be able to tell whether the criticisms you make in your paper have been common knowledge since 1982. Princeton obviously won't have this problem when they do another M&E hire, most departments only have one person in each area, so basing everything on the writing sample doesn't seem wise.
mr. zero, this is an interesting point, but there's a couple things to consider beyond the question of whether the research about doctors and patient interviews is directly applicable to APA interviews. Assume for now that it is.
First, it is a good question whether the system you suggest -- talking to the letter writers over the phone -- doesn't simply increase the noise. Suppose you, as is likely, do not know the letter writers well. Now you not only need to make a judgment on the candidate, but on the reliability of the letter writers, whose intuitions are presumably as affected by bias as anyone else's, except that you have no way to evaluate that except by reputation.
If you do know the letter writers well, then we're headed towards an old boys' network.
Second, ideally this process isn't just about the college or university selecting a young scholar like so much horseflesh, but the young scholar getting to learn whether she would be a good fit at your school. And that requires an opportunity for the interviewee to see the school and meet the professors. We job seekers are by and large desperate, but we shouldn't design a system that presumes that.
Please help! I am working up my job talk. Should I 1) be able to talk through it extemporaneously; 2) Read it but make tons of eye-contact and make asides; 3) Give a hand-out? 4) Any combination there-in?
Thanks.
"Over the years I have seen some excellent CV’s, yet when you meet people and they have no social skills, why bother bringing them to campus."
But the point some are trying to make is that interviews do not and cannot reliably tell you whether or not people have social skills. Sure, you might be able to rule out a booger-eating moron once in a while. But the point seems to be that interviews don't actually give you the data you think you need. I think the Harmon (?) point is apt here. If you heard that some candidate that you like bombed an interview somewhere else, you'd say, "who cares?"
"One reason that I'm in favor of the interviews for philosophy jobs is that there really isn't a lot of "objective" evidence for them to interfere with."
But if, as some claim, interview data is more bad data, how does the interview help? How does more bad data make your choice easier?
In truth, I'm not sure how I feel about this interview-is-noise research. The point I'm really trying to push here is that if the research is robust, then claims like the ones above don't seem reasonable.
"But of course, the way averages work, it's impossible for 90% of people to be above-average at anything."
That's not true. Consider 10 students who write a quiz. One student gets a 0 (out of 10) and the other nine students get a perfect 10.
The class average is 9/10. 10 percent of the students get a grade below average, and 90 percent get a grade above average.
Anon 3:57,
I think you're right, about the old boy's network. We need to be getting further away from that, not closer.
dernbt,
Point taken. However, I'm still right.
On whether reading letters is just as noisy as doing an interview:
My impression is that it is not at all as noisy. First, it is true, of course, that letters are 'hyped'. But that doesn't make them noisy. It just means you have to know how to calibrate properly, and that's something that you can learn to do, especially if you've been reading letters for ten years. I find that recommendation letters are loaded with information, in fact.
Second, the letter-writers are not at all in the same position as interviewers. Sure, you can call the information 'subjective', but I don't see why that's a big problem. The writers have seen a lot of the candidate's work, have interacted with her for years, and are experts in the area. That puts them in a much better position to evaluate her than I am when I talk to her at an interview.
I'm surprised there's so much skepticism about recommendation letters. I wonder if that's because the skeptics are newly on the market themselves, so they haven't read letters? (Sorry, I don't mean to sound patronizing, I'm just curious about why my gut feeling about this is so different from many commenters'.)
In all of this discussion, unless I've missed something, I haven't read any substantial argument that would justify being in favor of an apa interview over a phone interview. The only thing might be the one comment that from a candidate's point of view it's nice to have them all compressed into a couple of days. That's debatable, but even given that, is it worth the money and hassle (for both committee and candidates)?
The noise would seem to be the same in both apa and phone interviews, whether you think there's much noise or not, unless you want to argue that some noise is more prominent in one context than in the other, and that matters here (we're more prone to be swayed by attractive voices, and they're more noticeable on the phone, or that looks are what matter, and we see them at the APA... But I think this would be hard to establish.)
So, can anyone defend the APA over the phone?
dernbt: You are incorrect. The relevant notion of "average" here is the median, not the mean. Means are appropriate for arithmetic averages. Medians are appropriate for statistical averages. They come apart precisely when a non-symmetric distribution is being used, as in your example. The average person in a bar doesn't become a millionaire when Bill Gates walks in the door.
A basic problem with this discussion is that the empirically-minded folks are simply not going to be able to convince the more intuition/gut-feeeling folks of much here, because intuitions are self-supporting. A convenient bias is that one remembers when one's intuitions were correct, and not so much when they are incorrect. See any stereotype ever for evidence.
Another problem is that, though there has been a good amount of evidence gathered for various predictive tasks (medical diagnosis, prisoner parole decisions, marital success, etc), it is true that there has not been a study in say, philosophy hiring practices. And there probably won't ever be such a study. Why? Because there isn't really a feasible means to set up an "experimental group" and a "control group" of new hires in enough departments to get a high-enough population size to do any meaningful statistics. It would also take at least through a normal tenure clock cycle, if not longer, in order to determine any long-run effects of either hiring policy.
So either we can trust the evidence of other areas in which they have similarly difficult predictions (and none rely on or presuppose the existence of a single unitary factor "g" for their predictions), where they have done proper studies with experimental groups and controls, or we can decide that we have magical philosophy dust that makes us special.
"1) be able to talk through it extemporaneously"
Yes..none of the others matter - fail to make eye contact - your not gonna make it.
Captivate them - like you would a class full of students.
Bore them - you're toast
If you don't know your subject well enought to give a talk without notes, you are not ready to interview.
...philosophy is actually done on paper"
But teaching philosphy is done in the classroom.
"the dissertation is going to be the best, most complete philosophical statement the candidate has produced"
Do you honestly think we are going to read it?
anon 7:35,
So? Do you think you get a lot of great information out of just the one mock class? Hint: you're not.
Anon 7:37,
No, but don't come crying to me when you hire the wrong guy. And by that I mean, not me.
"If you don't know your subject well enought to give a talk without notes, you are not ready to interview."
I call bullshit, and for two reasons.
First--and here I'm speaking as an audience member, since I've never had any other experience with job talks--talks are more engaging when they're done extemporaneously other things being equal. But other things aren't always equal. You might know the ideas inside and out, but still not have this particular, 50 min., framed-for-an-unknown-generalist audience version of the talk down cold.
In that case, for some people, they'll give a better talk if they read from a paper, if: (i) it's been re-written to sound "talky"; (ii) they know, eg, the first and last sentences of each paragraph well enough to be able to say them looking directly at the audience, making lots of eye contact; and (iii) they do lots of breaks away from the text between key paragraphs to extemp the transition, or they do extemp recaps of key ideas. In this same case, where you're working with a possibly fresh version of stuff you know really well, other people might do a better job just doing the whole thing extemporaneously.
I've had the misfortune of seeing job talks from rising superstar (ie, already in jobs at top-10 departments) assistant profs, when they extemped through power-point slides when they really should just been reading the damn paper. Too many "ums" and ""uhs" and repeating themselves over and over, not because they were going over key points, but because they had to keep reminding themselves what they were talking about. The presentation lacked and flow. And conversely, I've sen assistant and associate level profs from top-10 places give AWESOME talks, when they actually read about a total of about 2/3 of the talk. So that can be done very well.
So, just to repeat mysself: other things being euqal, sure, extemp the whole talk. But the most important thing is to be comfortable--and that means no stuttering, no hemming and hawing, and no pointless repetition. (Though, obviously, some repetition has a very important point!) If that means you (mostly) read a "talky" sounding paper, while still making tons of eye contact, I can't see anyone reasonable holding that against you.
I said I called bullshit for two reasons. Here's the second: Anyone who gives advice of the form, "If you can't do X, you're not ready to interview", that person's not giving you real advice. They're just being a prick and trying to psych you out. Lots of people do that subconsciously, because if they had even a modicum of self-awareness, they'd realize what pricks they are.
So don't get psyched out. Practice your talk tons. Get comfy with it whatever the best way for you to get comfy with it. And good luck.
Anonymous, you can't just stipulate that the relevant notion is statistical. I was responding only to a specific claim to the effect that it's impossible for 90 percent to be above average. Pointing out that there is an interpretation of the original case on which it doesn't make sense to say 90 percent could be above average isn't really relevant.
Suppose Bill Gates is in the bar and we're wondering whether it's possible that 90 percent of the bar's population has 'below average income'. If one stipulates that what one has in mind is something like 'below the income of a typical person', then maybe 90 percent couldn't be like that. But nothing is forcing that interpretation.
Thanks PGS. It is true that the last post did indeed psych me out.
The fact is is that my talk is fairly technical and I don't want to screw it up. I know that it may be too technical for a general audience or SG, but it is good stuff. I am definately going to give a substantial handout and recap many of the points throughout so to signal the major ideas and their development.
"If you don't know your subject well enought to give a talk without notes, you are not ready to interview."
This is the umpteenth time some dumbass has said something like this; it all started back in the fall, when various signs were taken as evidence that so-and-so wasn't ready for an interview.
Clearly the original poster who asked about how to present isn't ready for interviews. I mean, the person went to the APA and interviewed, and then got a flyout. Yeah, clearly not ready for interviews. Whatever.
I'm waiting for this one: "if you only have one job offer, you're not ready to interview."
Why do there have to be so many arrogant pricks in philosophy?
Suppose you thought you were above average at something. Then you found out that almost everybody thinks that, no matter what. What should you do? Should you consider the possibility that probably a good chunk of them are wrong and, in the face of no concrete, independent evidence as to which group you're in, adjust your belief accordingly--that is, reduce your credence; should you mentally compensate for the phenomena and increase your credence--again, in the face of no concrete evidence whatsoever; or should you be come up with a hypothetical model according to which everyone who thinks she's above average is, and conclude that there's no problem?
The point I was trying to make is this: there is a lot of evidence that our face-to-face judgments are unreliable in a lot of ways. We are tricked by how short or tall people are; we are tricked by charm or lack of charm; we favor white people over black people; we favor men over women; etc. And all of these things are unconscious--when we are affected, we don't know that we're being affected.
In the face of this, a lot of people have come on to say that they don't think they're being affected because it seems to them that they're not being affected. But that's no response--it's totally ridiculous. That's why people are calling the phenomenon vivid noise, rather than just "noise". The point is, it's not reliable, but it seems like it is. The fact that it seems reliable doesn't trump the independent empirical evidence that it is not reliable even though it seems like it is.
Now, there are questions about the applicability of the evidence to our specific situation, which are legitimate. I think that the evidence does apply, but there's at least a little room there for disagreement. But reacting to evidence that interviewing doesn't provide reliable information even though it seems like it would by blithely claiming that it seems reliable is just missing the point.
Anon 9:17, I am merely pointing out the correct usage of the term when dealing with populations of people. I am not arbitrarily stipulating something. The argument you use is precisely what is used by Republicans to claim that their tax breaks benefit the "average American" by, say, $2000. And this is claim is called dishonest by pretty much everyone else precisely because they are misusing means when they should employ medians. So is it logically possible that someone could employ the arithmetic mean to talk about a statistical distribution? Of course. Is it consistent with established best practices? Not at all. Means are never used in this context unless they are called the mean, and are accompanied by the median. Unless someone is trying to deceive the reader.
9:17
you are right about something like income, but wrong about driving skill. It is possible for 90% of people in the bar to be below average in income because income has an objective measure--we measure it in dollars. But driving skill has no such objective measure--hence there is no way to define "average driving skill" as anything other than the amount of skill that 50% of the population exceeds. if we create a driving test and give it a numerical score--and its a good test--we normalize the scoring so that the average is the median. All (reasonably) good tests, like the SAT, GRE, IQ, other psychological tests, etc, do this. If you dont normalize a test that is not based on objectively measurable quatities, you create the allusion that you are measuring something objective.
If you don't have tenure, you're not ready to interview.
I don't think that the Princeton method is necessariy a good one. The departments that bypass the APA (like Harvard, Princeton, Cornell) tend to rely a lot on the old-boys' (maybe old girls' now) network. This might be true as well if they conducted APA interviews, but if everyone went to their methods, I think you'd see a lot less open accessibility in the job market than there is now (and I know it's not great now). I speak as one who had interviews via that network at two of the above-mentioned schools, and I know precisely why and who put me into their pool.
I've participated in phone interviews a number of times for hires in another field where I sometimes serve on search committees. These are often very artificial and I feel even more alienating than face-to-face interviews. It's difficult to know who asked you something and whom you're addressing, you can't see facial cues that indicate if a person is still puzzled or impressed, people interrupt one another and then both stop awkwardly... It's hard. The only advantage would be the lower expense, but I believe from the candidates' point of view the experience must be at least as bad and also less informative.
If interviews are inherently unfair due to noise and the immediate prejudices and judgments that interviewers form, I don't see why video ones would be any better, again, apart from expense. Trust me, it is easier to get a group of faculty from my institution together to do interviews at an APA meeting than to get them together while they are here in our home city and I need them all to come to some one room on campus on a given day or set of days.
In favor of in-person interviews: it is surprising what you do learn about people's philosophical temperaments, for want of a better word, in such a setting. I'm talking a full hour-long interview. You can learn whether the person has any interest in fields beyond what they list on their resume, whether they can explain why their dissertation topic is important or worthwhile (a surprising number of people can't, and seem not to have thought about this), and whether they are very broadly trained in their field. We have learned in the past few years that certain departments write ridiculously hyperbolic letters about students who don't particularly shine or can't answer basic questions at our interviews. People at interviews who don't do well at least SEEM to me not to do well for reasons such as narrowness, rigidity, poor speaking ability, lack of relevant teaching experience or interest, etc. These do not seem to me just to be "noise" factors.
To Anonymous 4:08: I think it's fine to read a paper. There are several people in my department who are suspicious of Power-point presentations. Although I like to use them myself, I've seen job candidates do poorly with them (nothing worse than having someone read to you from their Power Point slides). Speaking extemporaneously might be impressive but may also convey that you are slick or glib. Many philosphers just like the traditional air of a workmanlike presentation. We do not expect you to be without notes. Handouts are always good, especially if you're dealing with anything complex and technical and/or speaking about historical figures and quoting texts. Eye contact is good and above all else avoid the droning factor. Zzzzzz....
conti at 5:59 asked:
So, can anyone defend the APA over the phone?
yeah, sure. give me a call, and i'll defend the apa to you, over the phone.
Female Department Chair,
In your 11:41 comment, were you trying to make Mr. Zero pull his hair out?
Again:
The whole point about vivid noise is that it doesn't seem like noise to those who experience it! The whole problem with the information gained from interviews is that it is, in fact, very low quality, but it *seems* like high quality, so it crowds out the actual useful information that search committees do have.
What keeps the information in the files from being "noisy"? The letters are provably unreliable. The writing samples reveal certain things but not others (such as breadth of background training, ability to move from the given topic on to other ones in the future, ability to step back to consider implications or general significance of the work, etc.). The student evaluations are always taken, everywhere, with big grains of salt. Teaching and research statements may sound good but be wildly off the mark of the candidate's real abilities, plans, and background. Will we ever know what is good or bad, philosophically, about a candidate, even once they become our colleague? I doubt it. That's why there is so much political nastiness in departments, disputed tenure cases, jealousy over raises, comparisons of quality of journals and subject areas people publish in, etc. I do indeed mean to make Mr. Zero and others tear their hair out if they believe anything can be non-noisy in any significant sense in our profession.
Female Department Chair,
Sure, the files are noisy. Who says they're not noisy? Of course they're noisy. Interviews don't just provide noise, though. They provide vivid noise, with is noise that doesn't seem like it's noise.
The problem with interviewing is that it's really noisy, feels like it's not noisy at all, and will cause you to overrule your earlier, less noise-influenced judgments.
I readily agree that the features you mention are not easy to discern just by reading the file. I have no idea why you think you can get the info from a 1-hour chat, though, especially when there's all this evidence that interviews are super shitty ways to find that stuff out.
Or, maybe you've got the magic pixie dust.
"magic pixie dust" is the lamest straw-man I've seen in a long time. Maybe we can give it a rest.
The argument in favor of interviews, it seems to, makes the claim that there is something inherent and essential about the practice of philosophy that is irreducible to, and indiscernible from, the paper elements of job searches. This something (or set of characteristics which many have already listed) can best (albiet not perfectly) be evaluated in interviews. Not being familiar with the 'growing body of psychological research,' I am skeptical that analogous cases are really analogous, and this is not just self-confirming biases -- this is being discerning and critical, even (per impossibile) of something so scientific as psychology.
Perhaps an analogy is apt: philosophy interviews (APA or on-campus) are like theatrical auditions -- only more dehumanizing. Surely no one expects directors and casting agents to hire actors just off of their resumes. -- why? Because acting is something that has to be demonstrated to be evaluated. Moreover, it is probably even beneficial that an actor's performance be affected by the part s/he reads for -- not every actor is appropriate to every role -- and this cannot be learned only by looking at what their prior roles have been. So, how a philosopher acts in an interview (as artificial and occasional as it is) might reveal things essential to being a philosopher that their paper trail cannot.
I'm not bitter,
Thanks for being, like, maybe the only person to have a serious question as to whether the empirical data is applicable.
I have concerns about the analogy between acting and doing & teaching philosophy. Since acting is a whole-body performance art, you really do have to see the person's whole body performing in order to make an accurate judgment. This is unfortunate, though, because seeing the person's whole body introduces a huge amount of static and vivid noise into the equation.
Philosophical Scholarship isn't like acting. Although discussion plays a large role, the proof is in the pudding, and the pudding takes the form of journal articles and books. A fabulous discussant who doesn't publish enough isn't going to make anybody happy and isn't going to get tenure. And then you're going to have to do a whole new search, looking for someone who can actually do the scholarship instead of just talk about it.
And how much can you really learn about somebody's propensity for discussion in an interview situation? 45 minutes in an incredibly tense and nerve-wracking situation in which her future career and everything she's worked for since sophomore year in college is on the line, she's in a strange city paying too much for a hotel room and wearing clothes she's not comfortable in, being judged. In addition to the latent biases everyone has towards tall people, good-looking people, well-dressed people, people who don't stutter, etc.
Teaching is different, as I've acknowledged. However, it's not that different. If you're casting an actor to play Hamlet, you need to see whether the guy can play Hamlet. But you're not casting a teacher to play such a specific role. You're not hiring her to be exactly the Jaime Escalante of your SLAC or whatever. You need somebody who's a good teacher, not somebody who's good in just this specific, Hamlet-like way.
Furthermore, suppose you've got lots of paper evidence that the candidate is a good teacher, but she blows it in the mock class. You'd be hard-pressed to disregard it, even though she probably just had a bad class--which everyone has from time to time, especially in extremely artificial, stressful situations--and probably is as good a teacher as your other evidence suggests. This is where the Harman point is especially helpful: if the candidate blows it in your interview, that's the end of it for you. But if you just heard that she blew it in some other interview, you don't give a shit. This shows that the entire enterprise is totally arbitrary. This interview matters, but that one doesn't. Why? Because you witnessed this one and not that one.
Post a Comment