In a forum hosted by another site, there was an extremely lively and interesting discussion about the etiquette of citation in the astronomical literature. The discussion is valuable enough for me to re-hash the highlights here where it can be archived, permalinked, and continued amongst a wider group. Below, I’ve included slightly edited bits of many people’s contributions to the discussion without attribution. (If you recognize your words and want your name attached, or want them removed, lemme know and I’ll take care of it.)
Please note that this post is not about the specific case that motivated the discussion, it’s about how we decide what to cite and what not to. Like Gus has been saying over on the arXiv post, it’s about culture. I would call it etiquette. (To be specific, I do not think this is an ethical issue.)
It all started when someone posted about their relevant work not being cited in the recent highly publicized paper. The poster raised the issue of giving credit where credit is due. Interestingly enough, the author of the highly publicized paper in question that lacked the citation replied,
Your work did not influence our observational program, our thinking, or interpretation—we already knew where to look and what we were looking for. We did not omit your paper in the reference list as deliberate act of malice, but it would not be appropriate to cite it as the genesis of our thinking.
Which prompted this from the un-cited original poster:
…The point here is that this is not about me…Its the demoralizing effect that such things have on younger people…they get very demoralized by stuff like this and I know great talented people who left the field due to this kind of climate. The point is that the its really simple and easy to just be more careful and generous…We all have such stories which is sad. If we ****all decide to make a collective effort to be more careful, diligent and decent then this may not be such a recurring phenomenon.
A further response from the un-cited sent to me via email in advance of publication of this post:
Just one point of clarification. I wasn’t claiming genesis of the idea but the relevance of the earlier paper to the actual result presented.
Back on the forum, the discussion continued:
In principle, the referee should catch blatant oversights in citations. However, with the abundance of literature even experts can have a hard time keeping up with what’s out there.
Whether these omissions are done blatantly or in ignorance of the literature is another matter of debate.
in the worst cases, I seen people actually redo work that I did, with no awareness that I had already done the same thing.
One technical solution that I suggested to ADS was for them to make a database tool that would take the references in a manuscript, look at both the citations and references for each member of that list, and identify cases in which there were large numbers of common hits on a paper not in the original reference list. I don’t know if this would help, but some approach like this could identify “should have cited” papers.
I’ve got to say, this discussion has me feeling nervous, because as an observer who isn’t able to keep up with the literature as much as I should these days, I’m getting terrified that I’m going to publish a paper without knowing what the important theoretical predictions are that I should cite…Is it really always a problem of *ethics* if an observer doesn’t cite all of the theoretical papers out there, and vice versa?
It isn’t always malice, but once a paper is out of the citation cycle it can be lost forever.
Some general rules which might avoid hard feelings.
1. For the author: the point of citing previous work is to place a paper in its intellectual context and to aid the reader in understanding the field. In particular, this means that it may be important to cite papers that have had very little impact on you and your research, provided that they have played a role in the scientific development of the topic for the community.
2. For the author: sometimes one comes across a paper which has been neglected by everyone, including you, but which contains relevant and valuable insights. It should be cited. Maybe it hasn’t played much of a role in the development of the field until now, but citations are also about recognition for good work.
3. For the neglected citee: the first step should be a polite(!!!) reminder to the authors’ that they seem to have missed an important reference. Bear in mind that we are legends in our own minds and that the importance of your paper might not be universally recognized.
4. If you are rebuffed, you can try writing to the editor of the offending journal, but you might try your argument out on a few colleagues first. The editor is only going to intervene if the case is absolutely clear and the referee is going to do a face palm and admit to having missed the point.
5. For the referee: the citations are part of the paper, and making sure that they are appropriate is part of the review process.
Post your papers to arXiv, and post them early. This means at the same time as you submit to the journal. Doing things this way gives you “free” refereeing, allows others to point out missed references, and can even manage to catch blatant errors before they make their way into a journal
This comment started a whole different discussion and is the subject of its own blog post: To Post or Not to Post.
It is necessary to separate willful neglect from honestly missing a reference. Moreover, as the number of papers published each year has exploded, it is in many cases impractical to cite every single possible reference related to a subject.
We also need to consider why this is such a hot topic? Is it just an academic issue, or is the job-related aspects the major (or sole) cause for concern? If simple quantities like citation rate or h-index were devalued in comparison to, e.g., external assessments of one’s work or some other more hollistic evaluation, would we be sending/receiving the “why didn’t you cite me” emails as often? (and might they be more cordial).
Those general rules sound good to me! And I love the idea of both a paper discovery service! I’m also intrigued by the comment about asking why this is such a sensitive topic. Let’s hear what you think in the comments below. How do you decide what to cite or not? Do we overvalue citations as a measure of success?
(On the original forum, the discussion towards the end migrated a bit to how we can change the peer review process. Please, again I ask that we hold off on that discussion until next week. Let’s keep this one about citation etiquette.)
HI!
Here just my two cents on this interesting topic. I’ve followed the discussion that prompted this closely. I am myself a young astronomer (first postdoc) in fairly hot field (galaxy formation and evolution). Therefore, my h-index is mostly set by the papers I get out and not so much by the number of citations (that is to say, typically citations come in faster than new papers come out). This is very much dependent on your field. If you work in field that have far less people working in it, you are more dependent on citations. By the way: I think judging scientists on an h-index is one of the worst things to do. Talk to them for half an hour, and you will have a much better view of how good they are.
I think it is almost impossible to not miss ‘relevant’ papers. What does ‘relevant’ mean? In some cases this is obvious (e.g. people use numbers, equations, etc. that you have (first) published). On the other hand, when people do work in the same general field as you are they _can_ cite you, to give an overview of what other related work has been done, but in many cases this would be a ridiculously long list that no one is ever going to look at anyway. That leads to another question about citations: what are they for? Are they done in order to increase h-and-related-indices of people who you think do good work? Or is other work referenced because it can help the reader put your paper in context, understand your results, or see where your assumptions and ingredients come from? The answer to this question is very relevant for a discussion about citation ethics in my opinion.
Cheers, Marcel
I would like to raise a point which I think is related. I find that authors publishing on American journals often use citations of papers published in the same journal or other American journals, choosing them over similarly relevant articles published on European journals (and vice-versa!!). Do others find the same? I admit it could be a biased view, I have actually never looked into it in a quantitative way.
I have definitely noticed the same thing. My suspicion is that this is because most astronomy fields are small enough that you are likely to know many of the people you should be citing… at least on your side of the ocean. So I think this is mostly the “honest mistake” variety – you know about the work of the people near you, and therefore cite it, but are less likely see talks or go to conferences where people from the other continent are describing their equally-related work. From an admittedly limited sample, I get the feeling this is less of a problem in fields like chemistry where the personal contacts are less important relative to the just “I read that paper” body of knowledge.
^Abt journal is an ADS search that I would use to get started on quantitative or comprehensive tests of these kinds of questions.
Here is an inter-journal cite analysis along the lines that Catarina was looking for except dealing in bulk numbers not pairwise replacements cites
the idea that there are similarly relevant articles in parallel journals is an interesting assertion or statement about citation into itself, no?
I’ll take a somewhat longer view since I’ve been around longer than many of you. Most of the time this sort of thing just doesn’t matter. LIke statistics in baseball your citations average out, sometimes you get cited more than necessary sometimes less so. Generally speaking, “don’t sweat the small stuff.”
That said, sometimes it does matter a great deal. I’ve had one case where a paper I wrote in the 90’s was a little ahead of its time, but fundamentally made a number of predictions that were proved correct much later. However another prediction paper came out a couple of years later and did not cite me for whatever reason (see footnote 1). That second paper was very thorough and readable and turned out to be the one that everyone cites today. Since my paper was not cited by that popular paper, everyone just assumes it was the first in the field and my contribution is indeed (as one of the comments above worries about) almost invisible today. The moral of this story as I see it is that _rarely_ not being cited matters a great deal but most of the time it doesn’t matter that much.
So how much should you worry about it? I try to focus only a few key papers where I think my contribution was important and in those cases I do write authors who do not cite me. The rest of the time I just send a silent curse 🙂
Mark
footnote: Generally its best not to attribute to malice what is actually just laziness. Of course sometimes it really is malice 🙂
Is there a reason that the identity and location of “the original forum” are not given anywhere in this post? It seems kind of ironic given that the topic here is proper citation etiquette! If our goal is to place things in their proper intellectual context, it’s too bad I can’t click through to go read the full original discussion that you’re referring to…
The reason was to not publicize the other forum too widely. I will email you 😉
I think there are two main reasons to feel miffed for not being cited when you think you should have:
1. Getting credit. This has been hashed out a bit already, but in general we like being recognized for good things we’ve done, rather than ignored.
2. Getting an idea out into the thought sphere. Sometimes a result never spreads for whatever reason (perhaps it was ahead of its time, or not written well, or contained within a paper that is predominantly about another topic), and there is an idea that you think should have an influence on the direction of a field but has been missed.
I think both are valid – we *should* get credit for what we’ve done, and relevant ideas *should* be known within the field. But with the explosion of papers, it’s absolutely inevitable that not every paper that perhaps should get cited will. As for what to do about it, I think pointing out (yes, politely!) the most important cases is all you can really do, and as a referee you try to do a fairly thorough job of scanning the literature.
Some of the features in the new ADS labs search are good for ferreting out “should have cited” papers. There are various “explore the field” rankings which choose papers by citation networks, and the results page can be viewed as a network (the “view as…” item on the results page).
While I like the intention of capturing “relevant” work in a list of citations, I agree that in many fields the literature is too expansive to realistically capture it all. In papers I read, I primarily use citations for as a historical record and traceback to previous work that had bearing or influence on the paper. I would love to see a “web of science” way of looking at papers and their citations. I believe this is available in other fields, but has anyone seen one for astronomy?
Here are a couple of other links to discussions of citations in other fields:
Discussion of citation issues in social sciences:
http://www.bmartin.cc/pubs/08interfaces.html
The basics: a general view of the purpose of citations and their proper use. This has some interesting points about citing software, which is relevant to astronomy:
http://tim.thorpeallen.net/Courses/Reference/Citations.html
The Scientist magazine (for life scientists) did an informal poll on just this topic. The most interesting quote was about using “citations as a vote”.
http://classic.the-scientist.com/blog/display/55801/
Article on self-citation bias in Pyschology:
http://www.ingentaconnect.com/content/acad/psyb/2011/00000051/00000002/art00002
Thanks EB… you answered my question about the web of science before I even posted it!
Thanks Kelle for bringing up this topic. In general when you do some work it is a good idea to do some exploratory work on what has been done earlier. This kind of approach should be hard-wiered in our postgraduate education. We give citation in order to prove that what we write is not just random occurred thought but is based on previous work on the subject/object, etc. It is a good idea to start the literature exploration from a relevant Annual Reviews paper and work the rest from there. Sorry for putting these basic concepts here but I am guessing that many graduate students do read posts in this page.
The question is how much exploratory work to do. Sometimes an Annual Reviews article helps, sometimes not. And then you have to make decisions about what to cite anyway.
In grad school I cited a paper I had just seen on astro-ph with a result that was relevant but hardly central to my paper (a single sentence that could have been removed without harming the paper). My advisor read my draft and pointed me to [paper from a few years ago claiming same result via a different method]. Fine. Then an older co-author of my paper read a draft, and pointed me to [paper from two decades ago claiming same result with vastly less evidence]. I cited them all, since it was only three, but…
What you cite is largely a function of what you’re familiar with. And – particularly for observation or numerical work – there’s a tension between citing more recent work that is more conclusive, and earlier work that is less conclusive. Awareness of the earlier work is important, but there’s no very easy way to ensure this awareness, and it’s only a (small?) part of the issue.
I like Mark’s comments. My general advice would be to “do good
research on interesting topics, write good papers, and advertise your
results” and let things take care of themselves. Sometimes you don’t
get cited when you should’ve been, and sometimes you get cited when
you might not have deserved it. It seems so petty even talking about
this when even 15 years ago this seemed to be a non-topic.
I “get” that people like recognition – we get paid OK (at least after
getting a permanent/semi-permanent job), but we’re not paid like rock
stars or evil Wall Street bankers. But we’re not doing science for the
pay – we do it because we love it, it is an exciting job (or should
be), and I like waking up in the morning knowing that I might see
something or figure something out that no one else has seen before or
figured out before. Feel blessed to be counted among the
scientists. But the citations make us feel like our work is
appreciated – so that we are not wasting our time. I think we all get
this.
It might be worth breaking this down into how influential a paper is
to the authors writing the paper versus whether it is cited or not:
[How influential was the cited/non-cited paper?]
Action Influential? Not Influential?
——————————————————————————–
Cited OK Padding/unnecessary citation
Accidently didn’t Cite oversight=>action?(1) OK
Purposely didn’t Cite plagiarism=>action?(2) OK
——————————————————————————–
The question is whether anything can and should be done about instance
(1) and (2), and whether it should be done privately or publicly.
Without talking to the author it is difficult to distinguish neglect
(accidentally not citing a relevant study because of oversight – we
don’t have an infinite number of work hours for becoming aware of
everything ever published) from malice (author is aware of a study, it
did influence their thinking whether for better or for worse, but the
paper wasn’t cited). So distinguishing benign oversight from outright
plagiarism is like distinguishing manslaughter from murder.
Emails from the uncited to the author in case (1) might be warranted,
but I tend to find these annoying. As the author, usually one gets an
email by someone that either wrote a paper on a topic only
tangentially related to the one you just wrote, or its a paper thats
on the topic that you are vaguely aware of, but the paper was simply
not relevant for shaping the direction of your research. I haven’t had
to email an author yet for case (2), but I have called it out when
refereeing a paper (instances where there was simply no way that the
author was unaware of the work that they had ignored – e.g. going so
far as to claim “discovery” for things that had been “discovered” many
years before in the refereed literature). Case (2) should probably be
brought to the attention of the journal’s editors – especially if
there is a pattern of plagiarism. But for the younger astronomers that
feel slighted anytime there paper wasn’t cited, and feel compelled to
email the author each time (case 1) – get a life. Life’s not fair. Do
more interesting work and get it noticed.
Do good research on interesting topics, write good papers, advertise
your results, and your career will take care of itself.
I don’t think I’ve ever gotten a “You didn’t cite me” email from a young astronomer. The small number of these emails I’ve gotten always come from more established people. Occasionally they are helpful or legitimate, pointing out something I genuinely overlooked, but often they have an ax to grind.
I’ve been tempted to send such emails in a few egregious cases but almost always metaphorically crumple them up before sending. I just don’t think it’s going to help.
The main problems are (1) there’s no dis-incentive against exaggerating your importance by being petty with the citations to previous work – most people don’t do that, but some do; and (2) everybody’s so busy, even the well-intentioned don’t read the literature anymore. This was also a factor in the posting-to-arXiv discussion.
I think complaining about omitted citations was an issue 15 years ago too, the difference was we didn’t have Internet forums to share the complaints in.
“Do good research on interesting topics, write good papers, advertise
your results, and your career will take care of itself.”
I generally agree with your post, but I think this is an extremely rose tinted view of astronomical careers.
The arguments in Kelle’s post from the secret forum boil down to whether citations are for Humans (jobs/careers) or for Science (the furthering of humanity’s knowledge). I disagree with the common notion, mentioned somewhat by Eric, that our (professional scientists as individuals) love for Science should be above all, implying that our human needs to be happy, fed, housed, etc should be secondary. (This notion is very common. I heard it in grad school a lot, in connection with being exploited in my job. I saw it somewhat in Astro2010, when the future of the field was discussed, including the career paths for young people…).
In fact, Mark Marley’s post illustrates the problem with this “service to Science should rise above my own Humantiy” notion very strongly: Mark’s paper was earlier; he was more original, he was the innovating Human. But the later paper was more thorough and readable, so it was a better paper for Science, as it did a better job of disseminating the idea to become part of humanity’s knowledge (sorry Mark — I don’t actually know what paper you’re talking about, I’m sure it was great!). For those people who think Science is served by our work and who cares about our own rewards, then what is Mark’s complaint? In this case, the purpose of citations is the dissemination of knowledge, not credit. For those people who think that our work is not only for Science but is also for our own health/security/well-being (i.e. Career), then Mark’s complaint is very, very valid. In this case, the purpose of citations is not just knowledge (Science) but also credit (Humans).
My own position is that astronomers are humans. We love advancing Science, but we also love feeling secure in our jobs and home lives. It is heartless for people who are secure in their positions to be dismissive of those who feel insecure in their futures.
I wonder how much of this is a function of (and influences the dynamics of) subfields? As Marcel noted all the way at the top, the citation dynamics for a “hot” field are very different from something more established where even a new thorough, seminal work only garners a slow, gradual increase in citations over a decade. And for those of us without job security, the latter case probably won’t cut it when some review committee of non-astronomers just count citations and say “this person’s h-index clearly doesn’t warrant tenure.” The end result is that younger people may gravitate to the hot fields only because everyone’s there and will cite them for anything they do, rather than because either the Science is good *or* their Human side is enjoying actually working there.
So that means that in these hot fields, the pressure is on to make sure you get cited a few times as soon as your paper is out to make sure you claim your spot in the citation network. Thus the feeling of being offended if your citation is left out becomes much stronger, because it can cut you out of a lucrative batch of citations (and thus keep you from getting that postdoc, or faculty job, or tenure). This also ties back to the arXiv discussion… these are fields where everybody posts at submission instead of acceptance.
To connect to KT’s dichotomy, this is the essentially Human aspect coming out more strongly in these fields – those of us who have not yet been granted the privilege to take the “long view” are often *forced* to give up some of the Science for the Human, because otherwise we don’t get to keep doing the Science.
As alluded to above, Helmut Abt has written many papers delving into this subject. I think these papers make interesting reading, and perhaps are worth studying before voicing strong opinions about what astronomers generally do and don’t do in referencing.