Paper
Writing and Research Strategies
[Copied from the net in 2012. Source unknown]
Introduction
Here is a brief reminder of how to write papers and what kinds of
papers to write. Please also see the other materials on our web site on paper
structure.
papers to write. Please also see the other materials on our web site on paper
structure.
What matters most about a paper is what the reader gets out of it,
not how much work you put into it. That is, after reading your paper, the
reader should feel that he knows something new, has a better insight into
something, or has a better idea of how to do something.
not how much work you put into it. That is, after reading your paper, the
reader should feel that he knows something new, has a better insight into
something, or has a better idea of how to do something.
Each paper you write should stick to one main idea, and that idea
should be clear enough that you can formulate it in a single sentence. If you
can’t tell someone else what the main contribution of your paper is in a single
sentence, you haven’t worked it out enough.
should be clear enough that you can formulate it in a single sentence. If you
can’t tell someone else what the main contribution of your paper is in a single
sentence, you haven’t worked it out enough.
Writing papers doesn’t need to be all that hard because papers fall
into a small number of common categories. Most of them involve some problem
that needs to be solved or some observation that needs to be explained; for the
paper to be accepted, it’s important that this problem or observation is
interesting in the first place.
into a small number of common categories. Most of them involve some problem
that needs to be solved or some observation that needs to be explained; for the
paper to be accepted, it’s important that this problem or observation is
interesting in the first place.
Solving problems that aren’t interesting to reviewers won’t get you
published. It is therefore important that you keep track of the literature and
figure out what topics are likely going to be of interest to reviewers.
published. It is therefore important that you keep track of the literature and
figure out what topics are likely going to be of interest to reviewers.
In writing your paper, keep in mind that many reviewers are
graduate students like yourself. Think about how you approached your last few
paper reviews. How well did you yourself know the literature? Did you give the
authors the benefit of the doubt when you didn’t understand something? Did you
ask for more experiments?
graduate students like yourself. Think about how you approached your last few
paper reviews. How well did you yourself know the literature? Did you give the
authors the benefit of the doubt when you didn’t understand something? Did you
ask for more experiments?
Paper
Types
Now let’s look at common different paper types. I have given these
papers short, catchy names because you should be able to identify clearly what
kind of paper you are writing.
papers short, catchy names because you should be able to identify clearly what
kind of paper you are writing.
“I
reviewed the literature about solutions to problem X.” (Literature Review)
A literature review should pick a well-defined research area,
something that is neither too large nor too small. An area might be something
in which there are 100-200 research papers total that you need to read.
something that is neither too large nor too small. An area might be something
in which there are 100-200 research papers total that you need to read.
A literature review is a little bit like an experimental paper. It
usually starts with a question, then looks at data, and then draws its
conclusions. The question it asks is usually of the form “what are the unsolved
problems” or “where are there areas for improvement”. In the conclusions, you
should answer these questions. They form the basis for further research.
usually starts with a question, then looks at data, and then draws its
conclusions. The question it asks is usually of the form “what are the unsolved
problems” or “where are there areas for improvement”. In the conclusions, you
should answer these questions. They form the basis for further research.
In fact, when you start on your thesis, you need to do a good
literature review anyway, and you need to incorporate it into your thesis. If
you can get this kind of review published, that’s really good. However,
publishable review papers are a lot of work and usually require upwards of a
hundred original references to be organized and analyzed. Nevertheless, you
should at least aim for a group seminar on your chosen topic.
literature review anyway, and you need to incorporate it into your thesis. If
you can get this kind of review published, that’s really good. However,
publishable review papers are a lot of work and usually require upwards of a
hundred original references to be organized and analyzed. Nevertheless, you
should at least aim for a group seminar on your chosen topic.
“I
looked at the best ways of solving problem X, benchmarked, and compared
them.” (Benchmark Paper)
A benchmarking paper asks the question: “given these standard
methods, how do they perform under different conditions?” It isn’t necessarily
for finding “the best” of the methods (usually, there is no uniformly best
method), but for identifying tradeoffs between different methods.
methods, how do they perform under different conditions?” It isn’t necessarily
for finding “the best” of the methods (usually, there is no uniformly best
method), but for identifying tradeoffs between different methods.
A significant component of a benchmarking paper is the datasets. If there are
standard datasets and standard algorithms, and they simply haven’t been
benchmarked against each other, then it’s easy to do that. Otherwise, you may
have to create your own benchmark dataset. This can be a lot of work, but it
can also be rewarding: if your dataset is novel and interesting, other people
will use it and cite your benchmarks.
This is often a good paper to write at the start of your thesis
work: you gain experience with the problem you are trying to solve, you
implement algorithms that you need to implement as control experiments later,
and you collect and/or create databases you can evaluate on. Papers like this
have the potential of getting highly cited. They are also conceptually fairly
simple and you are almost guaranteed to get a useful result. However, they are
a lot of work to do well.
work: you gain experience with the problem you are trying to solve, you
implement algorithms that you need to implement as control experiments later,
and you collect and/or create databases you can evaluate on. Papers like this
have the potential of getting highly cited. They are also conceptually fairly
simple and you are almost guaranteed to get a useful result. However, they are
a lot of work to do well.
“I
came up with a new/better/interesting way for solving problem X.” (Method
Paper)
This is perhaps the most common paper people write; it’s usually
the kind of paper that takes an engineering view of the world. You usually need
to show experimentally that your method is better than existing methods, at
least under some circumstances or on some kinds of data (rarely, a complexity
analysis may be sufficient). Here, “better” may mean one of several
things:
the kind of paper that takes an engineering view of the world. You usually need
to show experimentally that your method is better than existing methods, at
least under some circumstances or on some kinds of data (rarely, a complexity
analysis may be sufficient). Here, “better” may mean one of several
things:
- It yields lower error rates.
- It is faster.
- It is simpler to implement.
Additional useful distinguishing features can be the following, but
they are rarely sufficient by themselves:
they are rarely sufficient by themselves:
- It is easier to understand.
- It is theoretically better
justified. - It is available in open
source form.
The required experiments make these kinds of papers similar to
benchmarking papers, but there are some differences. While the goal of a
benchmarking paper is to gain insights into existing methods (data sets
represent a wide range of conditions, authors are supposed to have no
preferences), the goal of the experiments of a “new method” paper is
to show that the method is actually better. But that also means that the reviewers
will take a more adversarial view of your paper, meaning, they will try to poke
holes into your arguments. Key questions they will ask are:
benchmarking papers, but there are some differences. While the goal of a
benchmarking paper is to gain insights into existing methods (data sets
represent a wide range of conditions, authors are supposed to have no
preferences), the goal of the experiments of a “new method” paper is
to show that the method is actually better. But that also means that the reviewers
will take a more adversarial view of your paper, meaning, they will try to poke
holes into your arguments. Key questions they will ask are:
- Is the problem you’re solving
interesting at all? - Are the conditions under
which your method is better than other methods too restrictive? - Are the advantages of your
method sufficient large? - Does your method have so many
parameters that it is hard to implement? - Did you tune your method to
the specific dataset, i.e. do the results generalize? - Is the comparison fair? I.e.,
did you unfairly bias the data or preprocessing in favor of your method?
Generally, when you write up your paper, you need to think about
the list of objections that reviewers might make, and you need to have answers
to them.
the list of objections that reviewers might make, and you need to have answers
to them.
“I performed observations/measurements on
X” (Research Paper)
X” (Research Paper)
This is the more standard scientific research paper; you don’t
create something and you don’t invent a new method for doing something, instead
you observe and interpret existing systems. In some way, it is like a benchmark
paper.
create something and you don’t invent a new method for doing something, instead
you observe and interpret existing systems. In some way, it is like a benchmark
paper.
These papers are, on balance, easier to write than method papers,
because it is usually easier to get publishable results out of observations.
There are a lot of opportunities for these kinds of papers in computer vision,
pattern recognition, social network analysis, etc.
because it is usually easier to get publishable results out of observations.
There are a lot of opportunities for these kinds of papers in computer vision,
pattern recognition, social network analysis, etc.
“I
have a new mathematical model/theory for observation X.” (Model Paper)
Mathematical modeling or theory is a very different approach from
experimental approaches, but the basic rules still apply: your result needs to
be novel, interesting, relevant, and useful somehow. Generally, mathematical
models or theories should make predictions. Some of those predictions, you can
compare against existing data, others may suggest new experiments.
experimental approaches, but the basic rules still apply: your result needs to
be novel, interesting, relevant, and useful somehow. Generally, mathematical
models or theories should make predictions. Some of those predictions, you can
compare against existing data, others may suggest new experiments.
“I found a new/different interpretation of observation
X.” (Response Paper)
These papers are generally responses to the literature, where you
read a paper and decide that its conclusions, model, or theory aren’t quite
right. These papers tend to be shorter and fairly easy to write (because
someone else has already done all the background research). They may involve
experiments illustrating the point you are trying to make. They are often good
papers to write, but they are difficult to plan for because, of course, you
don’t know when you’ll come across a paper you can respond to.
read a paper and decide that its conclusions, model, or theory aren’t quite
right. These papers tend to be shorter and fairly easy to write (because
someone else has already done all the background research). They may involve
experiments illustrating the point you are trying to make. They are often good
papers to write, but they are difficult to plan for because, of course, you
don’t know when you’ll come across a paper you can respond to.
“A
new framework/’theory’ for X.” (Framework Paper)
These papers often don’t involve any experiments, theory, or
serious review of the literature. Instead, they provide a view of a field,
observations, or data. If you hit the “Zeitgeist” just right, these
kinds of papers can be spectacularly successful and highly cited, in particular
if you have already made a name for yourself. However, most of them end up just
languishing at a conference.
serious review of the literature. Instead, they provide a view of a field,
observations, or data. If you hit the “Zeitgeist” just right, these
kinds of papers can be spectacularly successful and highly cited, in particular
if you have already made a name for yourself. However, most of them end up just
languishing at a conference.
Preparation
for Ph.D.
Although you may be enrolled in a Ph.D. program, you may first need
to learn how to work with the literature and how to conduct experimental
research. At some other universities, this phase is delimited from the actual
start of the Ph.D. program by the submission and presentation of a formal Ph.D.
proposal. We don’t have that requirement at our university, but the two phases
really still exist.
to learn how to work with the literature and how to conduct experimental
research. At some other universities, this phase is delimited from the actual
start of the Ph.D. program by the submission and presentation of a formal Ph.D.
proposal. We don’t have that requirement at our university, but the two phases
really still exist.
- Seminars and
Projects. Before you can write reviews and papers, perform
benchmarks, and perform experiments, you need to learn these things. As
part of our graduate program, you should learn reviewing as part of your
seminar course, and benchmarking and/or experimenting is commonly a
project topic. The seminar and project are the main time during which
someone will look at your work and give you detailed feedback. They can do
this because it’s area and research they understand well. Later, your
advisor will be able to give you much less input and help on your Ph.D.
project because they will know fewer of the details of your Ph.D. work. - Collaborate. Another
way of gaining experience with reviews and research work is to collaborate
on research and papers with others. This may include detailed
proof-reading of their papers, contributing to their writing, taking over
experimental work from them using their methods, or applying special
skills you have to their problems. When collaborating, think not only
about the obvious benefit (like maybe another publication), but also what
you can learn from the collaboration. - Small
Projects. If you need more experience or still don’t know what
exactly you want to do for your Ph.D., in addition to collaborations, you
can also take on smaller projects. These are projects similar to what your
advisor might give you in a project course, but they are not formal parts
of your course requirements and you don’t get ECTS points for them. You
should still aim for getting publications out of them. There’s usually a
long list of such small projects to choose from.
The previous steps are preparatory for your Ph.D. You really
shouldn’t spend too much time on them because you don’t want to spend too much
time on your Ph.D. At some point, when you have a reasonable handle on how to
work with the literature, how to run experiments, and what you want to do, you
need to pick a Ph.D. topic. That’s when your Ph.D. studies usually start.
shouldn’t spend too much time on them because you don’t want to spend too much
time on your Ph.D. At some point, when you have a reasonable handle on how to
work with the literature, how to run experiments, and what you want to do, you
need to pick a Ph.D. topic. That’s when your Ph.D. studies usually start.
Ph.D.
Research
Actual Ph.D. research, like all research, really means three steps:
- familiarizing yourself with
the literature - getting familiar with the
research methods and tools - conducting actual research
Note that you don’t usually start this process from zero;
presumably, you have already done a little bit of work in the area–maybe as
part of collaborations or projects–simply to determine that you actually are
interested in it.
presumably, you have already done a little bit of work in the area–maybe as
part of collaborations or projects–simply to determine that you actually are
interested in it.
Each of these steps should produce some output, in the form of a
review, a benchmark, and research contributions.
review, a benchmark, and research contributions.
- Review/Proposal.
Start by writing a review paper of the field your thesis is in. You should
plan on presenting this at a seminar. You can actually do this for seminar
credit. Sometimes, you can publish this as a review article, although that
usually involves a lot more work. This should also give you a good idea of
what the open problems are and how success is measured in your area. A
good review with meaningful conclusions is basically equivalent to a Ph.D.
proposal. - Benchmark.
Next, implement algorithms, collect datasets, and perform benchmarks. You
can do this as part of a project course. If your results warrant it, this
is fairly easy to publish at least at a conference. This will give you
better understanding of the data and get you set up for your own
experiments. - Research
Contributions. There are actually two major kinds of research
contributions: new methods or observational studies. New
Methods. Then you start implementing your own ideas and methods,
benchmark them against the standard methods, and start writing up the
results, resulting in several publications. Observational
Studies. You can also make part or all of your thesis observational
studies. You should consider this possibility seriously because it can be
easier than “new methods” work. However, while for engineering and methods
research, the primary deliverable is something that works better, for
observational studies, scientific reasoning and understanding is more of a
challenge. (The “benchmarks” for observational studies are replication of
other people’s observations.)
This division and three step approach are useful, and it’s the way
researchers generally approach starting research in a new field. But it is not
formally required and many students end up working on a lot of different
projects and ideas and then at some point putting everything together into a
thesis. Each of project implicitly still requires a literature review,
benchmarks, and new contributions, but they happen in different orders and
perhaps aren’t made explicit in the form of publications. There is nothing wrong
with that if that works for you and if you get a good thesis out of it.
However, if you are limited in the amount of time you have for your Ph.D., if
you feel lost or are concerned about completing your Ph.D., then you should
consider picking a single topic and focusing on the three steps for research.
researchers generally approach starting research in a new field. But it is not
formally required and many students end up working on a lot of different
projects and ideas and then at some point putting everything together into a
thesis. Each of project implicitly still requires a literature review,
benchmarks, and new contributions, but they happen in different orders and
perhaps aren’t made explicit in the form of publications. There is nothing wrong
with that if that works for you and if you get a good thesis out of it.
However, if you are limited in the amount of time you have for your Ph.D., if
you feel lost or are concerned about completing your Ph.D., then you should
consider picking a single topic and focusing on the three steps for research.
Scholarships
and Ph.D. Proposals
Some scholarship organizations will require you to submit a Ph.D.
proposal in order to get the scholarship. That is a useful starting point, but
it is rarely as detailed as a true Ph.D. proposal and literature review should
be (although the more detailed you make it, the less work and uncertainty you
will have later). Furthermore, after getting the scholarship, you may still
need to spend some time with seminars, projects, etc. to get more familiar with
research methodology.
proposal in order to get the scholarship. That is a useful starting point, but
it is rarely as detailed as a true Ph.D. proposal and literature review should
be (although the more detailed you make it, the less work and uncertainty you
will have later). Furthermore, after getting the scholarship, you may still
need to spend some time with seminars, projects, etc. to get more familiar with
research methodology.
This means that you shouldn’t assume that your Ph.D. proposal is
finished just because you submitted something with the scholarship application.
On the other hand, scholarship agencies are generally fairly flexible about
proposals and you can change them later as you learn more about a research
area.
finished just because you submitted something with the scholarship application.
On the other hand, scholarship agencies are generally fairly flexible about
proposals and you can change them later as you learn more about a research
area.
Problems
You Will Encounter
Of course, in real life, things don’t necessarily work as smoothly.
Common problems are:
Common problems are:
- The literature
seems to confusing and you can’t make a coherent review or presentation
out of it. This means that you really don’t understand the field
well enough yet to even start your research work. When you get serious
about your Ph.D. thesis (or research in general), you should have a good answer
to the question of “what are the open questions in your area”,
“what are promising ways of attacking them”, and “what are
the most common ways in which people have tried to solve these problems
before and why have they failed.” As part of this, you should become
more of an expert on your chosen area than your Ph.D. advisor. - The
benchmarking doesn’t yield meaningful results. One of the most
important skills you need to learn as part of your Ph.D. is how to conduct
meaningful experiments. This means asking the right questions, choosing
the right datasets, performing the right measurements, and presenting the
results in a meaningful and compelling way. Your Ph.D. advisor can help
you and give you feedback, although it will largely be of the kind of
feedback you would get from a reviewer: he will point out problems and
limitations with your approach. Fixing those problems is your job. Your
advisor may be able to give you a little input on that, but benchmarks and
data sets are very hands-on problems, so that kind of input is limited. - Your new
method isn’t working better than the old one. Perhaps your idea
just isn’t a good one, or maybe the dataset you are testing it on isn’t
good for showcasing its advantages. This is the kind of thing your advisor
can help most with: he can provide ideas for other, new approaches and
suggest other ways of evaluating your method. However, you shouldn’t just
abandon an approach if it isn’t working; you probably invested a lot of
time in trying to make it work and you should try to learn from the
experience and understand why it isn’t working. If you just keep trying
one idea after another and never figure out why they are failing, you are
going to make very slow progress. On the other hand, if you learn from
your failures, even if you don’t get a publication out of it, you are
still making progress.
Other
Strategies
Here are some other strategies you need to keep in mind:
- Talk with other people about
your work. It doesn’t matter whether they are experts in your area or not.
One of the most important parts about talking to other people is that you
learn to explain your work better. If you are lucky, they will give you
useful feedeback. - When you talk to your
advisor, have clear and concise explanations of what you have done, and
have specific questions you ask. If you don’t, your advisor simply can’t
help you very much. - In addition to your primary
work, do some other things: collaborate with others on projects
(contributing the expertise you have obtained from your primary research),
and also take on some small and easy additional side-projects. - Pick your research area so
that there are lots of opportunities for collaboration and additional
projects.
See Also:
- Motivation: Why PhD?
What is PhD?
- What does it Mean to Have a PhD: Myths of Specialization and Departmental Expertise
- What is the Difference between MS/MPhil Research and PhD Research
Why PhD is Difficult:
- Why PhD is Difficult to Complete and Why there are so many ABDs and PhD Dropouts
- How Progress of Research is related to the Mood and Psychology of a PhD Student
Starting with your PhD
Reading Research and Writing Your Research
- What is a Problem Statement and its role in MS-PhD Research
- What is a Thesis Statement and its Role in PhD-MS Research
- How Literature Review of a PhD Dissertation Presents the State of the Art: Synthesis vs Listing
- How to Read a Research Paper and Extract Problem Statement and Thesis Statement
- What is meant by Rigor of PhD Research
- Dynamic Role of Abstract in Guiding the Flow of Writing of a PhD Dissertation
- Conclusion vs Assumption in Research Writing- Flipping the Thread of Argument in your PhD Thesis
- PhD is about Pursuit of Excellence. Pursuit of Excellence vs Guzara: How to teach excellence through everyday examples
Qualitative Learning from a PhD
- Myth: Impact Factor Measures Real Impact
- Pursuit of Excellence vs Guzara: How to teach excellence through everyday examples
- Discerning the Forest from the Trees – The Insights from my PhD Supervisor JC Browne
- A Formula is Worth a Thousand Pictures: Dijkstra vs Buzan’s Mind-Maps
- Fairness in Grading: A Lesson by the Great Dijkstra
- Lesser known dimensions of US Universities – Archives of history and literature
Myths:
- Myth: Impact Factor Measures Real Impact
- Myth: We are Backward because we Lag Behind in Science and Technology
- Beauty is Our Business – Mathematics, Excellence and the Great Dijkstra
- 5 Myths of Higher Education in Pakistan
Leave a Reply