AAAS Symposium in Feb. 2015: Cutting-Edge Research with 1 Million Citizen Scientists

[This is an expanded version of a post I wrote for the Galaxy Zoo blog.]

Some colleagues and I successfully proposed for a symposium on citizen science at the annual meeting of the American Association for the Advancement of Science (AAAS) in San Jose, CA in February 2015. (The AAAS is the world’s largest scientific society and is the publisher of the Science journal.) Our session will be titled “Citizen Science from the Zooniverse: Cutting-Edge Research with 1 Million Scientists.” It refers to the more than one million volunteers participating in a variety of citizen science projects. This milestone was reached in February, and the Guardian and other news outlets reported on it.


“Citizen science” (CS) involves public participation and engagement in scientific research in a way that educates the participants, makes the research more democratic, and makes it possible to perform tasks that a small number of researchers could not accomplish alone. (See my recent post on new developments in citizen science.)


The Zooniverse began with Galaxy Zoo, which recently celebrated its seventh anniversary, and which turned out to be incredibly popular. (I’ve been heavily involved in Galaxy Zoo since 2008.) Galaxy Zoo participants produced numerous visual classifications of hundreds of thousands of galaxies, yielding excellent datasets for statistical analyses and for identifying rare objects. Its success led to the development of a variety of CS projects coordinated by the Zooniverse in a diverse range of fields. For example, they include: Snapshot Serengeti, where people classify different animals caught in millions of camera trap images; Cell Slider, where they classify images of cancerous and ordinary cells and contribute to cancer research; Old Weather, where participants transcribe weather data from log books of Arctic exploration and research ships at sea between 1850 and 1950, thus contributing to climate model projections; and Whale FM, where they categorize the recorded sounds made by killer and pilot whales. And of course, in addition to Galaxy Zoo, there are numerous astronomy-related projects, such as Disk Detective, Planet Hunters, the Milky Way Project, and Space Warps.


We haven’t confirmed the speakers for our AAAS session yet, but we plan to have six speakers from the US and UK who will introduce and present results from the Zooniverse, Galaxy Zoo, Snapshot Serengeti, Old Weather, Cell Slider, and Space Warps. I’m sure it will be exciting and we’re all looking forward to it! I’m also looking forward to the meeting of the Citizen Science Association, which will be a “pre-conference” preceding the AAAS meeting.

Comparing Models of Dark Matter and Galaxy Formation

I just got back from the “nIFTy” Cosmology workshop, which took place at the IFT (Instituto de Física Teórica) of the Universidad Autonoma de Madrid. It was organized primarily by Alexander Knebe, Frazer Pearce, Gustavo Yepes, and Francisco Prada. As usual, it was a very international workshop, which could’ve been interesting in the context of the World Cup, except that most of the participants’ teams had already been eliminated before the workshop began! In spite of Spain’s early exit, the stadium of Real Madrid (which I visited on a day of sightseeing) was nonetheless a popular tourist spot. I also visited the Prado museum, which had an interesting painting by Rubens involving the Milky Way.


This was one of a series of workshops and comparison projects, and I was involved in some of the previous ones as well. For example, following a conference in 2009, some colleagues and I compared measures of galaxy environment—which are supposed to quantify to what extent galaxy properties are affected by whether they’re in clustered or less dense regions—using a galaxy catalog produced by my model. (The overview paper is here.) I also participated in a project comparing the clustering properties of dark matter substructures identified with different methods (here is the paper). Then last year, colleagues and I participated in a workshop in Nottingham, in which we modeled galaxy cluster catalogs that were then analyzed by different methods for estimating masses, richnesses and membership in these clusters. (See this paper for details.)

This time, we had an ambitious three week workshop in which each week’s program is sort of related to the other weeks. During the first week, we compared codes of different hydrodynamical simulations, including the code used by the popular Illustris simulation, while focusing on simulated galaxy clusters. In week #2, we compared a variety of models of galaxy formation as well as models of the spatial distributions and dynamics of dark matter haloes. Then in week #3, we’re continuing the work from that Nottingham workshop I mentioned above. (All of these topics are also related to those of the conference in Xi’an that I attended a couple months ago, and a couple other attendees were here as well.)

The motivation of these workshops and comparison workshops is to compare popular models, simulations, and observational methods in order to better understand our points of agreement and disagreement and to investigate our systematic uncertainties and assumptions that are often ignored or not taken sufficiently seriously. (This is also relevant to my posts on scientific consensus and so-called paradigm shifts.)

Last week, I would say that we had surprisingly strong disagreement and interesting debates about dark matter halo masses, which are the primary drivers of environmental effects on galaxies; about the treatment of tidally stripped substructures and ‘orphan’ satellite galaxies in models; and various assumptions about ‘merger trees’ (see also this previous workshop.) These debates highlight the importance of such comparisons: they’re very useful for the scientific community and for science in general. I’ve found that the scatter among different models and methods often turns out to be far larger than assumed, with important implications. For example, before we can learn about how a galaxy’s environment affects its evolution, we need to figure out how to properly characterize its environment, but it turns out that this is difficult to do precisely. Before we can learn about the physical mechanisms involved in galaxy formation, we need to better understand how accurate our models’ assumptions might be, especially assumptions about how galaxy formation processes are associated with evolving dark matter haloes. Considering the many systematic uncertainties involved, it seems that these models can’t be used reliably for “precision cosmology” either.

A few thoughts on the peer-review process

How does the peer-review process work? How do scientists critically review each others’ work to ensure that the most robust results and thorough analyses are published and that only the best research proposals are awarded grants? How do scientists’ papers and articles change between submission and publication? It’s a process that has advantages and shortcomings, and maybe it’s time for us as a community to try to improve it. (I’m not the only person who’s written about this stuff, and you may be interested in other scientists’ perspectives, such as Sarah Kendrew, Andrew Pontzen, and Kelle Cruz. This blog on Guardian has interesting relating posts too.)

For us scientists, writing about our scientific research and writing proposals for planned research is a critically important aspect of the job. The ubiquity of the “publish or perish” maxim highlights its significance for advancing one’s career. Publishing research and evaluating and responding to others’ publications are crucial for scientists to try to debate and eventually reach a consensus on particular issues. We want to make sure that we are learning something new and converging on important ideas and questions rather than being led astray by poorly vetted results. Therefore, we want to make the peer-review process as effective and efficient as possible.

Female researcher taking notes

For readers unfamiliar with the process, it basically goes like this: scientist Dr. A and her colleagues are working on a research project. They obtain a preliminary result—which may be a detection of something, the development of a new model, the refutation of a previously held assumption, etc.—which they test and hone until they have something they deem publishable. Then Dr. A’s group write a paper explaining the research they conducted (so that it potentially could be repeated by an independent group) and lay out their arguments and conclusions while putting them in the context of other scientists’ work. If they can put together a sufficiently high-quality paper, they then submit it to a journal. An independent “referee” then reviews the work and writes a report. (Science is an international enterprise, so like the World Cup, referees can come from around the world.) The paper goes through a few or many iterations between the authors and referee(s) until it is either rejected or accepted for publication, and these interactions may be facilitated by an editor. At that point, the paper is typeset and the authors and copy editors check that the proof is accurate, and then a couple months later the paper is published online and in print.

(In my fields, people commonly publish their work in the Astrophysical Journal, Monthly Notices of the Royal Astronomical Society, Astronomy & Astrophysics, Physical Review D, and many others, including Nature, where they publish the most controversial and provocative results, which sometimes turn out later to be wrong.)

In general, this system works rather well, but there are inherent problems to it. For example, authors are dependent on the whim of a single referee, and some referees do not spend enough time and effort when reviewing papers and writing reports for authors. On the other hand, sometimes authors do not write sufficiently clearly or do not sufficiently double-check all of their work before submitting a paper. Also, sometimes great papers can be delayed for long periods because of nitpicking or unpunctual referees, while other papers may appear about they were not subjected to much critical scrutiny, though these things are often subjective and depend on one’s perspective.

There are other questions that are worthwhile discussing and considering. For example, how should a scientific editor select an appropriate referee to review a particular paper? When should a referee choose to remain anonymous or not? How should authors, referees, and editors deal with language barriers? What criteria should we use for accepting or rejecting a paper, and in a dispute, when and in what way should an editor intervene?

Some authors post their papers online for the community on (the astronomy page is here) before publication while others wait until a paper is in press. It’s important to get results out to the community, especially for junior scientists early in their careers. The early online posting of papers can yield interesting discussions and helpful feedback which can improve the quality of a paper before it is published. On the other hand, some of these discussions can be premature; some papers evolve significantly and quantitative and qualitative conclusions can change while a paper is being revised in the referee process. It is easy to jump to conclusions or to waste time with a paper that still needs further revision and analysis or maybe even is fundamentally flawed. Of course, this can also be said about some published papers as well.

implications for science journalists

These issues are also important to consider when scientists and journalists communicate and when journalists write or present scientific achievements or discoveries. Everyone is pressed for time, and journalists are under pressure to write copy within strict deadlines, but it’s very important to critically review the relevant science whenever possible. Also, in my opinion, it’s a good idea for journalists to talk to a scientists colleagues and competitors to try to learn about multiple perspectives and to determine which issues might be contentious. We should also keep in mind that achievements and discoveries are rarely accomplished by a single person but by a collaboration and were made possible by the work of other scientists upon which they’ve built. (Isaac Newton once said, “If I have seen further it is by standing on the shoulders of giants.”)

Furthermore, while one might tweet about a new unpublished scientific result, for more investigative journalism, it’s better of course to avoid rushing the analysis. We all like to learn about and comment on that new scientific study that everyone’s talking about, but unfortunately people will generally pay most attention to what they hear first rather than retractions or corrections that might be issued later on. We’re living in a fast-paced society and there is often demand for a quick turnaround for “content”, but the scientific enterprise goes on for generations—a much longer time-scale than the meme of the week.

improving the peer-review process

And how can this peer-review system be improved? I’ve heard a variety of suggestions, some of which are probably worthwhile to experiment with. We could consider having more than one person review papers, with the extra referees providing an advisory role. We could consider paying scientists for fulfilling their refereeing duties. We could make it possible for the scientific to comment on papers on the arXiv (or VoxCharta or elsewhere), thus making these archives of papers and proceedings more like social media (or rather like a “social medium”, but I never hear anyone say that).

Another related issue is that of “open-access journals” as opposed to journals that have paywalls making papers inaccessible to people. Public access to scientific research is very important, and there are many advantages of promoting open journals and of scientists using them more often. Scientists (including me) should think more seriously about how we can move in that direction.

Thoughts on the Academic Job Market in the Physical Sciences

I decided to add “Thoughts on…” at the beginning of the title to emphasize that, although I’ll present some facts, I’ll be expressing my personal opinions on the academic job market. These are my “2 cents”, and some people may disagree with them. And though there are some similar issues and concerns in the social sciences and humanities, most of my experience comes from the physical sciences, especially physics and astronomy, and I’ll focus on that. If you don’t have the time to read the whole post, my main (and obvious) point is this: for a number of reasons, the job market has been getting worse over the past decade or more, with detrimental effects to scientific research and education (and to scientists, educators, and students). This is just a brief intro to the issues involved, and I’m not sure what the best solutions might look like, but I’ll try to write about that more in another post.

Soft Money

For people with Ph.D.’s, in the past, they’d decide upon earning their degree (or earlier) whether to proceed with the “traditional” academic career or shift to another kind of career. Those who continue would consider moving to a tenure-track faculty or other long-term position at a college, university, or other institution. With the growth of “soft money, a euphemism for uncertain funding from external federal (e.g., National Science Foundation) or occasionally private sources, short-term postdoctoral positions and fellowships have proliferated. For various reasons, soft money has become a very important part of the funding landscape (see this article in Science in 2000 and this more recent article).

One consequence of this is that most people in astrophysics now need to work at two or three or even more postdoc/fellowship positions before potentially having a shot at a long-term or more secure position. In my case, I’ve already done two postdocs myself, at the Max Planck Institute of Astronomy in Heidelberg and at the University of Arizona, and now I’m a research scientist at UC San Diego and this and my previous position were funded by soft money. The job market for the tenure-track faculty positions has become increasingly worse, and it has worsened with the financial crisis. Note that there are other career options as well, such as those associated with particular projects or programs.

Another consequence is that every couple years people need to spend a considerable amount of time and effort applying for the next round of jobs. In addition, people spend a lot of time writing and submitting research grants—to try to obtain more soft money. As a result, grant acceptance rates are now very low (sometimes less than 10%) and senior positions are very competitive. All of these applications also take time away from research, outreach, and other activities, so one could argue that a lot of scientists’ time is thereby wasted in the current system.

Moreover, this system perpetuates inequalities in science, which I’ll describe more below. It also reinforces a workforce imbalance (as pointed out in this article by Casadevall & Fang) where the senior people are mostly well-known males and the larger number of people at the bottom of the hierarchy are more diverse. In addition, although it can be fun to travel and live in different places, for people in couples or with families, it becomes difficult to sustain an academic career. (See these posts for more on diversity and work-life balance issues.)

The Adjunct Crisis

The job market and economic situation at US colleges and universities has spawned the “adjunct crisis” in teaching and education. Much has been written about this subject—though maybe not enough, as it’s still a major problem. (There’s even a blog called “The Adjunct Crisis.”) The number and fraction of adjunctions continues to grow: the NY Times reported last year that 76% (and rising) of US university faculty are adjunct professors.

The problem is that adjuncts are like second-class faculty. Employers are able to exploit the “reserve army of labor” and create potentially temporary positions, but now adjuncts are relied upon much more heavily than before to serve as the majority of college instructors. According to this opinion piece on Al-Jazeera, most adjuncts teach at multiple universities while still not making enough to stay above the poverty line. Some adjuncts even depend on food stamps to get by. The plight of adjuncts received more media attention when Margaret Mary Vojtko, an adjunct who taught French for 25 years at Duquesne University in Pittsburgh, died broke and nearly homeless. Adjuncts clearly need better working conditions, rights, and a living wage.

Inequalities in Science

As I mentioned above, the current job market situation reinforces and exacerbates inequalities in science. The current issue of Science magazine has a special section on the “science of inequality,” which includes this very relevant article. The author writes that one source of inequality is what Robert Merton called the “Matthew effect,” such that the rich get richer: well-known scientists receive disproportionately greater recognition and rewards than lesser-known scientists for comparable contributions. As a result, a talented few can parlay early successes into resources for future successes, accumulating advantages over time. (If you’re interested, Robert Merton was a sociologist of science whose work is relevant to this post.) From the other side of things, we’re all busy, and it’s easy to hire, cite the work of, award funding to, etc. people who know are successful scientists, even though many lesser known scientists may be able to accomplish the same thing with that grant or position or may have published equally important work; but then more time needs to be spent to research all of the lesser known people, who can publish and still perish.

The author, Yu Xie, also points out that the inequality in academics’ salaries has intensified, some academic labor is being outsourced, and one can be effected down the road by one’s location in global collaborative networks. If one does not obtain a degree at a top-tier university, then this can be detrimental in the future regardless of how impressive one’s work and accomplishments are. We can attempt to get around this last point by spending the time to recognize those who aren’t the most well-known in a field or at the most well-known institutions but who have considerable achievements and produced important work.

“Love What You Do”

Finally, I’ll end by talking about the “Do what you love. Love what you do” (DWYL) nonsense. While this seems like good advice, since it’s great to try to follow your passions if you can, nonetheless it’s both elitist and denigrates work. (I recommend checking out this recent article in Jacobin magazine.) People are encouraged to identify with the work that they love, even if the working conditions and job insecurity shouldn’t be tolerated. The author argues that there are many factors that keep PhDs providing such high-skilled labor for such extremely low wages, including path dependency and the sunk costs of earning a PhD, but one of the strongest is how pervasively the DWYL doctrine is embedded in academia. The DWYL ideology hides the fact that if we acknowledged all of our work as work, we could set appropriate limits for it, demanding fair compensation and humane schedules that allow for family and leisure time. These are things that every worker, including workers in academia, deserve.

From Dark Matter to Galaxies

Since I just got back from the From Dark Matter to Galaxies conference in Xi’an, China, I figured I’d tell you about it. I took this photo in front of our conference venue:
photo 2

Xi’an is an important historical place, since it was one of the ancient capitals of the country (not just the Shaanxi province) and dates back to the 11th century BCE, during the Zhou dynasty. Xi’an is also the home of the terra cotta warriors, horses, and chariots, which (along with a mausoleum) were constructed during the reign of the first emperor, Qin Shi Huang. The terra cotta warriers were first discovered in 1974 by local farmers when they were digging a well, and they are still being painstakingly excavated today.


Back to the conference. This was the 10th Sino-German Workshop in Galaxy Formation and Cosmology, organized by the Chinese Academy of Sciences and the Max Planck Gesellschaft and especially by my friends and colleagues Kang Xi and Andrea Macciò. This one was a very international conference, with people coming from Japan, Korea, Iran, Mexico, US, UK, Italy, Austria, Australia, and other places.

Now scientific conferences aren’t really political exactly, unlike other things I’ve written about on this blog, though this conference did include debates about the nature of dark matter particles and perspectives on dark energy (which is relevant to this post). I should be clear that dark matter is much better understood and determined by observations though, such as by measurements of galaxy rotation curves, masses of galaxy clusters, gravitational lensing, anisotropies in the cosmic microwave background radiation, etc. (On a historical note, one conference speaker mentioned that the CMB was first discovered fifty years ago, on 20 May 1964, by Penzias and Wilson, who later won the Nobel Prize.) In contrast, the constraints on dark energy (and therefore our understanding of it) are currently rather limited.

the main points

I’ll start with the main points and results people presented at the conference. First, I thought there were some interesting and controversial talks about proposed dark matter (DM) particles and alternate dark energy cosmologies. (The currently favored view or standard “paradigm” is ΛCDM, or cold dark matter with a cosmological constant.) People are considering various cold dark matter particles (WIMPS, axions), warm dark matter (sterile neutrino), and self-interacting dark matter. (Warm dark matter refers to particles with a longer free-streaming length than CDM, which results in the same large-scale structure but in different small-scale behavior such as cored density profiles of dark matter haloes.) The jury is still out, as they say, about which kind of particle makes up the bulk of the dark matter in the universe. There were interesting talks on these subjects by Fabio Fontanot, Veronica Lora, Liang Gao, and others.

Second, people showed impressive results on simulations and observations of our Milky Way (MW) galaxy the “Local Group”, which includes the dwarf galaxy satellites of the MW and the Andromeda (M31) galaxy’s system. Astrophysicists are studying the abundance, mass, alignment of satellite galaxies as well as the structure and stellar populations of the MW. Some of these analyses can even be used to tell us something about dark matter and cosmology, because once we know the MW dark matter halo’s mass, we can predict the number and masses of the satellites based on a CDM or WDM. (Current constraints put the MW halo’s mass at about one to two trillion solar masses.) There were some interesting debates between Carlos Frenk, Aldo Rodriguez-Puebla, and others about this.

The third subject many people discussed involves models, and observations of the large-scale structure of the universe and the formation and evolution of galaxies. There are many statistical methods to probe large-scale structure (LSS), but there is still a relatively wide range of model predictions and observational measurements at high redshift, allowing for different interpretations of galaxy evolution. In addition, simulations are making progress in producing realistic disk and elliptical galaxies, though different types of simulations disagree about the detailed physical processes (such as the treatment of star formation and stellar winds) that are implemented in them.

There were many interesting talks, including reviews by Rashid Sunyaev (famous for the Sunyaev-Zel’dovich effect), Houjun Mo, Joachim Wambsganss, Eva Grebel, Volker Springel, Darren Croton, and others. Mo spoke about impressive work on reconstructing the density field of the local universe, Springel spoke about the Illustris simulation, and Wambsganss gave a nice historical review of studies of gravitational lensing. I won’t give more details about the talks here unless people express interest in learning more about them.

my own work

In my unbiased opinion, one of the best talks was my own, which was titled “Testing Galaxy Formation with Clustering Statistics and ΛCDM Halo Models at 0<z<1.” (My slides are available here, if you’re interested.) I spoke about work-in-progress as well as results in this paper and this one. The former included a model of the observed LSS of galaxies, and you can see a slice from the modeled catalog in this figure:

I also talked about galaxy clustering statistics, which are among the best methods for analyzing LSS and for bridging between the observational surveys of galaxies and numerical simulations of dark matter particles, whose behavior can be predicted based on knowledge of cosmology and gravity. I’m currently applying a particular set of models to measurements of galaxy clustering out to redshift z=1 and beyond, which includes about the last eight billion years of cosmic time. I hope that these new results (which aren’t published yet) will tell us more about how galaxies evolve within the “cosmic web” and about how galaxy growth is related to the assembly of dark matter haloes.

International Collaborations

(I actually wrote this post a week ago while I was in China, but many social media sites are blocked in China. Sites for books, beer, and boardgames weren’t blocked though—so they must be less subversive?)

Since I’m having fun on a trip to Nanjing and Xi’an now, seeing old friends and colleagues and attending a conference (From Dark Matter to Galaxies), I figured I’d write a lighter post about international collaborations. By the way, for you Star Trek fans, this month it’s been twenty years since the end of The Next Generation, which had the ultimate interplanetary collaboration. (And this image is from the “The Chase” episode.)


In physics and astrophysics, and maybe in other fields as well, scientific collaborations are becoming increasingly larger and international. (The international aspect sometimes poses difficulties for conference calls over many timezones.) These trends are partly due to e-mail, wiki pages, Dropbox, SVN repositories, Github, remote observing, and online data sets (simulations and observations). Also, due to the increasing number of scientists, especially graduate students and postdoctoral researchers, many groups of people work on related subjects and can mutually benefit from collaborating.

On a related note, the number of authors on published papers is increasing (see this paper, for example). Single-author papers are less common than they used to be, and long author lists for large collaborations, such as Planck and the Sloan Digital Sky Survey, are increasingly common. Theory papers still have fewer authors than observational ones, but they too have longer author lists than before. (I’ll probably write more about scientific publishing in more detail in another post.)

Of course, conferences, workshops, collaboration meetings and the like are important for discussing and debating scientific results. They’re also great for learning about and exposing people to new developments, ideas, methods, and perspectives. Sometimes, someone may present a critical result or make a provocative argument that happens to catch on. Furthermore, conferences are helpful for advancing the career of graduate students and young scientists, since they can advertise their own work and meet experts in their field. When looking for their next academic position (such as a postdoctoral one or fellowship), it helps to have personally met potential employers. Working hard and producing research is not enough; everyone needs to do some networking.

Also, note that for international conferences and meetings, English has become the lingua franca, and this language barrier likely puts some non-native English speakers at a disadvantage, unfortunately. I’m not sure how this problem could be solved. I’m multilingual but I only know how to talk about science in English, and I’d have no confidence trying to talk about my research in Farsi or German. We’ve talked about privilege before, and certainly we should consider this a form of privilege as well.

Finally, I’ll make a brief point about the carbon footprint of scientists and the impact of (especially overseas) travel. For astrophysicists, the environmental impact of large telescopes and observatories in Hawaii and Chile, for example, is relatively small; it’s the frequent travel that takes a toll. I enjoy traveling, but we should work more on “sustainability” and reducing our carbon footprint. There are doubts about the effectiveness of carbon-offset programs (see the book Green Gone Wrong), so what needs to be done is to reduce travel. Since conferences and workshops are very important, we should attempt to organize video conferences more often. In order for video conferences and other such organized events to be useful though, I think more technological advances need to be made, and people need to be willing to adapt to them. Another advantage to these is that they’re beneficial for people who have family, children, or other concerns and for people from outside the top-tier institutions who have smaller budgets. In other words, video conferences could potentially help to “level the playing field,” as they say.

Frontiers of Citizen Science

Since some colleagues and I recently submitted a proposal for a symposium on citizen science at a conference next year, I thought this would be a good time to write some more about citizen science and what people are doing with it. I previously gave a brief introduction to the “citizen science” phenomenon (also called “crowd science”, “crowd-sourced science”, “networked science”, “civic science”, “massively-collaborative science”, etc.) in an earlier post. The presence of massive online datasets and the availability of high-speed internet access and social media provide many opportunities for citizen scientists to work on projects analyzing and interpreting data for research.

Citizen science (CS) is an increasingly popular activity, it’s produced impressive achievements already, and it clearly has potential for more. (It also even has a meme!) You don’t have to look hard to see accomplishments of CS projects in the news. A quick online search brought up citizen scientists studying bumblebees, bird nests, weather events, plankton, and other projects. The growing phenomenon of CS has drawn the interest of social scientists as well, and I’ll say more about their research later in this post.


I’m particularly familiar with the Zooniverse, a platform that hosts projects in a variety of fields. It began in 2007 with the Galaxy Zoo project, which I’ll say more about below, and its other astronomy/astrophysics projects include Disk Detective, Planet Hunters, Moon Zoo, and Space Warps. To give other examples, outside of astronomy, there are projects in zoology, such as Snapshot Serengeti to study animals and their behavior with “camera trap” photos (the graph above describes herbivores they’ve cataloged, from a recent blog post); in biology/medicine, such as Cell Slider to identify cancer cells and aid research; and in climate science, there is Old Weather, which examines ship’s logs to study historical weather patterns. In addition, people at Adler Planetarium and elsewhere are working on producing educational resources and public outreach programs.


Galaxy Zoo (GZ) invites volunteers to visually classify the shapes and structures of galaxies seen in images from optical surveys. The project resulted in catalogs of hundreds of thousands of visually classified galaxies—much much better than anything achieved before—allowing for novel statistical analyses and the identification of rare objects and subtle trends. If you’re interested in my own research, I’m leading clustering and astrostatistical analyses of GZ catalogs to study the spatial distribution of galaxies and determine how their morphologies are related to the dark matter distribution and large-scale structure of the universe. For example, with more and better data than pre-GZ studies, my colleagues and I obtained statistically significant evidence that galaxies with stellar bars tend to reside in denser environments (see this paper). In the figure above, you can see examples of barred galaxies (lower panels) and unbarred ones (upper panels). In 2009, we used the impressive GZ datasets to disentangle the environmental dependence of galaxy color and morphology, since we tend to see redder and elliptical galaxies in denser regions (see this paper). Time permitting, I’d like to extend this work by using those results with detailed dark matter halo models, and we could potentially compare our results to galaxies in the Illustris simulation (which has been getting a lot of media attention and was misleadingly described as “the first realistic model of the universe“).

Galaxy Zoo scientists have many other achievements and interesting research. For example, a Dutch schoolteacher, Hanny van Arkel, discovered a unique image of a quasar light echo, which was dubbed “Hanny’s Voorwerp” (Lintott et al. 2009). GZ volunteers also identified galaxies that appeared to look like “green peas”, and most of them turned out to be small, compact, star-bursting galaxies (Cardamone et al. 2009). In addition, Laura Trouille is leading the Galaxy Zoo Quench project, in which participants contribute to the whole research process by classifying images, analyzing data, discussing results, and writing a paper about them.

Citizen science is related to “big data” and data-driven science (see also this article), and in particular to data mining and machine learning. According to a new astrostatistics book by Ivezic, Connolly, VanderPlas, & Gray, data mining is “a set of techniques for analyzing and describing structured data, for example, finding patterns in large data sets. Common methods include density estimation, unsupervised classification, clustering, principal component analysis, locally linear embedding, and projection pursuit.” Machine learning is a “term for a set of techniques for interpreting data by comparing them to models for data behavior (including the so-called nonparametric models), such as various regression methods, supervised classification methods, maximum likelihood estimators, and the Bayesian method.” Kaggle has data prediction competitions for machine learning, and their most recent one involved challenging people to develop automated algorithms to classify GZ galaxy morphologies like as well as the “crowd-sourced” classifications, and the winning codes performed rather well. Nothing beats numerous visual classifications, but there is clearly much to be learned along these lines.

Finally, sociologists, political scientists, economists and other social scientists have been studying CS, such as the organization and efficacy of CS projects, motivations of participants, and applications to industry and policy making. For example, Amy Freitag has written about how citizen science programs define “success” and their rigorous data collection. The sociologist Anne Holohan has written a book Community, Competition and Citizen Science on collaborative computing projects around the world. Eugenia Rodrigues is studying the views and experiences of participants in CS initiatives, and Hauke Riesch has written on this subject as well. (This is also related to the work by Galaxy Zoo scientists in Raddick et al. on participants’ motivations.)

In a recent interesting article, Chiara Franzoni & Henry Sauermann analyze the organizational features, dimensions of openness, and benefits of CS research. As case studies, they examine GZ, Foldit (an online computer game about protein folding), and Polymath (involving many mathematicians collectively solving problems). They argue that the open participation and open disclosure of inputs, which they mention is also characteristic of open source software, distinguish CS from traditional “Mertonian” science. (Robert Merton was a sociologist who emphasized—perhaps too much—social and cultural factors in science, such as scientists’ desire for peer recognition and career benefits, disputes between scientists, etc. I ended up not discussing him in my post on “paradigm shifts“.) They also discuss knowledge-related and motivational benefits, and they point out that CS projects that involve subjects less popular than astronomy or ornithology, for example, or that address very narrow and specific questions may face challenges in recruiting volunteers. Finally, they discuss organizational challenges, such as division of labor and the need for project leadership and infrastructure. If you’re interested, Bonney et al. in Science magazine is another shorter article about organizational challenges and developments in citizen science.

How scientists reach a consensus

Following my previous post on paradigm shifts and on how “normal science” occurs, I’d like to continue that with a discussion of scientific consensus. To put this in context, I’m partly motivated by the recent controversy about
Roger Pielke Jr., a professor of environmental studies at the University of Colorado Boulder, who is also currently a science writer for Nate Silver’s FiveThirtyEight website. (The controversy has been covered on Slate, Salon, and Huffington Post.) Silver’s work has been lauded for its data-driven analysis, but Pielke has been accused of misrepresenting data, selectively choosing data, and presenting misleading conclusions about climate change, for example about its effect on disaster occurrences and on the western drought.

This is also troubling in light of a recent article I read by Aklin & Urpelainen (2014), titled “Perceptions of scientific dissent undermine public support for environmental policy.” Based on an analysis of a survey of 1000 broadly selected Americans of age 18-65, they argue that “even small skeptical minorities can have large effects on the American public’s beliefs and preferences regarding environmental regulation.” (Incidentally, a book by Pielke is among their references.) If this is right, then we are left with the question about how to achieve consensus and inform public policy related to important environmental problems. As the authors note, it is not difficult for groups opposed to environmental regulation to confuse the public about the state of the scientific debate. Since it is difficult to win the debate in the media, a more promising strategy would be to increase awareness about the inherent uncertainties in scientific research so that the public does not expect unrealistically high degrees of consensus. (And that’s obviously what I’m trying to do here.)

Already a decade ago, the historian of science Naomi Oreskes (formerly a professor at UC San Diego) in a Science article analyzed nearly 1000 article abstracts about climate change over the previous decade and found that none disagreed explicitly with the notion of anthropogenic global warming–in other words, a consensus appears to have been reached. Not surprisingly, Pielke criticized this article a few months later. In her rebuttal, Oreskes made the point that, “Proxy debates about scientific uncertainty are a distraction from the real issue, which is how best to respond to the range of likely outcomes of global warming and how to maximize our ability to learn about the world we live in so as to be able to respond efficaciously. Denying science advances neither of those goals.”

The short answer to the question, “How do scientists reach a consensus?” is “They don’t.” Once a scientific field has moved beyond a period of transition, the overwhelming majority of scientists adopt at least the central tenets of a paradigm. But even then, there likely will be a few holdouts. The holdouts rarely turn out to be right, but their presence is useful because a healthy and democratic debate about the facts and their interpretation clarifies which aspects of the dominant paradigm are in need of further investigation. The stakes are higher, however, when scientific debate involves contentious issues related to public policy. In those situations, once a scientific consensus appears to be reached and once scientists are sufficiently certain about a particular issue, we want to be able to respond effectively in the short or long term with local, national, or international policies or regulations or moratoria, depending on what is called for. In the meantime, the debates can continue and the policies can be updated and improved.

Of course, it is not always straightforward to determine when a scientific consensus has been reached or when the scientific community is sufficiently certain about an issue. A relevant article here is that of Shwed & Bearman (2010), which was titled “The Temporal Structure of Scientific Consensus Formation.” They refer to “black boxing,” in which scientific consensus allows scientists to state something like “smoking causes cancer” without having to defend it, because it has become accepted by the consensus based on a body of research. Based on an analysis of citation networks, they show that areas considered by expert studies to have little rivalry have “flat” levels of modularity, while more controversial ones show much more modularity. “If consensus was obtained with fragile evidence, it will likely dissolve with growing interest, which is what happened at the onset of gravitational waves research.” But consensus about climate change was reached in the 1990s. Climate change skeptics (a label which may or may not apply to Pielke) and deniers can cultivate doubt in the short run, but they’ll likely find themselves ignored in the long run.

Finally, I want to make a more general point. I often talk about how science is messy and nonlinear, and that scientists are human beings with their own interests and who sometimes make mistakes. As stated by Steven Shapin (also formerly a professor at UC San Diego) in The Scientific Revolution, any account “that seeks to portray science as the contingent, diverse, and at times deeply problematic product of interested, morally concerned, historically situated people is likely to be read as criticism of science…Something is being criticized here: it is not science but some pervasive stories we tend to be told about science” (italics in original). Sometimes scientific debates aren’t 100% about logic and data and it’s never really possible to be 0% biased. But the scientific method is the most reliable and respected system we’ve got. (A few random people might disagree with that, but I think they’re wrong.)

Big Science and Big Data

I’d like to introduce the topic of “big science.” This is especially important as appropriations committees in Congress debate budgets for NASA and NSF in the US (see my previous post) and related debates occurred a couple month’s ago in Europe over the budget of the European Space Agency (ESA).

“Big science” usually refers to large international collaborations on projects with big budgets and long time spans. According to Harry Collins in Gravity’s Shadow (2004),

small science is usually a private activity that can be rewarding to the scientists even when it does not bring immediate success. In contrast, big-spending science is usually a public activity for which orderly and timely success is the priority for the many parties involved and watching.

He goes on to point out that in a project like the Laser Interferometer Gravitational-Wave Observatory (LIGO), it’s possible to change from small science to big but it means a relative loss of autonomy and status for most of the scientists who live through the transition. Kevles & Hood (1992) distinguish between “‘centralized’ big science, such as the Manhattan Project and the Apollo program; ‘federal’ big science, which collects and organizes data from dispersed sites; and ‘mixed’ big science, which offers a big, centrally organized facility for the use of dispersed teams.”

In addition to LIGO, there are many other big science projects, such the Large Hadron Collider (LHC, which discovered the Higgs boson), the International Thermonuclear Experimental Reactor (ITER), and in astronomy and astrophysics, the James Webb Space Telescope (JWST, the successor to Hubble), the Large Synoptic Survey Telescope (LSST, pictured below), and the Wide-Field InfraRed Survey Telescope (WFIRST), for example.


Note that some big science projects are primarily supported by government funding while others receive significant funding from industry or philanthropists. LSST and LIGO are supported by the NSF, JWST and WFIRST are supported by NASA, and LHC is supported by CERN, but all of these are international. In the case of the fusion reactor ITER (see diagram below), on which there was a recent detailed New Yorker article, it has experienced many delays and has gone over its many-billion-dollar budget, and it has had management problems as well. While budget and scheduling problems are common for big science projects, ITER is in a situation in which it needs produce results in the near future and avoid additional delays. (The US is committing about 9% to ITER’s total cost, but its current contribution is lower than last year’s and its future contributions may be reevaluated at later stages of the project.)

in-cryostat overview 130116

As scientists, we try to balance small-, mid-, and large-size projects. The large ones are larger than before, require decades of planning and large budgets, and often consist of collaborations with hundreds of people from many different countries. It’s important to be aware that relatively small- and mid-scale projects (such as TESS and IBEX in astronomy) are very important too for research, innovation, education, and outreach, and as they usually involve fewer risks, they can provide at least as much “bang for the buck” (in the parlance of our times).

In the context of “big science” projects these days, the concepts of “big data” and “data-driven science” are certainly relevant. Many people argue that we are now in an era of big data, in which we’re obtaining collections of datasets so large and complex that it becomes difficult to process them using on-hand database management tools or traditional data processing applications. Since the volume, velocity, and variety of data are rapidly increasing, it is increasingly important to develop and apply appropriate data mining techniques, machine learning, scalable algorithms, analytics, and other kinds of statistical tools, which often require more computational power than traditional data analyses. (For better or for worse, “big data” is also an important concept in the National Security Agency and related organizations, in government-funded research, and in commercial analyses of consumer behavior.)

In astronomy, this is relevant to LSST and other projects mentioned above. When LSST begins collecting data, each night for ten years it will obtain roughly the equivalent amount of data that was obtained by the entire Sloan Digital Sky Survey, which was until recently the biggest survey of its kind, and it will obtain about 800 measurements each for about 20 billion sources. We will need new ways to store and analyze these vast datasets. This also highlights the importance of “astrostatistics” (including my own) and of “citizen science” (which we introduced in a previous post) such as the Galaxy Zoo project. IT companies are becoming increasingly involved in citizen science as well, and the practice of citizen science itself is evolving with new technologies, datasets, and organizations.

I’ll end by making a point that was argued in a recent article in Science magazine: we should avoid “big data hubris,” the often implicit assumption that big data are a substitute for, rather than a supplement to, traditional data collection and analysis.

Paradigm Shifts?

In addition to physics and astronomy, I used to study philosophy of science and sociology. In my opinion, many scientists could learn a few things from sociologists and philosophers of science, to help them to better understand and consider how scientific processes work, what influences them and potentially biases scientific results, and how science advances through their and others’ work. In addition, I think that people who aren’t professional scientists (who we often simply call “the public”) could better understand what we are learning and gaining from science and how scientific results are obtained. I’ll just write a few ideas here and we can discuss these issues further later, but my main point is this: science is an excellent tool that sometimes produces important results and helps us learn about the universe, our planet, and ourselves, but it can be a messy and nonlinear process, and scientists are human–they sometimes make mistakes and may be stubborn about abandoning a falsified theory or interpretation. The cleanly and clearly described scientific results in textbooks and newspaper articles are misleading in a way, as they sometimes make us forget the long, arduous, and contentious process through which those results were achieved. To quote from Carl Sagan (in Cosmos), who inspired the subtitle of this blog (the “pale blue dot” reference),

[Science] is not perfect. It can be misused. It is only a tool. But it is by far the best tool we have, self-correcting, ongoing, applicable to everything. It has two rules. First: there are no sacred truths; all assumptions must be critically examined; arguments from authority are worthless. Second: whatever is inconsistent with the facts must be discarded or revised.

As you may know, the title of this post refers to Thomas Kuhn (in his book, The Structure of Scientific Revolutions). “Normal science” (the way science is usually done) proceeds gradually and is based on paradigms, which are collections of diverse elements that tell scientists what experiments to perform, which observations to make, how to modify their theories, how to make choices between competing theories and hypotheses, etc. We need a paradigm to demarcate what is science and to distinguish it from pseudo-science. Scientific revolutions are paradigm shifts, which are relatively sudden and unstructured events, and which often occur because of a crisis brought about by the accumulation of anomalies under the prevailing paradigm. Moreover, they usually cannot be decided by rational debate; paradigm acceptance via revolution is essentially a sociological phenomenon and is a matter of persuasion and conversion (according to Kuhn). In any case, it’s true that some scientific debates, especially involving rival paradigms, are less than civil and rational and can look something like this:

I’d like to make the point that, at conferences and in grant proposals, scientists (including me) pretend that we are developing research that is not only cutting edge but is also groundbreaking and Earth-shattering; some go so far as to claim that they are producing revolutionary (or paradigm-shifting) research. Nonetheless, scientific revolutions are actually extremely rare. Science usually advances at a very gradual pace and with many ups and downs. (There are other reasons to act like our science is revolutionary, however, since this helps to gain media attention and perform outreach in the public, and it helps policy-makers to justify investments in basic research in science.) When a scientist or group of scientists does obtain a critically important result, it is usually the case that others have already produced similar results, though perhaps with less precision. Credit often goes to a single person who packaged and advertised their results well. For example, many scientists are behind the “Higgs boson” discovery, and though American scientists received the Nobel Prize for detecting anisotropies in the cosmic microwave background with the COBE satellite, Soviets actually made an earlier detection with the RELIKT-1 experiment.


Let’s briefly focus on the example of quantum mechanics, in which there were intense debates intense debates in the 1920s about (what appeared to be) “observationally equivalent” interpretations, which in a nutshell were either probabilistic or deterministic and realist ones. My favorite professor at Notre Dame, James T. Cushing, wrote a provocative book on the subject with the subtitle, “Historical Contingency and the Copenhagen Hegemony“. The debates occurred between Neils Bohr’s camp (with Heisenberg, Pauli, and others, who were primarily based in Copenhagen and Göttingen) and Albert Einstein’s camp (with Schrödinger and de Broglie). Bohr’s younger followers were trying to make bold claims about QM and to make names for themselves, and one could argue that they misconstrued Einstein’s views. Einstein had essentially lost by the 1930s, in which the nail in the coffin was von Neumann’s so-called impossibility proof of “hidden variables” theories–a proof that was shown to be false thirty years later. In any case, Cushing argues that in decisions about accepting or dismissing scientific theories, sometimes social conditions or historical coincidences can play a role. Mara Beller also wrote an interesting book about this (Quantum Dialogue: The Making of a Revolution), and she finds that in order to understand the consolidation of the Copenhagen interpretation, we need to account for the dynamics of the Bohr et al. vs. Einstein et al. struggle. (In addition to Cushing and Beller, another book by Arthur Fine, called The Shaky Game, is also a useful reference.) I should also point out that Bohr used the rhetoric of “inevitability” which implied that there was no plausible alternative to the Copenhagen paradigm. If you can convince people that your view is already being adopted by the establishment, then the battle has already been won.

More recently, we have had other scientific debates about rival paradigms, such as in astrophysics, the existence of dark matter (DM) versus modified Newtonian dynamics (MOND); DM is more widely accepted, though its nature–whether it is “cold” or “warm” and to what extent it is self-interacting–is still up for debate. Debates in biology, medicine, and economics, are often even more contentious, partly because they have policy implications and can conflict with religious views.

Other relevant issues include the “theory-ladenness of observation”, the argument that everything one observes is interpreted through a prior understanding (and assumption) of other theories and concepts, and the “underdetermination of theory by data.” The concept of underdetermination dates back to Pierre Duhem and W. V. Quine, and it refers to the argument that given a body of evidence, more than one theory may be consistent with it. A corollary is that when a theory is confronted with recalcitrant evidence, the theory is not falsified, but instead, it can be reconciled with the evidance by making suitable adjustments to its hypotheses and assumptions. It is nonetheless the case that some theories are clearly better than others. According to Larry Laudan, we should not overemphasize the role of sociological factors over logic and the scientific method.

In any case, all of this has practical implications for scientists as well as for science journalists and for people who popularize science. We should be careful to be aware of, examine, and test our implicit assumptions; we should examine and quantify all of our systematic uncertainties; and we should allow for plenty of investigation of alternative explanations and theories. In observations, we also should be careful about selection effects, incompleteness, and biases. Finally, we should remember that scientists are human and sometimes make mistakes. Scientists are trying to explore and gain knowledge about what’s really happening in the universe, but sometimes other interests (funding, employment, reputation, personalities, conflicts of interest, etc.) play important roles. We must watch out for herding effects and confirmation bias, where we converge and end up agreeing on the incorrect answer. (Historical examples include the optical or electromagnetic ether; the crystalline spheres of medieval astronomy; the humoral theory of medicine; ‘catastrophist’ geology; etc.) Paradigm shifts are rare, but when we do make such a shift, let’s be sure that what we’re transitioning to is actually our currently best paradigm.

[For more on philosophy of science, this anthology is a useful reference, and in particular, I recommend reading work by Imre Lakatos, Paul Feyerabend, Helen Longino, Nancy Cartwright, Bas van Fraassen, Mary Hesse, and David Bloor, who I didn’t have the space to write about here. In addition, others (Ian Hacking, Allan Franklin, Andrew Pickering, Peter Galison) have written about these issues in scientific observations and experimentation. For more on the sociology of science, this webpage seems to contain useful references.]