Making every click count
We’re not much more than six weeks away from the DPS conference in France where Meg and I (but mostly Meg) are presenting our (but mostly Meg’s) work on estimating the number of planets revealed by Planet Hunters’ analysis of Kepler data.
For many of us working on or taking part in Planet Hunters, the motivation is discovery – and who wouldn’t want a piece of cosmic real estate, a planet discovered through our efforts? Much of the Keck follow-up we’ve been doing has been aimed at exactly this, but it’s not easy.
Distribution of Kepler candidates as of February
Kepler, you see, was never really designed as a mission which would definitively pin down the discovery of a vast number of new planets. Most of the 160,000 or so stars being monitored in the Kepler field are faint, on the limits of what can be easily studied from the ground. That’s why – even with 1235 candidates on the scoreboard at the Kepler site this morning, only 17 have been confirmed.
But this doesn’t matter. The main science goal for Kepler is to characterize the population of planets that are lurking out there – to determine how many Jupiter-, Neptune- and, of course, Earth-like planets there might be. If we know, for example, that only 95% of Neptune-sized planet detections are real, then we need not waste time determining which are the 1 in 20 that are confusing the sample.
How would we come up with that 95% number? The Kepler team could measure the number of false positives – planet candidates that aren’t actually planets – by following up on a carefully selected subset of their candidates. In fact, you can read what they’ve been up to in this paper from February.
But what about false negatives – planets that the Kepler pipeline passed by? We already suspect that Planet Hunters have been successful in finding candidates that the Kepler team originally missed. It’s here that we can help, and that’s the thrust of the work that we’ll be presenting at DPS. If all goes well, we should be able to make an independent estimate of the frequencies of types of planets.
We’ll write more about how we’re doing as the conference approaches, but in the meantime the more classifications we get the better the data we can use. And, of course, with each click comes the chance of making that elusive discovery…
New Site Guide
Today’s post comes from Thomas Esty, our undergraduate summer student working on Planet Hunters.
Hello again, Planet Hunters.
I’m happy to say that we are now rolling out the new Site Guide. It includes a collection of information from the science page, the early blogs and some all new stuff. We also have examples of the different types of light curves so that you can all see prototypical regular, irregular, pulsating, and quiet curves. I’ve been putting it together piece by piece for a couple weeks while also working on doing modeling and coding to analyze the best of your classifications so far. I hope that it will help get more people involved in this project. If you have friends who you haven’t introduced to Planet Hunters, now is a great time. You’ll be able to find the new material under the Tutorial tab as Site Guide. We’ve recently passed 3.5 million classifications and are still going strong. With a paper in the pipeline, the classifications rolling in, plenty of data left to analyze, and conferences coming up, we’d like to thank all of you for your hard working in making Planet Hunters so successful in just over 7 months of operation.
As always, Happy Hunting.
Thomas Esty
What happens next… Peer Review
Following the news that the first Planethunters paper has been submitted, we thought we’d write a little bit about what peer review is and what it does. This is relevant not just in charting out the future of our first paper, but also in the wider discussion of scientific results in the media. This post is an adapted version of a post I wrote for the first Galaxy Zoo paper that was submitted in 2008.
What’s scientific publishing all about? How does it work? If you’ve followed the blog and the forum, you have a pretty good idea of the first part of the scientific process: discovery! We figure out something new about the universe and how it works.
This is one of the amazing and unique things about science. Good scientists spend most of their time arguing against the effects they see in their own data, to avoid falling into traps of seeing only what they expect to see. To see how unique and amazing this is, try to imagine a politician arguing against a piece of legislation s/he is sponsoring! This process of double, triple, and quadruple-checking one’s own work is a very important part of science.
Once we were convinced that we really understood what is going on, we could then write up our conclusions in the form of a scientific paper. Over the past few weeks, the Planethunters team, together with colleagues from the Kepler team, wrote up a paper describing the first results. The paper was passed back and forth between people who made edits and comments and the paper thus passed the first through the first check — our own examination of our results.
e next step in scientific research is to submit the paper to a journal. This has now happened, and the paper Fischer et al. (2011) (where “et al.” means “and the rest,” including YOU!!) has been submitted to the top UK journal Monthly Notices of the Royal Astronomical Society (MNRAS).
The editor of this journal will now select an anonymous referee who can comment on the scientific and technical merits of the paper. The referee is another astronomer or cosmologist whom the editor can ask for an expert assessment of the work. He or she will have a few weeks to read it, think about it, and then make a number of recommendations to the editor of the journal. There are three options. The referee can reject the paper outright. This generally happens very rarely, except in highly competitive top journals like Nature and Science. They can support publication of the paper, asking for only a few minor modifications. This also happens quite rarely, though! The most common outcome is for her to write a “referee report,” suggesting a number of modifications and ask for clarifications. The referee might have questions about some part of the analysis, suggest some alternative thoughts and ideas, or criticise the methodology. Sometimes referees can be hostile to a paper; but often, they are genuinely helpful and constructive.
After receiving the report, we get a few weeks to digest it and modify the paper according to the referee’s comments, and argue against the points raised that we disagree with. This process may repeat itself a number of times if the referee isn’t happy with our modifications, and so it can often take weeks and months for a paper to get to a decision by the editor (acceptance or rejection). If a referee is being particularly unreasonable, we can write to the editor requesting a new referee. In extreme circumstances, we could even choose to submit the paper to a different journal and hope for a more reasonable referee.
The whole process is generally known as peer review since the referee is a peer — a fellow scientist and expert in the field. If the paper is accepted, it will appear both in the online and print version of the journal after another few weeks or months. A paper accepted in such a journal is then considered peer-reviewed.
It’s important to note that something said in a “peer-reviewed” paper isn’t necessarily true. The point of peer-review is to weed out obviously flawed paper whose logic has holes or whose data don’t support the conclusion. Knowing that a paper has been peer-reviewed should give you extra confidence that its results are believable – that means that an expert in the field has read through the paper and thinks its conclusions are believable.It’s really just the first step of proper “peer-review,” because the process continues.
As the community of astrophysicists digests the paper, they too pass judgement on whether they consider the paper important and whether they believe the conclusion. Thus, in the years after publication, other astrophysicists might deem Fischer et al. (2011) a key paper and cite it in the future, commenting on it positively. Or they might disagree with it, but that would still be a sign that it was important enough to comment on. Or it might just fade into obscurity if astronomers don’t consider it important. That’s the historical legacy of a paper – and that’s the ultimate peer-review.
Good news (with any luck)
Just a quick note to say that the Planet Hunters team have submitted the first scientific paper to come from the project. It’s been sent to the same journal the Galaxy Zoo team uses, MNRAS, and we’re waiting on the edge of our seats to see what the referee makes of it. Once it gets accepted, we’ll share the results with you…
Data loss
Dear planet hunters. First let me start off by saying that its fantastic to see the planetometer getting so high, it constantly impresses me how much work the planet hunters community has done and continues to do each day. We worked it out recently and the community is the equivalent of 51 full times employees constantly working on the site!
We take the work you do very seriously and try to make sure that none of your time and clicks on the site are wasted. We make sure that every night the entire database of your classifications on every one of our sites is backed up. We rarely have to use these backups and have only had to resort to them twice, once was after the big outage of amazon web services about a month back and the other was last week on the 28 of June. While trying to look at some of your data and prepare it for the science team I accidentally issued a command to the database that the live site uses rather than the copy of that database I have on my machine. The result was not good and we had to restore the database from the pervious nights back up meaning we lost any classifications that where made between the backup and my error. This was a terrible thing to happen and I cant apologise enough to the people who lost classifications.
Thankfully we only lost about 4 hours of classifications between 6:40 am and 10:40 am (uk time) which is a quiet time for the site. Hopefully this means that most of our users will have been snugly in bed or getting ready for work so I hope the damage was minimal. Even still we always try and learn from our mistakes so that they dont happen again in future.
As always, we value your continued hard work and effort.
Thanks
Stuart
Undergraduate Summer Research
Today’s post comes from Thomas Esty, our undergraduate summer student working on Planet Hunters.
Hello Planet Hunters, my name is Thomas Esty and I am an undergraduate at Harvard who is working with the PH team at Yale this summer. I’ve been here for 5 weeks now, and have been working on a number of projects related to the Planet Hunters site. The first one is going to roll out soon—it’s a new How-To/Getting Started page that brings together information from the science page, the blog, and even some new material to give new users a more in-depth look at how to use the website and what they might see in looking at the light curves. Take a look at it when it goes up, you might learn something new—we discuss simulations, flares, and eclipsing binary stars among other things. It’s also makes a great time to introduce new people to Planet Hunters. Another project is doing statistical analysis on the 1.5 million point data set from the Planet Hunters Quarter 1 classifications. We’re looking at how people classify stars as variable or quiet and then the type of variability. The last thing that I’ll be doing is starting to build an educational outreach program using the Planet Hunters site. I’m looking to create lesson plans and worksheets for middle and high school level students to get involved in the search for exoplanets and get excited about doing real science. All of us are excited by how far we’ve come with your help in making Planet Hunters a success and we’re looking forward to what we will be able to do to expand the reach of this site and of citizen science in general.
Happy Hunting,
Thomas
Live Chat Video
Last’s night live chat on UStream was great fun. The video is now available on Vimeo and you should be able to see it below:
We got through a lot of questions, and Meg has said she will go back onto the Talk thread and answer a few of the remaining ones in the next few days.
We hope to do more live chats in the future, and we’re hoping to learn from our efforts during last night’s chat (there was one spectacular technical fault, but we recovered from it within a few minutes). If you have any suggestions for how they could improved let us know in the comments.
Candidates page down
We’ve been having some trouble with the candidates page today (tentatively because we’re receiving repeated hits on it from a single external source), and so we’ve taken it down temporarily. We hope to have it back soon.
Chris
P.S. Don’t forget the live chat later today…
6 Months Poster
To celebrate Planet Hunters turning six months old we’ve created this massive poster [27MB download] containing the names of our community of more than 30,000 volunteers*! We hope you enjoy it, and that you are able to find your name in amongst the many that make up this lovely image of a planetary system far, far away.
Don’t forget to join us this afternoon/evening/other for our first ever live video chat on UStream. We’ll be answering your questions from 8pm UT – which is 9pm in London and 4pm in New York. Come and see Meg Schwamb, Chris Lintott and Stuart Lynn answering your questions by visiting the Zooniverse UStream channel.
*Names are only shown for users who gave permission for us to show their name on the Zooniverse account settings. To update your settings login to https://www.zooniverse.org/account and update the ‘name’ field.
A Very Unusual System?
Today’s blog post is brought to you by zookeeper extraordinaire Chris Lintott.
Last week Meg visited me in Chicago for two days of number crunching and data discussion and analysis. We got a lot done (and crushed Zooniverse developer Michael Parrish at shuffleboard) and I’ll write more about that soon, but a lot of our talk centered on a very unusual system unearthed by a few persistent Planet Hunters.
Once the team had – prompted by posts at Talk – taken a closer look at the system in question (we’re keeping the name under wraps for now) we were pretty excited, but also slightly worried. The system in question seems to have multiple transits, but they imply that there would be two large planets relatively close to each other. So close, in fact, that a back of the envelope calculation suggests that they would be expected to disrupt each other’s orbits. So unless our rough working is wrong (certainly possible) or we’ve caught the system in an unusual time during its evolution (surely unlikely) then there’s something mysterious here.
That, of course, is the perfect excuse for an observing run. We’re lucky enough to have access to the Keck telescopes on Mauna Kea in Hawai’i (probably my favourite place in the whole world). Observing at Keck is a little different from the telescopes that I’m used to – rather than trekking to the summit where a lack of oxygen can make observing difficult, Keck astronomers observe remotely, either from sea level in Hawai’i or from their home institutions.
The initial goal of our observations wasn’t to confirm the existence of the planets – the star in question is too faint to make that easy – but to rule out obvious problems. In particular, the team were worried that we weren’t looking at a single star, but receiving light from a combination of a nearby star and a background eclipsing binary which would then be responsible for some or all of the transits. If that’s the case then we should be able to see relatively large wobbles revealed by the stellar spectrum as the binary stars move back and forth. These will be larger than the faint wobble induced by the planets (if they exist) and checking whether they exist or not will take no more than a couple of observations, each lasting less than an hour.
I’ll report back on the Keck observations. Fingers crossed!
Cheers,
Chris
Planet Hunters 