Simulated Transits
In this blog post, we wanted to focus on the simulated transits you’ve been seeing and why they’re important to the project, as well as answer some of the questions regarding them.
One of the goal of Planet Hunters is to explore the diversity of the terrestrial and giant planet populations and begin to understand the spectrum of solar systems providing crucial context for own solar system. How many Jupiter sized planets are out there? How many Neptune-sized? How many Earth-sized? are solar systems like ours common? These questions are fundamental to understanding how planets form and evolve.
With just the planet discoveries alone you can’t answer these questions because you don’t know how complete the sample is. This is because you don’t know how sensitive to detecting planets of different types the project is, particularly since this is a new way to look for planets that has never been done before. If we found one earth-sized planet for example. We can’t say anything about their abundance compared to gas giant planets, since we don’t know how many we might have missed in the data set -that’s where the simulated transits come in.
We added Kepler lightcurves into the PH interface with simulated transits, spanning the range of exoplanet radii and orbital periods, to test which kinds of transiting planets can be detected with Planet Hunters to assess the fraction of missed planets. If users flag 100% of the Jupiter-sized planets with orbital periods shorter than 30 days, but only 50% of the Neptune-size planets with orbital periods shorter than 30 days, then we know that the number of transiting Neptunes in the real light curves is a factor of two larger than what has been flagged. This provides a powerful statement about the fraction of transiting planets that could only be made with the Planet Hunter collective.
It might seem like we’re testing you or trying to train you to identify transits , but we’re really testing the project. This is a really vital part of the project, with these simulated transits we can answer these really interesting and fundamental questions about how solar systems and planets form.
Some of the simulated planets like large Jupiter-sized planets will be really easy to spot while others will be near impossible to identify especially for the extremely small planets, but don’t be discouraged if you didn’t find the simulated transit. That’s okay, that’s part of the experiment. We don’t know what Planet Hunters we will be able to detect so we have to look at the look at range of possible planet radii and orbits. Can we find 1.2 Earth radii planets? 1.6? and how does incompleteness change in this critical range of radius? How much worse does detectability get when there is just 1 transit instead of 3?- with the simulated transits we will be able to answer these questions. With this information we can then start putting a picture together of the abundance and variety of solar systems.
We will always identify the simulated transit points in red after you’ve classified the star and will mark the lightcurve as simulated data in Talk. The reason we don’t identify the simulated data first, is that if you knew the lightcurve had simulated events you might look at it differently. To be able to use the data from the simulated transits accurately, we need them to be examined in exactly the same conditions as the real lightcurves.
Users on PH Talk have said that for some of the simulated transits the red points are in the wrong spot. The points we are marking for the simulated transits are correct. There are two reasons why it might look like the points are wrong
- The lightcurve have really a small and distat planet injected, and the flux drop caused by this planet would be so small it doesn’t look any different than the normal lightcurve. Right now we’re working to display the radius and period of the simulated transit signal injected, once you’ve classified it, so you can identify this for yourself.
- The star is an eclipsing binary or already has a transit signal from a larger planet than the one we injected into the lightcurve, since we don’t know beforehand which of the lightcurves in the Kepler data set have transiting planets or stars. These simulated events and your classifications for them are still useful because it gives us estimates for multiplanet system and how sensitive we could detect an additional transiting planet.
We know there were a few glitches we needed to work out with the simulated lightcurves that were making them conspicuous, we’ve fixed those, and the zoom works for the simulated lightcurves. We’ve also dialed back how frequently a user will see a simulated lightcurve. We’ll post some examples of the simulated transits in the next blog post.
Happy Hunting,
25 responses to “Simulated Transits”
Trackbacks / Pingbacks
- - May 20, 2014
Planet Hunters 
I really enjoy the simulated transits, even the tiniest ones. They give me some impressions on my own limits of perception. One thing is for sure – I won’t find any small planets in these sets of data… And I bet few people will, if any.
Two questions:
1. I’m sure you will publish a paper describing the outcome of this experiment and I’m also sure a lot of us will be eager to know what this outcome will be. Any chance of making such paper publicly available and/or publishing a more down-to-Earth-sized-planet abstract that a person without statistical background will be able to understand?
2. I haven’t come across any simulated transits in light curves of obvious variable stars (not including extrinsic, eclipsing vars). Woulnd’t they be particularly useful? There are lots of vars in Kepler data and it’s extremely hard to distinguish changes caused by variability from changes caused by a transit, unless the transit is absolutely obvious. What is the ration of simulations in vars to simulations in quiet stars?
The problem with those “wrongly placed red dots” is that it is IMPOSSIBLE to discerne those transits because the interfa(r)ce auto-zooms to the binaries transit amplitude, what makes the planetary transits vanish into nothing.
Using the rescaling option isn’t one, because the resulting constantly jumping y-scale makes things unbearable …
Great post. Its easy to see how if results from these simulated transits showed us that PH users found 3% of the simulated 1.5 Earth transits, and a few years from now we have confirmed discovering 30 actual 1.5 Earth planets based on transits seen by PH users in real star data, that data from these simulations would imply that there are likely close to a thousand more similar 1.5 Earth planet transits in the data that have not been seen. Looking forward to the follow-up blog with examples.
Is there any way to simply remove the simulated transits from Talk altogether? I understand why they are valuable to the project as a whole, but I’m not entirely what benefit we get from having them on Talk.
I am looking forward to seeing the examples, especially of the less obvious transits
Hi leftfish,
We will be absolutely making any papers we public available to all you and we write some blog posts explaining the results. As for the quiet to variable star ratio – we added simulated transits to all types of stars in the dataset – we used every star to generate a simulated transit lightcurve so there’s a whole variety
~Meg
Hi Derek,
It was choice made early on because we thought people might want to look at them – I’ve asked our design team if we can not have the simulated lightcurves show up on the community page
~Meg
Hi,
You say it’s important that we don’t know which curves are simulations and which are not before we do our analysis, because it would change the way we look at the curve.
I fully understand this, but then why do the simulated stars all have the same radius and temperature??
The Simulated Transits have been helpful as has this explanation. I came looking for information about them because one particular simulated transit came up three times within 30-45 minutes. There might be a bug.
The site is great fun. Thanks for letting us contribute to the project.
Thanks Megan. I can’t wait to see both the paper and my first simulated var 🙂
“I fully understand this, but then why do the simulated stars all have the same radius and temperature??”
It was a small bug that we have fixed. The radii and temperature should be showing the correct values for the simulated transits now
Thanks for your answer Megan!
This project really is a great job 😉
‘We don’t know what Planet Hunters we will be able to detect so we have to look at the look at range of possible planet radii and orbits. Can we find 1.2 Earth radii planets? 1.6? and how does incompleteness change in this critical range of radius? How much worse does detectability get when there is just 1 transit instead of 3?’
I’d hope the Stellar radius also plays a part your calculations on our accuracy! A 2x earth radii planet is a lot easier to spot around a 1x sol radii than one with 5x! Another thing that might be very interesting is how we do on stars that we classify ‘quiet’ versus ones that are variable(I bet we do the worst on pulsating variables!)
I think those two variables are just as important to whether I spot a given transit as planetary size and orbital period.
I have seen several simulated stars, some of them show transits and other nonsense.
It’s impossible to say that where they put the red dots are actual transits if they are not clearly below the average brightness of the star.
The brightness of some stars is jumping all over the place and then they put put red dots at some random places, in the upper range and lower range of the graph.
If they continue to tell these are transits I’m not going to waste my time on this project anymore.
I already have questions of the use to discover planets this way, what is the use of planets which rotate ate least more than one every month (the window in which the data is measured).
These planets are so close to their sun they are very probable not able to support any kind of life.
So if anyone can convince me to continue please do, I have been interested in science, the universe, etc soI was glad to help in any way…..
Patrick. They are not looking at charts to find transits, then offering them as simulations. They are not randomly selecting dots. They are, in fact, 100% exactly where they are suppose to be to simulate a planet. You need to understand what they are doing.
They are taking charts with no known transits, selecting a series of sample dots (such as 10 consecutive dots every 9 days) and lowing the position of those dots a small amount to SIMULATE at transit. The number of dots and the amount lowers will all depend on the size of the planet being simulated and the type of star it is orbiting.
So, lets look at 10 sample dots. Lets say the first two are high points (above average), then a low point, then 5 high points and then 2 more low points. Now lower those so that they drop a small amount to simulate a planet. You do not automatically get 10 lower points because of this. Instead you still have 2 high, 1 low, 5 high and 2 low points relative to each other. If the amount of the dip of the simulated planet is less than the noise of that star, you are not likely going to notice it.
That is the purpose of the simulated planets you have likely been seeing. You are helping the science team confirm that a planet of that type (Jupiter, saturn, nepturn, earth) orbiting a star of that type (noisy, pulsating, quiet, etc) will NOT be noticed (or noticed only a VERY small amount) even though it is 100% known to be there (100% since the data point were knowingly manipulated to place them right where the red dots in the simulation are).
As to finding an earth-like planet, that is just one type of thing that is doable using the Kepler data. What is the use of finding planets which orbit so fast? The fact that they are what is actually there. The fact that it helps us learn about what type of planetary systems are out there. Is our solar system the norm? Or the lucky exception where most are giants orbit close to their stars? If we find one large planet close, do those systems have other planets as well? Is it more likely that they do or less likely? Many questions can come up and they are not all about finding new earth-like planets.
Hi,
Even with a long training, I’m not sure I will ever find this sort of transit:
http://talk.planethunters.org/objects/APH10167655
@Greg_78_FR
We don’t really expect anyone to, and the machines are no better. However, it is still important to _know_ that we won’t see many (or any) 1.4 R_earths in 900 day orbits even if they are there. Think of it is measuring our collective sensitivity…what is the smallest thing (as a group) that we can find.
I have been helping put together the list of candidates which you all have identified and planet hunters is doing a fantastic job with a sensitivity that is pretty impressive.
I like the “fakes” it validates my gut feelings and reassures me about not making too many false positives (by nature I err on the side of caution).
I am curious about two points. How do you interpret partial recognition of difficult transits? I had a couple of earth-like “fakes” for which I could spot one or two transits, but missed out others (in one case there were 4 transits in total, in the other there were loads but get mixed in with the noise). Would you count me as having spotted the planet? Even though I didn’t pick up on all the transits?
Second point, the last fake I looked at, I could tell it was a simulation because it didn’t have the usual star info in the box (I can’t remember what it said, maybe unknown or something).
Could we have the “simulations” marked out in out “My Stars” Statistics? I learn a lot from them and I am sad they go away afterwards and are not easy to recognise. Do they count to the total counter?
Finally very stupid question: what about those planets with long orbits?
“This light curve has a simulated planet null times the radius of the Earth with a period of null days. What does ‘null’ mean in this context?” I thought null=0 but I expect that I’m wrong.
It means we don’t have the values loaded into the database. I’ll notify the development team and see if we can get the values loaded.
Cheers,
~Meg
Thanks Meg, you’ve saved me from a lot of head scratching!
This might be a silly question but, what’s the point of simulated transits being in quarters other than the one which is being shown? It means that it’s impossible for the hunter to have seen it and so tends to be dispiriting unless they go into the ‘examine star’to find it and the hit rate figures derived surely can’t be an accurate measure of success.
The sims right now are currently for Q4 which is currently being shown in the classify interface – previously they were from Q1 which was the data were analyzing. If you’re seeing one from other quarters do let us know.
~Meg
I have seen several of these simulated plots, but have never seen the red dots. I am using the Chrome browser; do the red spots work in Chrome?