On January 16th, from 6:00 – 6:20 pm Pacific Time there was an error in our production system, causing every event sent to Mixpanel to be counted four times. All of our customers, if looking at an hourly view of their data, will see an artificial spike for the hour between 6:00pm and 7:00pm, similar to the one pictured below:
We are very proud of the accuracy of our data, and are extremely sorry that this error occurred. We know that you make key business decisions based on the data you see in Mixpanel, and even though the miscounting only lasted for twenty minutes, we wanted to make you immediately aware of it. The rest of this blog post will discuss the full details of this error and possible ramifications for your decision-making.
The over-counting impacts all reports on Mixpanel.com or any API calls that request a total count for any event and include 6pm PST on 1/16/12 within the time period of the query. However, this over counting does not impact queries for unique event counts. This means that the Funnels report and the Retention report, which are entirely based on uniques, are completely unaffected. In addition, any queries in the Segmenation or Trends report in uniques mode are not impacted either.
How can I adjust my data to account for this? In most cases, we do not recommend trying to adjust your data to account for this error. For daily reports, the difference will be trivial. For the day of January 16th total event counts will be roughly 5% higher than they were in actuality. For monthly reports, the total event counts will only be 0.1% higher than they were in actuality. The only case where you might want to adjust the data coming out of Mixpanel is in the case of hourly reports. If you are basing a decision on an hourly report, then divide the count for your total events that happened during the hour of 6 PM by 2. The result will be very close to the true event count for that hour.
Will I be billed for these data points? Absolutely not. You will not be charged for any overages to your plan caused by this error.
I’m a geek – tell me what really happened
First, it’s necessary to describe a small part of our infrastructure. When a user sends an event to Mixpanel, we do a small amount of validation — mostly checking for syntactical correctness — and then immediately put the event on a queue. Under normal circumstances, the number of items on the queue stays very close to zero, meaning that within seconds of sending an event it should show up in your reports. However, decoupling receiving events from processing them allows us to easily perform server maintenance that would otherwise require significant downtime.
For a long time now, we’ve had multiple queue servers so we aren’t reliant on a single machine, but we haven’t had automated failover. In practical terms, that means that if a queue server goes down in the middle of the day we can do a manual failover within minutes, but if it goes down in the middle of the night, it could be quite a bit longer before we can switch everything over. The change we pushed out Monday was intended to remedy this situation. Basically, when an event comes in, we try each queue server that we currently think is up one at a time until we successfully enqueue an item. Unfortunately, our code to check whether putting an item on a queue was successful or not was incorrect and consequently each event was added to each queue server (currently, there are four). We noticed the problem almost immediately and had the fix within 20 minutes.
What we are doing to keep this from happening again
Unfortunately, although we have queueing related tests, in our test environment there is only one queue server and so none of our tests caught this particular problem. That particular hole will be fixed in the coming days.
Once again, our most sincere apologies and regrets for this error. If you have any questions please do not hesitate to reach out to us at support@mixpanel.com.
Today we are rolling out our new retention tracking system. Now, instead of choosing a user’s cohort on an event-by-event basis (the first time the event was fired) you tell Mixpanel their cohort by sending a $born or $signup event (in fact, this event can be called anything, the key is that you only send it once per distinct_id. If you call it $signup there will be a slight improvement in performance for your report). Usually you would send this event when they first make an account, but a lot of our customers have been asking us about their existing users. How can you tell Mixpanel which cohort to put your current users in?
You can do this, but there are a couple prerequisites:
You must be already be using mpq.identify() or distinct_id to keep track of your users within Mixpanel
You must have an existing record tying the identify() or distict_id value you have been sending to Mixpanel with a date you want to use as a user’s birthday. For example: a users table in your database with user_id and date_created fields.
If you meet those prerequisites, you can write a one-off script to send the $born event to Mixpanel for all your users (each request will count as a Mixpanel data point for billing purposes). To do this, for every user in your database you would make a GET request that looks like this:
This request is very similar to our standard HTTP API (documented here). The data parameter is a Base64 encoded JSON array with the event you are importing ($born) and the associated properties. By decoding the Base64 data parameter from the above request you can see this:
The ‘distinct_id‘ property is the user ID you have been sending to Mixpanel up to this point for that user. For Javascript Library users – this is the value you put in mpq.identify()
The time property determines which cohort the user will be in. It is in Unix Epoch format (seconds since 1970). Times should be GMT. The above example, 1321499371, represents November 17th at 3:09 AM GMT.
The token property is your Mixpanel project token.
There are three differences between this import method and the regular HTTP API:
this method lets you import events older than 48 hours
the endpoint is /import/ instead of /track/
as an added level of security, you must include your API key as a parameter outside the Base64
At the moment, the /import endpoint will only work for importing $born and $signup events into Mixpanel.
You can download a sample PHP script that you can modify with your own data, token and API key to send $born or $signup events to Mixpanel.
If you have trouble importing $born or $signup events for your current users, please reach out to support@mixpanel.com.
We rolled out some big changes to the way retention reports work. Now you can slice and dice your retention data the same way you can in the Segmentation report.
To our existing customers, the biggest change is in the way cohort groups are determined. In the past, you could only select a single event for retention analysis, now you can select two – one for determining the cohort (usually you would pick a signup or $born event for this) and one to look at the retention for. This is explained in detail on our documentation page, and in the video below the fold.
You may be wondering: “what happened to compounded retention?” Now all retention is compounded retention – when you pick the first event, we will include all users in the cohort who fired that event, not just those who fired it the first time. To see birth retention, you need to fire an event that will only be sent once and determine the birthday for each user. This event can be called whatever you want, but we recommend calling it $born or $signup. To learn how to properly send a $born or $signup event, read this blog post. To learn how to import these events for your existing users, we have a blog post about that as well!
If you have any questions about new retention, we will be hosting a Live Q&A Webinar tomorrow at 11 AM PST. To tune in visit http://www.mixpanel.com/webinar then. (more…)
This morning in San Francisco, and this afternoon in Europe, we conducted the European edition of our Mixpanel 101 webinar. Below is the recording of the event. Topics covered included:
– Streams
– Events
– Properties
– Funnels
Most Mixpanel users are used to treating properties as strings of text, but some data is more suitable to be treated as a number, or other data type.
Numbers
Let’s say you were instrumenting a music sharing site, and you sent a property with every “song played” event called “length” that was a number – the length of the song in seconds. If you send the property as an integer or decimal (122 or 3.45) Mixpanel will detect it as a number, and will build a histogram showing what song lengths people typically listened to in the segmentation report:
In the above case, 60 – 90 second movies were clearly the favorite! You can also build segments with greater than and less than operators on numeric data. This would let you examine the 60-90 second songs in more detail:
It looks like 77 and 78 second songs are the most popular within this segment.
Sometimes a histogram isn’t enough though – you just want an average. If your data is a number, and you’ve set at least one filter on it, you can have Mixpanel perform the following mathematical calculations on it for a given time frame:
Average
Sum
Minimum Value
Maximum Value
The average song length for songs between 60 and 90 long hovers around 75 seconds:
If you don’t want to put a filter around it and calculate the overall averages or sums, just choose a filter that wouldn’t eliminate any results. In this case “length is greater than 0” will do the trick. However, most of the time you will want to filter to eliminate extremes that represent bad data that will skew your averages.
If you are sending an event as a number but Mixpanel is not detecting it as a number, you can type cast it by hovering your mouse over the property in the segmentation report, moving it to the right, and selecting the “number” variable type. This will actually work for any property type:
Dates
You can also send dates to Mixpanel. Let’s say you wanted to break down your song plays only to videos that had been uploaded to your site in the past week. If you send a property formatted like a date, Mixpanel will automatically treat it as such, and let you build segments based on date logic.
Segment Example:
The above example would limit your report to plays of new songs that were added in the past month.
Mixpanel accepts a wide variety of date formats. If any of the following appear in a property value, Mixpanel will treat that value as a date:
yyyy-mm-dd
dd-mm-yyyy
yyyy/mm/dd
dd/mm/yyyy
a three letter month like Jan,Feb,Dec etc
a three letter day like Mon,Tue,Wed, etc
You can also include a time after the date, so “yyy-mm-dd HH:MM:SS.” If you are including time, please use UTC time.
Lists
Often you might want to set a property with more than one value. Some examples might be:
Products purchased in a “Order Placed” event
Multiple authors in an “Article Viewed” event
Different genre tags in a “Song Played” event.
If you send a JSON array as a property value, Mixpanel will know that it is a list, and allow you to build segments that query for individual list items.
Today we released a new data type on our platform to give you more power and flexibility with your data: Dates.
Now you can send Mixpanel a date and we’ll let you manipulate your data. You can find this in the segmentation report.
Examples
Last time a user logged in
The date that a user signed up
The date of a specific action: photo upload, tweet, etc.
Integration
Integration is very intuitive. All you have to do is pass a property with a date formed as: “YYYY-MM-DD” (2011-01-01). You can also give us up to the second granularity if you’d like: “2011-01-02 03:34:21.” We will automatically interpret data that looks like this as a date.
Keep in mind, that you should send us dates that are in UTC format if they have granularity beyond a day level. If you send us data on a day granularity then we’ll simply take that and not convert it to UTC as it would be ambiguous.
We’ve just launched a brand new feature for Mixpanel called Formulas. Formulas is a powerful tool that allows you to combine and filter events arithmetically. If you ever wanted to view a trend of two events combined, or looked at the ratio of one event to another, or even both, then Formulas is the tool for you. Like always, this is happening in real time, and it will work retroactively with the data you’re already sending us!
How to use it
You may have noticed the new Formulas icon already on the left.
Clicking on it brings up the Formulas creation interface.
At the top, you can enter a name to save your formula to view later.
In the body, you can select the events that you want to use in your formula. Selecting an event will also bring up its properties, on which you can filter further. You can also choose whether you want to count the total number of events, or count only unique events (counted once per user that sends the event).
Initially, you can only compute the ratio between two events. Hitting the tab on the right will expand your view, where you can then add, subtract, or multiply two events.
Hitting continue will save this formula for you and display the trend.
An example formula
Here’s an example of a formula that we use at Mixpanel:
In our Streams product, we have two tabs, Streams and Users. We send these respective events anytime a new user goes to view one of these tabs. This formula lets us view the percentage of unique people that view the Streams tab. We simply take the #streams event, and divide it by the sum of #users and #streams events.
We hope formulas will help you understand your data even more. As always, feel free to contact us at support@mixpanel.com if you have any questions!
Mixpanel has an extremely powerful tool for creating and viewing funnels. A funnel is simply a sequence of events. Once you have created a funnel, you can track the conversion rate of people that go from one event to the next in your sequence. Mixpanel also lets you filter each stage by properties to finely control the types of events you want in your funnel.
In this post, we’ll show you how to do all this with Mixpanel.
1. You can find funnels by clicking on the funnel icon on the left hand side:
Once here, you can click the dropdown to view and edit funnels you already have, or to create a new one. Click on ‘Create a funnel’ to do exactly that.
1. Defining your funnel
You should see a screen similar to the following:
At the top, type in a name for the funnel you want to create. Then, click on the input box for each step to see all your events. You can select an event by clicking on it directly, or begin typing in the input box to filter. Once you have selected an event, you can click on the right arrow if you want to add an additional condition on this event in your funnel. To add another step, simply click the ‘Add step’ button on the right hand side.
In our example above, we have created a funnel to track conversions from viewing our homepage, to viewing our signup page, to actually signing up. In addition, we only care about homepage views that came from a search keyword of ‘mixpanel.’ This powerful filtering gives you an incredible amount of control over what you want to see in your funnel.
Finally, hit continue to create your funnel.
Understanding your funnel
Now you should see a screen similar to the following. At the bottom, you should see tabs for all the steps you defined. Above them, you should see a graph containing the number of people that made it to each step in your funnel for the date range. In our example, 3,686 people viewed our homepage, and 378 of them viewed the signup page after that from September 23rd to September 30th.
You can view data for different dates by simply clicking on the date widget. Clicking on an arrow will show you a graph of how the conversion rate between those two steps has changed in the past. This can help you determine whether changes you make are actually boosting the number of users that are getting through your funnel.
You can also click on a tab to see specific details about that step.
This lets you see additional information, such as the median time it took users to go from the last step to this one. Below this, you can also see the trend of the conversion rate from the very first step to the currently one. This will help you track whether changes you make are increasing the overall conversion rate to each step.
We also let you segment each step. By selecting a property from the dropdown menu, we divide up all the users that made it to the selected step by that property. This can help you determine whether there are groups of users that are better or worse at converting to the next step, or what types of users you should focus on in increasing conversion.
Remember that our funnels are completely retroactive. If you have already been sending us events, you can already build funnels using them. There isn’t any additional configuration that you have to do, or additional data that you have to send us.
