About results and weighted data
Adjustment to weighting scheme notification (came into effect on 15th December 2011)
Please note that a minor adjustment has been made to the GPPS weighting scheme for the 2011-2012 survey.
In Summer 2011, Ipsos MORI undertook a weighting investigation to find out if the weighting scheme could be refined and developed further to help improve the accuracy of the results. The investigation was conducted on the previous years’ data (i.e. fieldwork conducted between 5th April 2010 and 7th April 2011). The results of the investigation concluded that when neighbourhood statistics such as ethnicity and deprivation are accounted for, this results in reducing non-response bias, thus improving the accuracy of findings. Please click below to read the reports on the rationale for this approach.
PLEASE NOTE: results published on 15th December 2011 use the new weighting scheme. As the weighting scheme has changed, as well as changes to questionnaire design and survey frequency, it is not possible to make direct comparisons with previous years’ data, even in cases where the same questions have been asked.
Why has the weighting scheme been adjusted?
The weighting scheme has been adjusted to make the data better represent the views of the population as a whole. They now take into account local factors (such as deprivation, crime levels, ethnicity, marital status, overcrowding in households, household tenure and employment status).
Why won’t it be possible to make direct comparisons from previous years’ data once 2011-2012 data are published?
As the weighting scheme has changed, incorporating neighbourhood statistics, as well as changes to the questionnaire design and survey frequency, it is not possible to make like-for-like comparisons. This means that even in cases where the same questions have been asked in the new survey, and in previous years, direct comparisons between 2011-2012 results, and previous years' results cannot be made.
What is weighting? Why do you weight the data?
Weighting adjusts the data to account for potential differences between the demographic profile of all eligible patients in a practice and the patients who actually complete a questionnaire: for example, if there are more 18-24 year olds registered with a GP surgery compared to the number of 75-84 year olds, we would expect to receive more questionnaires from the 18-24 year olds. But we know from previous results that we usually receive more questionnaires from 75-84 year olds. We also know that younger and older people have very different views of their GP and very different experiences of visiting their GP surgery, so this difference will have an impact on the overall results.
Using this example, if 5% of returned questionnaires are from 18-24 year olds, but we know that this group makes up 10% of the eligible practice population (patients over 18 registered with a GP in England), then we can ‘weight’ the responses to reflect this. By applying weights, the results for a practice will more accurately reflect the views of the practice population.
Why do you recommend using the weighted data for analysis of the dental question results?
Weighted results are more accurate when analysing the NHS dentistry questions, as we are interested in how the results reflect the NHS dental access of the PCT population and are not interested in the results relating to a particular GP practice.
As the sampling of the Survey is dependent on the size and levels of response at GP Practice level (rather than the PCT level), the unweighted results at PCT level may not be as precise and using the weighted results would reduce this effect.
I've seen cases where, when adding up the number of people who have selected each different response for a question, the total does not match the figure in the 'total number of responses' column. Why is this?
This can happen when weighted data is rounded to a whole number.
When weights are applied, decimals are added to the number of responses in each category and the total number of responses. This means that, occasionally there can be cases where the number of responses differs from the base size. For example, if a report says that 59 people say ‘yes’ and 14 say ‘no’, but the number of responses is 74 (not 73), that means that the weighted values are actually 59.345 and 14.456, which add up to 73.801, (which is then rounded up to 74).
There are some cases where the responses for a question are not showing, and instead there is a "~" symbol. Why?
In cases where fewer than 10 people have answered the question, the data has been suppressed. This is to prevent individuals and their responses being identifiable in the data.
In the weighted reports, there are some cases where this suppression is also applied to questions where the total number of responses is 10. This is again due to rounding. If the total number of responses when weighted is less than 10 (e.g. 9.856), but has been rounded to 10 in the report, then the data will be suppressed. If the weighted total number of responses is, for example, 10.245, then the total number of responses will also show as 10 but the responses will be shown.
There are cases where, for example, when adding the % of 'very easy' and 'easy', the total does not match what you would get if you were to add the percentages manually. Why is this?
This is again due to rounding the weighted totals to whole numbers.
I have noticed cases where the results seem to show that nobody answering gave a particular response to a question, but the percentage which relates to that response shows as *% (indicating it is less than 0.5% but more than 0%). Can you explain why this is?
This is because, in the unweighted data, there may be a single person who has given a response. However, when weighted, this response has been given a value of greater than zero but less than 0.5 and, therefore, rounded to 0. Because the actual value is still greater than zero, in percentage terms this shows as *%.
I have seen cases where the same numbers of responses are given to two different answer categories, but the percentage values are different. Why?
There are examples in the reports where, for example, it looks like one person has selected 'Other' and one person selected 'I would prefer not to say', but their corresponding percentages are 1% and 2%. Again, this happens when the results and number of responses are rounded but the percentages are calculated on un-rounded data.
