Questionnaires a necessity or a burden?
Surveys and questionnaires have been in place for centuries. The word ‘census’, the mother of all surveys, is of Latin origin. During the Roman Republic period (509 BC-27 BC), the census consisted of a list that kept track of all adult males who were fit for military service.
In modern times surveys have acquired a different meaning and have become a crucial element to understand the patterns of our evolving society, economy and lives, and these go beyond the enquiries carried out by statistical offices.
When we apply for a discount card in a retail outlet, we are asked to complete a survey asking for our details like age and place of residence together with other questions that exhibit personal preferences when it comes to choose a product or a service. Likewise when we contact any service provider by phone, we are often asked to participate in customer satisfaction surveys shortly after being assisted. The list is endless and has become a common occurrence. Yet in each case, we all question the usefulness of the exercise and what is in it for us.
Truth is that a minute spent answering a survey, be it self-administered or coordinated by an interviewer, is a minute taken away from our busy lives.
In recent times, the demand for information has increased considerably. Massive amounts of data are generated as we speak and have gained greater value for individuals, commercial entities and government administration. This includes, but is not limited to, the internet of things, big data and administrative information collected by government entities. These and other factors led the concept of research to evolve over time and have put additional pressure on statistics producers, with statistical offices in the frontline, to combine multiple data sources and to release more information in less time.
Most national statistical offices, particularly European, are mandated to produce high-quality statistics for better decision making: statistics that present a factual picture of economic and social spheres. To do this laborious and incessant task, national and European laws are in place to empower statistical offices fulfil their mission – from the collection of data to the extraction, analysis and release of official statistics in different fields.
Vigorous quality requirements also require statistically representative samples to better reflect the populations they are meant to characterise. The challenge, for both the statistical agencies that need to administer the projects and respondents who have to provide the information, is that surveys are bound to become longer and more frequent to meet the demands for more sophisticated and accurate data.
At present, an estimated 135,000 surveys are carried out by the Maltese statistical office (NSO) each year1, including enquiries for social, business and economic use. Of these, 36,000 questionnaires are sent to businesses, among an enterprise population of roughly the same size. The sampling rate, namely the number of units that are selected as a proportion of the population they represent, is alarmingly high, ranking first in the EU.
To alleviate this burden, the NSO applies an unbiased rotational sampling system, such that whenever possible, same households or enterprises are not re-selected in the same or similar surveys within a specified period of time. Of course, this is more easily implemented among individuals and households as opposed to the business sector, where target groups are notoriously small in size and not more than 50 enterprises dominate the respective sectors of our economy.
In statistics, the response burden, namely the effort needed by interviewees to respond to a particular survey, is measured in terms of how long the survey takes to complete and the costs involved. The Standard Cost Model (SCM) approach, developed in 2004 by a network of organisations, is an example of a standardised course of action to measuring administrative burden imposed by regulation. In 2008, a similar model focusing on citizens was also developed by the Dutch. These models are highly dependent on a number of inputs and are somewhat subjective in nature – factors that make them rather unpopular for some
Ironically, some statistical agencies adopt the so-called ‘response burden questionnaire’ in an attempt to quantify the effort required to participate in surveys. This is done through a dedicated survey which is carried out among past participants to enquire about the time required to complete the questionnaire, the staff involved in the process and other related issues. These factors allow quantifying the overall burden incurred by data providers.
All data users value the fact that having accurate statistics is crucial for decision and policy making purposes. However many argue that the always increasing appetite for data is likely to lead to an increased burden on data providers in the years to come. This concern is equally shared by statistical agencies who already struggle to keep response rates sufficiently high to reach the desired quality levels.
The answer lies in a step change in information technology and increase re-use of information which is already available instead of collecting data from first-hand experience.
It is difficult to estimate the extent to which administrative data held by various entities and departments is utilised at national level. Following an inter-ministerial exercise carried out in Malta in the last couple of years, more than 400 registers were identified. Of these, some are still not available electronically. Every effort must be made to make sure that data in such registers is utilised efficiently to reduce the national costs of collecting data and producing statistics.
One may argue that apart from the challenges presented by inter-operability issues of computer systems, there are other issues related to restrictions imposed by the data protection legislation. Other problems however relate to reluctance from register owners to open access to such data for statistical purpose despite the provisions in the statistical law that cater for such eventuality.
Whilst administrative data is a natural substitute for surveys and questionnaires, the number of surveys that are issued each year and the vast majority of variables included in each, could be drastically reduced if we start making better use of what is readily available.
Many countries are investing heavily in harmonising the data that is collected nationally and promoting data sharing practices and the sooner Malta follows suit, the earlier these unnecessary burdens for businesses will be eliminated, or at least, reduced to more acceptable levels. In the end, a sustainable structure may only be achieved through effective collaboration and communication between all parties.
- 1 Around 50,000 of these refer to the ongoing frontier national survey known as Tourstat which is carried out at the departure lounge of the Malta International Airport for the collection of tourism data.